- google.go: use inline (?s) flag instead of regexp.DotAll second arg
- youtube.go: remove Metadata field (not in MainResult contract)
- config_test.go: fix expected engine count from 9 to 11 (google+youtube)
html/template requires template names to be string literals, not field
accesses. Use {{if eq .Template "videos"}} to branch and call the
appropriate template by literal name.
Go html/template doesn't support function calls as template names in
{{template (func .Arg) .}}. Instead, precompute TemplateName in
FromResponse and use {{template .TemplateName .}} in the template.
MainResult: add Thumbnail field (used by YouTube, images, etc.)
video_item.html: new partial for video results with thumbnail display
views.go: add templateForResult func + video_item.html to template parse
results_inner.html: dispatch to video_item when Template="videos"
kafka.css: add .video-result flex layout with thumbnail styling
- Replace document.body.innerHTML with panel.querySelector('.settings-popover-body').innerHTML
- Use theme buttons (.theme-btn) with icons instead of radio buttons
- Use .engine-toggle class for engine checkboxes in 2-column grid
- Include settings-notice paragraph for engine changes
- Use dropdowns for safe search and format with proper ids
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
YouTube Data API v3 engine:
- Add YouTubeConfig to EnginesConfig with api_key field
- Add YOUTUBE_API_KEY env override
- Thread *config.Config through search service to factory
- Factory falls back to env vars if config fields are empty
- Update config.example.toml with youtube section
Also update default local_ported to include google and youtube.
Uses the official YouTube Data API v3. Requires YOUTUBE_API_KEY
environment variable (free from Google Cloud Console).
Returns video results with title, description, channel, publish
date, and thumbnail URL. Falls back gracefully if no API key.
SearXNG approach: use Google Search Appliance (GSA) User-Agent
pool — these are whitelisted enterprise identifiers Google trusts.
Key techniques:
- GSA User-Agent (iPhone OS + GSA/ version) instead of Chrome desktop
- CONSENT=YES+ cookie to bypass EU consent wall
- Parse /url?q= redirector URLs (unquote + strip &sa= params)
- div.MjjYud class for result containers (SearXNG selector)
- data-sncf divs for snippets
- detect sorry.google.com blocks
- Suggestions from ouy7Mc class cards
- Rename cmd/searxng-go to cmd/kafka
- Remove all SearXNG references from source comments while keeping
"SearXNG-compatible API" in user-facing docs
- Update binary paths in README, CLAUDE.md, and Dockerfile
- Update log message to "kafka starting"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Inline JS in base.html: debounced fetch from /autocompleter on keyup
- Keyboard nav: arrows to navigate, Enter to select, Esc to close
- Highlight matching prefix in suggestions
- Click to select and submit
- Dropdown positioned absolutely below search input
- Dark mode compatible via existing CSS variables
Proxies to upstream SearXNG /autocompleter if configured, otherwise
falls back to Wikipedia OpenSearch API. Returns a JSON array of
suggestion strings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vendorHash was empty, causing build failures. The hash was
obtained by running the build once and using the error output.
Tests are skipped because they require network access; CI runs
them via the test workflow.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
duckduckgo, github, reddit, and bing were registered in factory.go
and config.go but missing from planner.go, so they were silently
skipped when LOCAL_PORTED_ENGINES was not set.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
data.forgejo.org is unreachable from the runner. Using https://github.com/
prefix for all action uses to fetch directly from GitHub instead of the
Forgejo marketplace mirror.
Forgejo Actions workflow that builds and pushes Docker images on:
- Push to main → tagged 'latest' + short SHA
- Tags matching 'v*' → semver tags (1.0.0, 1.0, latest)
- PRs → build only (no push)
Pushes to both GHCR and Docker Hub with GitHub Actions cache.
Required secrets:
- DOCKERHUB_USERNAME: Docker Hub username
- DOCKERHUB_PASSWORD: Docker Hub access token
- GITHUB_TOKEN: automatically provided by Forgejo (or set manually for GHCR)
Note: Requires a Forgejo Actions runner to be registered. See:
https://forgejo.org/docs/latest/user/actions/#runner-setup
- Updated to reflect all current features (9 engines, HTMX frontend, Valkey cache, 3-layer rate limiting, CORS, OpenSearch)
- Added quick start for binary, Docker Compose, and NixOS
- Documented all endpoints, API parameters, and response format
- Configuration reference with environment variable table
- Engine table with source and notes
- ASCII architecture diagram
- Docker and NixOS deployment sections
- flake.nix now exports packages, NixOS module, and dev shell
- NixOS module: services.gosearch
- Configurable port, base URL, user/group, state dir
- Creates system user automatically
- Runs as systemd service with auto-restart
- Optional firewall opening
- To deploy on your NixOS VPS:
1. Get the vendor hash: nix build .#packages.x86_64-linux.default (copy the hash)
2. Add to your flake inputs and imports
3. Enable in configuration.nix
- Serve /opensearch.xml with configurable base URL
- Browsers can now add gosearch as a search engine from the address bar
- Configurable via [server] base_url or BASE_URL env var
- XML template embedded in the binary via go:embed
- Added base_url to config.example.toml
DuckDuckGo:
- Fixed parser to handle single-quoted class attributes (class='result-link')
- Decode DDG tracking URLs (uddg= parameter) to extract real URLs
- Match snippet extraction to actual DDG Lite HTML structure (</td> terminator)
Bing:
- Switched from HTML scraping (blocked by JS detection) to RSS endpoint
(?format=rss) which returns parseable XML
- Added JSON API response parsing as fallback
- Returns graceful unresponsive_engines entry when blocked
Live test results:
- DuckDuckGo: 9 results ✅
- GitHub: 10 results (14,768 total) ✅
- Bing: 10 results via RSS ✅
- Reddit: skipped (403 from sandbox, needs browser-like context)
- DuckDuckGo: scrapes Lite HTML endpoint for results
- Language-aware region mapping (de→de-de, ja→jp-jp, etc.)
- HTML parser extracts result links and snippets from DDG Lite markup
- Shared html_helpers.go with extractAttr, stripHTML, htmlUnescape
- GitHub: uses public Search API (repos, sorted by stars)
- No auth required (10 req/min unauthenticated)
- Shows stars, language, topics, last updated date
- Paginated via GitHub's page parameter
- Reddit: uses public JSON search API
- Respects safesearch (skips over_18 posts)
- Shows subreddit, score, comment count
- Links self-posts to the thread URL
- Bing: scrapes web search HTML (b_algo containers)
- Extracts titles, URLs, and snippets from Bing's result markup
- Handles Bing's tracking URL encoding
- Updated factory, config defaults, and config.example.toml
- Full test suite: unit tests for all engines, HTML parsing tests,
region mapping tests, live request tests (skipped in short mode)
9 engines total: wikipedia, arxiv, crossref, braveapi, qwant,
duckduckgo, github, reddit, bing
