kafka/internal/engines
Franz Kafka df8fe9474b feat: add DuckDuckGo, GitHub, Reddit, and Bing engines
- DuckDuckGo: scrapes Lite HTML endpoint for results
  - Language-aware region mapping (de→de-de, ja→jp-jp, etc.)
  - HTML parser extracts result links and snippets from DDG Lite markup
  - Shared html_helpers.go with extractAttr, stripHTML, htmlUnescape

- GitHub: uses public Search API (repos, sorted by stars)
  - No auth required (10 req/min unauthenticated)
  - Shows stars, language, topics, last updated date
  - Paginated via GitHub's page parameter

- Reddit: uses public JSON search API
  - Respects safesearch (skips over_18 posts)
  - Shows subreddit, score, comment count
  - Links self-posts to the thread URL

- Bing: scrapes web search HTML (b_algo containers)
  - Extracts titles, URLs, and snippets from Bing's result markup
  - Handles Bing's tracking URL encoding

- Updated factory, config defaults, and config.example.toml
- Full test suite: unit tests for all engines, HTML parsing tests,
  region mapping tests, live request tests (skipped in short mode)

9 engines total: wikipedia, arxiv, crossref, braveapi, qwant,
duckduckgo, github, reddit, bing
2026-03-21 16:52:11 +00:00
..
arxiv.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
arxiv_test.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
bing.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
bing_test.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
braveapi.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
braveapi_test.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
crossref.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
crossref_test.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
duckduckgo.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
duckduckgo_parse.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
duckduckgo_test.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
engine.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
factory.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
github.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
github_test.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
html_helpers.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
http_mock_test.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
planner.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
qwant.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
qwant_lite_test.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
qwant_test.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
reddit.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
reddit_test.go feat: add DuckDuckGo, GitHub, Reddit, and Bing engines 2026-03-21 16:52:11 +00:00
wikipedia.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
wikipedia_test.go feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00