Commit graph

11 commits

Author SHA1 Message Date
Franz Kafka
41b80a939a feat: add YouTube engine with config file and env support
YouTube Data API v3 engine:
- Add YouTubeConfig to EnginesConfig with api_key field
- Add YOUTUBE_API_KEY env override
- Thread *config.Config through search service to factory
- Factory falls back to env vars if config fields are empty
- Update config.example.toml with youtube section

Also update default local_ported to include google and youtube.
2026-03-22 01:57:13 +00:00
21b77f25bf refactor: remove SearXNG references and rename binary to kafka
Some checks failed
Build and Push Docker Image / build-and-push (push) Failing after 8s
Mirror to GitHub / mirror (push) Failing after 3s
Tests / test (push) Successful in 38s
- Rename cmd/searxng-go to cmd/kafka
- Remove all SearXNG references from source comments while keeping
  "SearXNG-compatible API" in user-facing docs
- Update binary paths in README, CLAUDE.md, and Dockerfile
- Update log message to "kafka starting"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:47:03 +01:00
9b280ad606 feat: add /autocompleter endpoint for search suggestions
Some checks failed
Mirror to GitHub / mirror (push) Waiting to run
Tests / test (push) Waiting to run
Build and Push Docker Image / build-and-push (push) Has been cancelled
Proxies to upstream SearXNG /autocompleter if configured, otherwise
falls back to Wikipedia OpenSearch API. Returns a JSON array of
suggestion strings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:06:25 +01:00
6346fb7155 chore: update Go module path to github.com/metamorphosis-dev/kafka
Module path now matches the GitHub mirror location.
All internal imports updated across 35+ files.
2026-03-21 19:42:01 +00:00
13040268d6 feat: add global and burst rate limiters
Three layers of rate limiting, all disabled by default, opt-in via config:

1. Per-IP (existing): 30 req/min per IP
2. Global: server-wide limit across all IPs
   - Lock-free atomic counter for minimal overhead
   - Returns 503 when exceeded
   - Prevents pool exhaustion from distributed attacks
3. Burst: per-IP burst + sustained windows
   - Blocks rapid-fire abuse within seconds
   - Returns 429 with X-RateLimit-Reason header
   - Example: 5 req/5s burst, 60 req/min sustained

Config:
[global_rate_limit]
requests = 0  # disabled by default
window = "1m"

[burst_rate_limit]
burst = 0  # disabled by default
burst_window = "5s"
sustained = 0
sustained_window = "1m"

Env overrides: GLOBAL_RATE_LIMIT_REQUESTS, GLOBAL_RATE_LIMIT_WINDOW,
BURST_RATE_LIMIT_BURST, BURST_RATE_LIMIT_BURST_WINDOW,
BURST_RATE_LIMIT_SUSTAINED, BURST_RATE_LIMIT_SUSTAINED_WINDOW

Full test coverage: concurrent lock-free test, window expiry, disabled states,
IP isolation, burst vs sustained distinction.
2026-03-21 18:35:31 +00:00
4ec600f6c0 feat: add OpenSearch XML endpoint
- Serve /opensearch.xml with configurable base URL
- Browsers can now add gosearch as a search engine from the address bar
- Configurable via [server] base_url or BASE_URL env var
- XML template embedded in the binary via go:embed
- Added base_url to config.example.toml
2026-03-21 17:40:05 +00:00
28b61ff251 feat: HTMX + Go Templates HTML frontend
- Add internal/views/ package with embedded templates and static files
- Go html/template with SearXNG-compatible CSS class names
- Dark mode via prefers-color-scheme, responsive layout, print styles
- HTMX integration:
  - Debounced instant search (500ms) on the search input
  - Form submission targets #results via hx-post
  - Pagination buttons are HTMX-powered (swap results div only)
  - HX-Request header detection for fragment vs full page rendering
- Template structure:
  - base.html: full page layout with HTMX script, favicon, CSS
  - index.html: homepage with centered search box
  - results.html: full results page (wraps base + results_inner)
  - results_inner.html: results fragment (HTMX partial + sidebar + pagination)
  - result_item.html: reusable result article partial
- Smart format detection: browser requests (Accept: text/html) default to HTML,
  API clients default to JSON
- Static files served at /static/ from embedded FS (CSS, favicon SVG)
- Index route at GET /
- Empty query on HTML format redirects to homepage
- Custom CSS (gosearch.css): clean, minimal, privacy-respecting aesthetic
  with light/dark mode, responsive breakpoints, print stylesheet
- Add views package tests
2026-03-21 16:10:42 +00:00
ebeaeeef21 feat: add CORS and rate limiting middleware
CORS:
- Configurable allowed origins (wildcard "*" or specific domains)
- Handles OPTIONS preflight with configurable methods, headers, max-age
- Exposed headers support for browser API access
- Env override: CORS_ALLOWED_ORIGINS

Rate Limiting:
- In-memory per-IP sliding window counter
- Configurable request limit and time window
- Background goroutine cleans up stale IP entries
- HTTP 429 with Retry-After header when exceeded
- Extracts real IP from X-Forwarded-For and X-Real-IP (proxy-aware)
- Env overrides: RATE_LIMIT_REQUESTS, RATE_LIMIT_WINDOW, RATE_LIMIT_CLEANUP_INTERVAL
- Set requests=0 in config to disable

Both wired into main.go as middleware chain: rate_limit → cors → handler.
Config example updated with [cors] and [rate_limit] sections.
Full test coverage for both middleware packages.
2026-03-21 15:54:52 +00:00
94322ceff4 feat: Valkey cache for search results
- Add internal/cache package using go-redis/v9 (Valkey-compatible)
- Cache keys are deterministic SHA-256 hashes of search parameters
- Cache wraps the Search() method: check cache → miss → execute → store
- Gracefully disabled if Valkey is unreachable or unconfigured
- Configurable TTL (default 5m), address, password, and DB index
- Environment variable overrides: VALKEY_ADDRESS, VALKEY_PASSWORD,
  VALKEY_DB, VALKEY_CACHE_TTL
- Structured JSON logging via slog throughout cache layer
- Refactored service.go: extract executeSearch() from Search() for clarity
- Update config.example.toml with [cache] section
- Add cache package tests (key generation, nop behavior)
2026-03-21 15:43:47 +00:00
8649864971 feat: migrate from env vars to config.toml
- Add internal/config package with TOML parsing (BurntSushi/toml)
- Create config.example.toml documenting all settings
- Update main.go to load config via -config flag (default: config.toml)
- Environment variables remain as fallback overrides for backward compat
- Config file values are used as defaults; env vars override when set
- Add comprehensive tests for file loading, defaults, and env overrides
- Add config.toml to .gitignore (secrets stay local)
2026-03-21 14:56:00 +00:00
dc44837219 feat: build Go-based SearXNG-compatible search service
Implement an API-first Go rewrite with local engine adapters, upstream fallback, and Nix-based tooling so searches can run without matching the original UI while preserving response compatibility.

Made-with: Cursor
2026-03-20 20:34:08 +01:00