penal-colony/kafka
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
No description
9 commits 4 branches 0 tags 26 MiB
Find a file
Franz Kafka ebeaeeef21 feat: add CORS and rate limiting middleware 
CORS:
- Configurable allowed origins (wildcard "*" or specific domains)
- Handles OPTIONS preflight with configurable methods, headers, max-age
- Exposed headers support for browser API access
- Env override: CORS_ALLOWED_ORIGINS

Rate Limiting:
- In-memory per-IP sliding window counter
- Configurable request limit and time window
- Background goroutine cleans up stale IP entries
- HTTP 429 with Retry-After header when exceeded
- Extracts real IP from X-Forwarded-For and X-Real-IP (proxy-aware)
- Env overrides: RATE_LIMIT_REQUESTS, RATE_LIMIT_WINDOW, RATE_LIMIT_CLEANUP_INTERVAL
- Set requests=0 in config to disable

Both wired into main.go as middleware chain: rate_limit → cors → handler.
Config example updated with [cors] and [rate_limit] sections.
Full test coverage for both middleware packages.
2026-03-21 15:54:52 +00:00
cmd/searxng-go feat: add CORS and rate limiting middleware 2026-03-21 15:54:52 +00:00
internal feat: add CORS and rate limiting middleware 2026-03-21 15:54:52 +00:00
.gitignore feat: add multi-stage Dockerfile and docker-compose.yml 2026-03-21 15:46:33 +00:00
config.example.toml feat: add CORS and rate limiting middleware 2026-03-21 15:54:52 +00:00
docker-compose.yml feat: add multi-stage Dockerfile and docker-compose.yml 2026-03-21 15:46:33 +00:00
Dockerfile feat: add multi-stage Dockerfile and docker-compose.yml 2026-03-21 15:46:33 +00:00
flake.lock feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
flake.nix feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00
go.mod feat: Valkey cache for search results 2026-03-21 15:43:47 +00:00
go.sum feat: Valkey cache for search results 2026-03-21 15:43:47 +00:00
README.md feat: build Go-based SearXNG-compatible search service 2026-03-20 20:34:08 +01:00

README.md

gosearch (SearXNG rewrite in Go)

This repository contains a standalone Go HTTP service that implements a SearXNG-compatible API-first /search endpoint and proxies unported engines to an upstream SearXNG instance.

Endpoints

  • GET /healthz -> OK
  • GET|POST /search
    • Required form/body parameter: q
    • Optional: format (json | csv | rss; default: json)

Supported format=...

  • json: SearXNG-style JSON response (query, number_of_results, results, answers, corrections, infoboxes, suggestions, unresponsive_engines)
  • csv: CSV with header title,url,content,host,engine,score,type
  • rss: RSS 2.0 feed based on the opensearch_response_rss.xml template fields

Request parameters

The server accepts SearXNG form parameters (both GET query string and POST form-encoded):

  • q (required): search query
  • format (optional): json/csv/rss
  • pageno (optional, default 1): positive integer
  • safesearch (optional, default 0): integer 0..2
  • time_range (optional): day|week|month|year (or omitted/None)
  • timeout_limit (optional): float, seconds (or omitted/None)
  • language (optional, default auto): auto or a BCP-47-ish language code
  • engines (optional): comma-separated engine names (e.g. wikipedia,arxiv)
  • categories / category_<name> (optional): used for selecting the initial ported subset
  • engine_data-<engine>-<key>=<value> (optional): per-engine custom parameters

Environment variables

  • PORT (optional, default 8080)
  • UPSTREAM_SEARXNG_URL (optional for now, but required if you expect unported engines)
    • When set, unported engines are proxied to ${UPSTREAM_SEARXNG_URL}/search with format=json.
  • LOCAL_PORTED_ENGINES (optional, default wikipedia,arxiv,crossref,braveapi,qwant)
    • Controls which engine names are executed locally (Go-native adapters).
  • HTTP_TIMEOUT (optional, default 10s)
    • Timeout for both local engine API calls and upstream proxy calls.
  • Brave Search API:
    • BRAVE_API_KEY (optional): enables the braveapi engine when set
    • BRAVE_ACCESS_TOKEN (optional): if set, requests must include a token (header Authorization: Bearer <token>, X-Search-Token, X-Brave-Access-Token, or form field token)

Ported vs proxied strategy

  1. The service plans which engines should run locally vs upstream using LOCAL_PORTED_ENGINES.
  2. It executes local ported engines using Go-native adapters:
    • wikipedia, arxiv, crossref
  3. Any remaining requested engines are proxied to upstream SearXNG (format=json).
  4. Responses are merged:
    • results are de-duplicated by engine|title|url
    • suggestions/corrections are treated as sets
    • other arrays are concatenated

Running with Nix

This repo uses flake.nix to provide the Go toolchain.

nix develop
go test ./...
go run ./cmd/searxng-go

Example:

export UPSTREAM_SEARXNG_URL="http://127.0.0.1:8888"
export PORT="8080"
nix develop -c go run ./cmd/searxng-go