# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

kafka is a privacy-respecting metasearch engine written in Go. It provides a SearXNG-compatible `/search` API and an HTML frontend (HTMX + Go templates). 9 engines are implemented natively in Go; unlisted engines can be proxied to an upstream SearXNG instance. Responses from multiple engines are merged into a single JSON/CSV/RSS/HTML response.

## Build & Run Commands

```bash
# Enter Nix dev shell (provides Go 1.24 toolchain + curl)
nix develop

# Run all tests
go test ./...

# Run a single test
go test -run TestWikipedia ./internal/engines/

# Run tests in a specific package with verbose output
go test -v ./internal/engines/

# Run the server (requires config.toml)
go run ./cmd/searxng-go -config config.toml
```

There is no Makefile. There is no linter configured.

## Architecture

**Request flow:** HTTP request -> middleware chain (global rate limit -> burst rate limit -> per-IP rate limit -> CORS) -> HTTP handler -> `search.Service` (cache check) -> `engines.Planner` (splits into local vs upstream) -> parallel local engine execution + upstream proxy -> `MergeResponses` -> cache write -> serialize (JSON/CSV/RSS/HTML).

**Key packages:**

- `internal/contracts` — Shared types: `SearchRequest`, `SearchResponse`, `MainResult`, `OutputFormat`. `MainResult` preserves unknown JSON keys from upstream via a `raw map[string]any` field and round-trips them faithfully.
- `internal/config` — TOML-based configuration with env var fallbacks. `Load(path)` reads `config.toml`; env vars override zero-value fields. See `config.example.toml` for all settings.
- `internal/engines` — `Engine` interface and all 9 Go-native implementations. `factory.go` registers engines via `NewDefaultPortedEngines()`. `planner.go` routes engines to local or upstream based on `LOCAL_PORTED_ENGINES` env var.
- `internal/search` — `Service` orchestrates the pipeline: cache check, planning, parallel engine execution via goroutines/WaitGroup, upstream proxying, response merging. Individual engine failures are reported as `unresponsive_engines` rather than aborting the search. Qwant has fallback logic to upstream on empty results.
- `internal/httpapi` — HTTP handlers for `/`, `/search`, `/healthz`, `/opensearch.xml`. Detects HTMX requests via `HX-Request` header to return fragments instead of full pages.
- `internal/upstream` — Client that proxies requests to an upstream SearXNG instance via POST.
- `internal/cache` — Valkey/Redis-backed cache with SHA-256 cache keys. No-op if unconfigured.
- `internal/middleware` — Three rate limiters (per-IP sliding window, burst+sustained, global) and CORS. All disabled by default.
- `internal/views` — HTML templates and static files embedded via `//go:embed`. Renders full pages or HTMX fragments. Templates: `base.html`, `index.html`, `results.html`, `results_inner.html`, `result_item.html`.
- `cmd/searxng-go` — Entry point. Loads TOML config, seeds env vars for engine code, wires up middleware chain, starts HTTP server.

**Engine interface** (`internal/engines/engine.go`):
```go
type Engine interface {
    Name() string
    Search(ctx context.Context, req contracts.SearchRequest) (contracts.SearchResponse, error)
}
```

**Adding a new engine:**
1. Create a new struct implementing the `Engine` interface in `internal/engines/` (single file, e.g., `newengine.go`)
2. Add a test file alongside it (use `roundTripperFunc` and `httpResponse` helpers in `http_mock_test.go` for mocking HTTP)
3. Register it in `NewDefaultPortedEngines()` in `factory.go`
4. Add its name to `defaultPortedEngines` in `planner.go`
5. Add category mappings in `inferFromCategories()` if applicable

## Configuration

Config is loaded from `config.toml` (see `config.example.toml`). All fields can be overridden via environment variables (env vars take precedence over zero-value TOML fields). Key sections: `[server]`, `[upstream]`, `[engines]`, `[cache]`, `[cors]`, `[rate_limit]`, `[global_rate_limit]`, `[burst_rate_limit]`.

## Conventions

- Module path: `github.com/metamorphosis-dev/kafka`
- Tests use shared mock helpers in `internal/engines/http_mock_test.go` (`roundTripperFunc`, `httpResponse`)
- Engine implementations are single files under `internal/engines/` (e.g., `wikipedia.go`, `duckduckgo.go`)
- Response merging de-duplicates by `engine|title|url` key; suggestions/corrections are merged as sets
- `MainResult` uses custom `UnmarshalJSON`/`MarshalJSON` to preserve unknown upstream JSON keys
- HTML templates and static files are embedded at build time via `//go:embed` in `internal/views/`
- Structured logging via `log/slog` with JSON handler