{{.Content}}
+{{.SafeContent}}
{{end}}diff --git a/.forgejo/workflows/test.yml b/.forgejo/workflows/test.yml index bd05693..5f7efb4 100644 --- a/.forgejo/workflows/test.yml +++ b/.forgejo/workflows/test.yml @@ -11,12 +11,15 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: https://github.com/actions/checkout@v4 + uses: https://github.com/actions/checkout@v5 - name: Set up Go uses: https://github.com/actions/setup-go@v5 with: go-version-file: go.mod + - name: Clean vendor + run: rm -rf vendor + - name: Test run: go test -race -v ./... diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml new file mode 100644 index 0000000..47cc920 --- /dev/null +++ b/.github/workflows/test.yml @@ -0,0 +1,25 @@ +name: Tests + +on: + pull_request: + branches: [main] + push: + branches: [main] + +jobs: + test: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v5 + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version-file: go.mod + + - name: Clean vendor + run: rm -rf vendor + + - name: Test + run: go test -race -v ./... diff --git a/.gitignore b/.gitignore index a5388c7..6cea500 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,11 @@ node_modules/ .agent/ +internal/spa/dist/ +frontend/node_modules/ +frontend/dist/ +frontend/bun.lock +frontend/bun.lockb +frontend/package-lock.json *.exe *.exe~ *.dll diff --git a/CLAUDE.md b/CLAUDE.md index b7f254e..e136dd7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview -kafka is a privacy-respecting metasearch engine written in Go. It provides a SearXNG-compatible `/search` API and an HTML frontend (HTMX + Go templates). 9 engines are implemented natively in Go; unlisted engines can be proxied to an upstream metasearch instance. Responses from multiple engines are merged into a single JSON/CSV/RSS/HTML response. +samsa is a privacy-respecting metasearch engine written in Go. It provides a SearXNG-compatible `/search` API and an HTML frontend (HTMX + Go templates). 9 engines are implemented natively in Go; unlisted engines can be proxied to an upstream metasearch instance. Responses from multiple engines are merged into a single JSON/CSV/RSS/HTML response. ## Build & Run Commands @@ -22,7 +22,7 @@ go test -run TestWikipedia ./internal/engines/ go test -v ./internal/engines/ # Run the server (requires config.toml) -go run ./cmd/kafka -config config.toml +go run ./cmd/samsa -config config.toml ``` There is no Makefile. There is no linter configured. @@ -43,7 +43,7 @@ There is no Makefile. There is no linter configured. - `internal/cache` — Valkey/Redis-backed cache with SHA-256 cache keys. No-op if unconfigured. - `internal/middleware` — Three rate limiters (per-IP sliding window, burst+sustained, global) and CORS. All disabled by default. - `internal/views` — HTML templates and static files embedded via `//go:embed`. Renders full pages or HTMX fragments. Templates: `base.html`, `index.html`, `results.html`, `results_inner.html`, `result_item.html`. -- `cmd/kafka` — Entry point. Loads TOML config, seeds env vars for engine code, wires up middleware chain, starts HTTP server. +- `cmd/samsa` — Entry point. Loads TOML config, seeds env vars for engine code, wires up middleware chain, starts HTTP server. **Engine interface** (`internal/engines/engine.go`): ```go @@ -66,7 +66,7 @@ Config is loaded from `config.toml` (see `config.example.toml`). All fields can ## Conventions -- Module path: `github.com/metamorphosis-dev/kafka` +- Module path: `github.com/metamorphosis-dev/samsa` - Tests use shared mock helpers in `internal/engines/http_mock_test.go` (`roundTripperFunc`, `httpResponse`) - Engine implementations are single files under `internal/engines/` (e.g., `wikipedia.go`, `duckduckgo.go`) - Response merging de-duplicates by `engine|title|url` key; suggestions/corrections are merged as sets diff --git a/Dockerfile b/Dockerfile index e21960f..9f3443f 100644 --- a/Dockerfile +++ b/Dockerfile @@ -21,7 +21,7 @@ RUN apk add --no-cache ca-certificates tzdata COPY --from=builder /kafka /usr/local/bin/kafka COPY config.example.toml /etc/kafka/config.example.toml -EXPOSE 8080 +EXPOSE 5355 ENTRYPOINT ["kafka"] CMD ["-config", "/etc/kafka/config.toml"] diff --git a/README.md b/README.md index c03019e..7427922 100644 --- a/README.md +++ b/README.md @@ -1,20 +1,23 @@ -# kafka +# samsa + +*samsa — named for Gregor Samsa, who woke to find himself transformed. You wanted results; you got a metasearch engine.* A privacy-respecting, open metasearch engine written in Go. SearXNG-compatible API with an HTML frontend, designed to be fast, lightweight, and deployable anywhere. -**9 engines. No JavaScript. No tracking. One binary.** +**11 engines. No JavaScript required. No tracking. One binary.** ## Features - **SearXNG-compatible API** — drop-in replacement for existing integrations -- **9 search engines** — Wikipedia, arXiv, Crossref, Brave, Qwant, DuckDuckGo, GitHub, Reddit, Bing -- **HTML frontend** — HTMX + Go templates with instant search, dark mode, responsive design +- **11 search engines** — Wikipedia, arXiv, Crossref, Brave Search API, Brave (scraping), Qwant, DuckDuckGo, GitHub, Reddit, Bing, Google, YouTube +- **Stack Overflow** — bonus engine, not enabled by default +- **HTML frontend** — Go templates + HTMX with instant search, dark mode, responsive design - **Valkey cache** — optional Redis-compatible caching with configurable TTL - **Rate limiting** — three layers: per-IP, burst, and global (all disabled by default) - **CORS** — configurable origins for browser-based clients -- **OpenSearch** — browsers can add kafka as a search engine from the address bar +- **OpenSearch** — browsers can add samsa as a search engine from the address bar - **Graceful degradation** — individual engine failures don't kill the whole search -- **Docker** — multi-stage build, ~20MB runtime image +- **Docker** — multi-stage build, static binary, ~20MB runtime image - **NixOS** — native NixOS module with systemd service ## Quick Start @@ -22,17 +25,17 @@ A privacy-respecting, open metasearch engine written in Go. SearXNG-compatible A ### Binary ```bash -git clone https://git.ashisgreat.xyz/penal-colony/gosearch.git -cd kafka -go build ./cmd/kafka -./kafka -config config.toml +git clone https://git.ashisgreat.xyz/penal-colony/samsa.git +cd samsa +go build ./cmd/samsa +./samsa -config config.toml ``` ### Docker Compose ```bash cp config.example.toml config.toml -# Edit config.toml — set your Brave API key, etc. +# Edit config.toml — set your Brave API key, YouTube API key, etc. docker compose up -d ``` @@ -41,28 +44,28 @@ docker compose up -d Add to your flake inputs: ```nix -inputs.kafka.url = "git+https://git.ashisgreat.xyz/penal-colony/gosearch.git"; +inputs.samsa.url = "git+https://git.ashisgreat.xyz/penal-colony/samsa.git"; ``` Enable in your configuration: ```nix -imports = [ inputs.kafka.nixosModules.default ]; +imports = [ inputs.samsa.nixosModules.default ]; -services.kafka = { +services.samsa = { enable = true; openFirewall = true; baseUrl = "https://search.example.com"; - # config = "/etc/kafka/config.toml"; # default + # config = "/etc/samsa/config.toml"; # default }; ``` Write your config: ```bash -sudo mkdir -p /etc/kafka -sudo cp config.example.toml /etc/kafka/config.toml -sudo $EDITOR /etc/kafka/config.toml +sudo mkdir -p /etc/samsa +sudo cp config.example.toml /etc/samsa/config.toml +sudo $EDITOR /etc/samsa/config.toml ``` Deploy: @@ -76,7 +79,7 @@ sudo nixos-rebuild switch --flake .# ```bash nix develop go test ./... -go run ./cmd/kafka -config config.toml +go run ./cmd/samsa -config config.toml ``` ## Endpoints @@ -107,7 +110,7 @@ go run ./cmd/kafka -config config.toml ### Example ```bash -curl "http://localhost:8080/search?q=golang&format=json&engines=github,duckduckgo" +curl "http://localhost:5355/search?q=golang&format=json&engines=github,duckduckgo" ``` ### Response (JSON) @@ -140,6 +143,8 @@ Copy `config.example.toml` to `config.toml` and edit. All settings can also be o - **`[server]`** — port, timeout, public base URL for OpenSearch - **`[upstream]`** — optional upstream metasearch proxy for unported engines - **`[engines]`** — which engines run locally, engine-specific settings +- **`[engines.brave]`** — Brave Search API key +- **`[engines.youtube]`** — YouTube Data API v3 key - **`[cache]`** — Valkey/Redis address, password, TTL - **`[cors]`** — allowed origins and methods - **`[rate_limit]`** — per-IP sliding window (30 req/min default) @@ -150,13 +155,14 @@ Copy `config.example.toml` to `config.toml` and edit. All settings can also be o | Variable | Description | |---|---| -| `PORT` | Listen port (default: 8080) | +| `PORT` | Listen port (default: 5355) | | `BASE_URL` | Public URL for OpenSearch XML | | `UPSTREAM_SEARXNG_URL` | Upstream instance URL | | `LOCAL_PORTED_ENGINES` | Comma-separated local engine list | | `HTTP_TIMEOUT` | Upstream request timeout | | `BRAVE_API_KEY` | Brave Search API key | | `BRAVE_ACCESS_TOKEN` | Gate requests with token | +| `YOUTUBE_API_KEY` | YouTube Data API v3 key | | `VALKEY_ADDRESS` | Valkey/Redis address | | `VALKEY_PASSWORD` | Valkey/Redis password | | `VALKEY_CACHE_TTL` | Cache TTL | @@ -170,55 +176,64 @@ See `config.example.toml` for the full list including rate limiting and CORS var | Wikipedia | MediaWiki API | General knowledge | | arXiv | arXiv API | Academic papers | | Crossref | Crossref API | Academic metadata | -| Brave | Brave Search API | General web (requires API key) | +| Brave Search API | Brave API | General web (requires API key) | +| Brave | Brave Lite HTML | General web (no key needed) | | Qwant | Qwant Lite HTML | General web | | DuckDuckGo | DDG Lite HTML | General web | | GitHub | GitHub Search API v3 | Code and repositories | | Reddit | Reddit JSON API | Discussions | | Bing | Bing RSS | General web | +| Google | GSA User-Agent scraping | General web (no API key) | +| YouTube | YouTube Data API v3 | Videos (requires API key) | +| Stack Overflow | Stack Exchange API | Q&A (registered, not enabled by default) | Engines not listed in `engines.local_ported` are proxied to an upstream metasearch instance if `upstream.url` is configured. +### API Keys + +Brave Search API and YouTube Data API require keys. If omitted, those engines are silently skipped. Brave Lite (scraping) and Google (GSA UA scraping) work without keys. + ## Architecture ``` -┌─────────────────────────────────────┐ -│ HTTP Handler │ -│ /search / /opensearch.xml │ -├─────────────────────────────────────┤ -│ Middleware Chain │ -│ Global → Burst → Per-IP → CORS │ -├─────────────────────────────────────┤ -│ Search Service │ -│ Parallel engine execution │ -│ WaitGroup + graceful degradation │ -├─────────────────────────────────────┤ -│ Cache Layer │ -│ Valkey/Redis (optional, no-op if │ -│ unconfigured) │ -├─────────────────────────────────────┤ -│ Engines (×9) │ -│ Each runs in its own goroutine │ -│ Failures → unresponsive_engines │ -└─────────────────────────────────────┘ +┌───────────────────────────────────────┐ +│ HTTP Handler │ +│ /search / /opensearch.xml │ +├───────────────────────────────────────┤ +│ Middleware Chain │ +│ Global → Burst → Per-IP → CORS │ +├───────────────────────────────────────┤ +│ Search Service │ +│ Parallel engine execution │ +│ WaitGroup + graceful degradation │ +├───────────────────────────────────────┤ +│ Cache Layer │ +│ Valkey/Redis (optional; no-op if │ +│ unconfigured) │ +├───────────────────────────────────────┤ +│ Engines (×11 default) │ +│ Each runs in its own goroutine │ +│ Failures → unresponsive_engines │ +└───────────────────────────────────────┘ ``` ## Docker -The Dockerfile uses a multi-stage build: - -```dockerfile -# Build stage: golang:1.24-alpine -# Runtime stage: alpine:3.21 (~20MB) -# CGO_ENABLED=0 — static binary -``` +The Dockerfile uses a multi-stage build with a static Go binary on alpine Linux: ```bash +# Build: golang:1.24-alpine +# Runtime: alpine:3.21 (~20MB) +# CGO_ENABLED=0 — fully static docker compose up -d ``` Includes Valkey 8 with health checks out of the box. +## Contributing + +See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for a walkthrough of adding a new engine. The interface is two methods: `Name()` and `Search(context, request)`. + ## License -MIT +[AGPLv3](https://www.gnu.org/licenses/agpl-3.0.html) diff --git a/cmd/kafka/main.go b/cmd/samsa/main.go similarity index 82% rename from cmd/kafka/main.go rename to cmd/samsa/main.go index cdc81b5..199033b 100644 --- a/cmd/kafka/main.go +++ b/cmd/samsa/main.go @@ -1,4 +1,4 @@ -// kafka — a privacy-respecting metasearch engine +// samsa — a privacy-respecting metasearch engine // Copyright (C) 2026-present metamorphosis-dev // // This program is free software: you can redistribute it and/or modify @@ -25,13 +25,13 @@ import ( "net/http" "os" - "github.com/metamorphosis-dev/kafka/internal/autocomplete" - "github.com/metamorphosis-dev/kafka/internal/cache" - "github.com/metamorphosis-dev/kafka/internal/config" - "github.com/metamorphosis-dev/kafka/internal/httpapi" - "github.com/metamorphosis-dev/kafka/internal/middleware" - "github.com/metamorphosis-dev/kafka/internal/search" - "github.com/metamorphosis-dev/kafka/internal/views" + "github.com/metamorphosis-dev/samsa/internal/autocomplete" + "github.com/metamorphosis-dev/samsa/internal/cache" + "github.com/metamorphosis-dev/samsa/internal/config" + "github.com/metamorphosis-dev/samsa/internal/httpapi" + "github.com/metamorphosis-dev/samsa/internal/middleware" + "github.com/metamorphosis-dev/samsa/internal/search" + "github.com/metamorphosis-dev/samsa/internal/views" ) func main() { @@ -77,14 +77,20 @@ func main() { acSvc := autocomplete.NewService(cfg.Upstream.URL, cfg.HTTPTimeout()) - h := httpapi.NewHandler(svc, acSvc.Suggestions, cfg.Server.SourceURL) + h := httpapi.NewHandler(svc, acSvc.Suggestions, cfg.Server.SourceURL, searchCache) mux := http.NewServeMux() + + // HTML template routes mux.HandleFunc("/", h.Index) - mux.HandleFunc("/healthz", h.Healthz) mux.HandleFunc("/search", h.Search) + mux.HandleFunc("/preferences", h.Preferences) + + // API routes + mux.HandleFunc("/healthz", h.Healthz) mux.HandleFunc("/autocompleter", h.Autocompleter) mux.HandleFunc("/opensearch.xml", h.OpenSearch(cfg.Server.BaseURL)) + mux.HandleFunc("/favicon/", h.Favicon) // Serve embedded static files (CSS, JS, images). staticFS, err := views.StaticFS() @@ -94,8 +100,9 @@ func main() { var subFS fs.FS = staticFS mux.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.FS(subFS)))) - // Apply middleware: global rate limit → burst rate limit → per-IP rate limit → CORS → handler. + // Apply middleware: global rate limit → burst rate limit → per-IP rate limit → CORS → security headers → handler. var handler http.Handler = mux + handler = middleware.SecurityHeaders(middleware.SecurityHeadersConfig{})(handler) handler = middleware.CORS(middleware.CORSConfig{ AllowedOrigins: cfg.CORS.AllowedOrigins, AllowedMethods: cfg.CORS.AllowedMethods, @@ -107,6 +114,7 @@ func main() { Requests: cfg.RateLimit.Requests, Window: cfg.RateLimitWindow(), CleanupInterval: cfg.RateLimitCleanupInterval(), + TrustedProxies: cfg.RateLimit.TrustedProxies, }, logger)(handler) handler = middleware.GlobalRateLimit(middleware.GlobalRateLimitConfig{ Requests: cfg.GlobalRateLimit.Requests, @@ -120,7 +128,7 @@ func main() { }, logger)(handler) addr := fmt.Sprintf(":%d", cfg.Server.Port) - logger.Info("kafka starting", + logger.Info("samsa starting", "addr", addr, "cache", searchCache.Enabled(), "rate_limit", cfg.RateLimit.Requests > 0, diff --git a/config.example.toml b/config.example.toml index 042bb63..7fed53e 100644 --- a/config.example.toml +++ b/config.example.toml @@ -1,22 +1,22 @@ -# kafka configuration +# samsa configuration # Copy to config.toml and adjust as needed. # Environment variables are used as fallbacks when a config field is empty/unset. [server] # Listen port (env: PORT) -port = 8080 +port = 5355 # HTTP timeout for engine and upstream calls (env: HTTP_TIMEOUT) http_timeout = "10s" # Public base URL for OpenSearch XML (env: BASE_URL) -# Set this so browsers can add kafka as a search engine. +# Set this so browsers can add samsa as a search engine. # Example: "https://search.example.com" base_url = "" # Link to the source code (shown in footer as "Source" link) -# Defaults to the upstream kafka repo if not set. -# Example: "https://git.example.com/my-kafka-fork" +# Defaults to the upstream samsa repo if not set. +# Example: "https://git.example.com/my-samsa-fork" source_url = "" [upstream] @@ -27,7 +27,8 @@ url = "" [engines] # Comma-separated list of engines to execute locally in Go (env: LOCAL_PORTED_ENGINES) # Engines not listed here will be proxied to the upstream instance. -local_ported = ["wikipedia", "arxiv", "crossref", "braveapi", "qwant", "duckduckgo", "github", "reddit", "bing", "google", "youtube"] +# Include bing_images, ddg_images, qwant_images for image search when [upstream].url is empty. +local_ported = ["wikipedia", "wikidata", "arxiv", "crossref", "braveapi", "qwant", "duckduckgo", "github", "reddit", "bing", "google", "youtube", "bing_images", "ddg_images", "qwant_images"] [engines.brave] # Brave Search API key (env: BRAVE_API_KEY) @@ -56,6 +57,12 @@ db = 0 # Cache TTL for search results (env: VALKEY_CACHE_TTL) default_ttl = "5m" +[cache.ttl_overrides] +# Per-engine TTL overrides (uncomment to use): +# wikipedia = "48h" +# reddit = "15m" +# braveapi = "2h" + [cors] # CORS configuration for browser-based clients. # Allowed origins: use "*" for all, or specific domains (env: CORS_ALLOWED_ORIGINS) diff --git a/docker-compose.yml b/docker-compose.yml index 98eae96..538713f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -8,7 +8,7 @@ services: kafka: build: . ports: - - "8080:8080" + - "5355:5355" volumes: - ./config.toml:/etc/kafka/config.toml:ro depends_on: diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md new file mode 100644 index 0000000..bada2e2 --- /dev/null +++ b/docs/CONTRIBUTING.md @@ -0,0 +1,218 @@ +# Contributing — Adding a New Engine + +This guide walks through adding a new search engine to samsa. The minimal engine needs only an HTTP client, a query, and a result parser. + +--- + +## 1. Create the engine file + +Place it in `internal/engines/`: + +``` +internal/engines/ + myengine.go ← your engine + myengine_test.go ← tests (required) +``` + +Name the struct after the engine, e.g. `WolframEngine` for "wolfram". The `Name()` method returns the engine key used throughout samsa. + +## 2. Implement the Engine interface + +```go +package engines + +import ( + "context" + "github.com/metamorphosis-dev/samsa/internal/contracts" +) + +type MyEngine struct { + client *http.Client +} + +func (e *MyEngine) Name() string { return "myengine" } + +func (e *MyEngine) Search(ctx context.Context, req contracts.SearchRequest) (contracts.SearchResponse, error) { + // ... +} +``` + +### The SearchRequest fields you'll use most: + +| Field | Type | Description | +|-------|------|-------------| +| `Query` | `string` | The search query | +| `Pageno` | `int` | Current page number (1-based) | +| `Safesearch` | `int` | 0=off, 1=moderate, 2=strict | +| `Language` | `string` | ISO language code (e.g. `"en"`) | + +### The SearchResponse to return: + +```go +contracts.SearchResponse{ + Query: req.Query, + NumberOfResults: len(results), + Results: results, // []MainResult + Answers: []map[string]any{}, + Corrections: []string{}, + Infoboxes: []map[string]any{}, + Suggestions: []string{}, + UnresponsiveEngines: [][2]string{}, +} +``` + +### Empty query — return early: + +```go +if strings.TrimSpace(req.Query) == "" { + return contracts.SearchResponse{Query: req.Query}, nil +} +``` + +### Engine unavailable / error — graceful degradation: + +```go +// Rate limited or blocked +return contracts.SearchResponse{ + Query: req.Query, + UnresponsiveEngines: [][2]string{{"myengine", "reason"}}, + Results: []contracts.MainResult{}, + // ... empty other fields +}, nil + +// Hard error — return it +return contracts.SearchResponse{}, fmt.Errorf("myengine upstream error: status %d", resp.StatusCode) +``` + +## 3. Build the result + +```go +urlPtr := "https://example.com/result" +result := contracts.MainResult{ + Title: "Result Title", + Content: "Snippet or description text", + URL: &urlPtr, // pointer to string, required + Engine: "myengine", + Category: "general", // or "it", "science", "videos", "images", "social media" + Score: 0, // used for relevance ranking during merge + Engines: []string{"myengine"}, +} +``` + +### Template field + +The template system checks for `"videos"` and `"images"`. Everything else renders via `result_item.html`. Set `Template` only if you have a custom template; omit it for the default result card. + +### Category field + +Controls which category tab the result appears under and which engines are triggered: + +| Category | Engines used | +|----------|-------------| +| `general` | google, bing, ddg, brave, braveapi, qwant, wikipedia | +| `it` | github, stackoverflow | +| `science` | arxiv, crossref | +| `videos` | youtube | +| `images` | bing_images, ddg_images, qwant_images | +| `social media` | reddit | + +## 4. Wire it into the factory + +In `internal/engines/factory.go`, add your engine to the map returned by `NewDefaultPortedEngines`: + +```go +"myengine": &MyEngine{client: client}, +``` + +If your engine needs an API key, read it from config or the environment (see `braveapi` or `youtube` in factory.go for the pattern). + +## 5. Register defaults + +In `internal/engines/planner.go`: + +**Add to `defaultPortedEngines`:** +```go +var defaultPortedEngines = []string{ + // ... existing ... + "myengine", +} +``` + +**Add to category mapping in `inferFromCategories`** (if applicable): +```go +case "general": + set["myengine"] = true +``` + +**Update the sort order map** so results maintain consistent ordering: +```go +order := map[string]int{ + // ... existing ... + "myengine": N, // pick a slot +} +``` + +## 6. Add tests + +At minimum, test: +- `Name()` returns the correct string +- Nil engine returns an error +- Empty query returns zero results +- Successful API response parses correctly +- Rate limit / error cases return `UnresponsiveEngines` with a reason + +Use `httptest.NewServer` to mock the upstream API. See `arxiv_test.go` or `reddit_test.go` for examples. + +## 7. Build and test + +```bash +go build ./... +go test ./internal/engines/ -run MyEngine -v +go test ./... +``` + +## Example: Adding an RSS-based engine + +If the engine provides an RSS feed, the parsing is straightforward: + +```go +type rssItem struct { + Title string `xml:"title"` + Link string `xml:"link"` + Description string `xml:"description"` +} + +type rssFeed struct { + Channel struct { + Items []rssItem `xml:"item"` + } `xml:"channel"` +} + +dec := xml.NewDecoder(resp.Body) +var feed rssFeed +dec.Decode(&feed) + +for _, item := range feed.Channel.Items { + urlPtr := item.Link + results = append(results, contracts.MainResult{ + Title: item.Title, + Content: stripHTML(item.Description), + URL: &urlPtr, + Engine: "myengine", + // ... + }) +} +``` + +## Checklist + +- [ ] Engine file created in `internal/engines/` +- [ ] `Engine` interface implemented (`Name()` + `Search()`) +- [ ] Empty query handled (return early, no error) +- [ ] Graceful degradation for errors and rate limits +- [ ] Results use `Category` to group with related engines +- [ ] Factory updated with new engine +- [ ] Planner updated (defaults + category mapping + sort order) +- [ ] Tests written covering main paths +- [ ] `go build ./...` succeeds +- [ ] `go test ./...` passes diff --git a/docs/superpowers/plans/2026-03-22-settings-ui.md b/docs/superpowers/plans/2026-03-22-settings-ui.md deleted file mode 100644 index cf34df5..0000000 --- a/docs/superpowers/plans/2026-03-22-settings-ui.md +++ /dev/null @@ -1,747 +0,0 @@ -# Settings UI Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** A preferences popover panel (top-right on desktop, bottom sheet on mobile) that lets users set theme, enabled engines, safe search, and default format. All changes auto-save to `localStorage` and apply immediately to the DOM. - -**Architecture:** Pure client-side JS + CSS added alongside existing templates. No Go changes. Settings persist via `localStorage` key `kafka_prefs`. Theme applies via `data-theme` attribute on ``. - -**Tech Stack:** Vanilla JS (no framework), existing `kafka.css` custom properties, HTMX for search. - ---- - -## File Map - -| Action | File | -|--------|------| -| Create | `internal/views/static/js/settings.js` | -| Modify | `internal/views/static/css/kafka.css` | -| Modify | `internal/views/templates/base.html` | -| Modify | `internal/views/templates/index.html` | -| Modify | `internal/views/templates/results.html` | -| Modify | `internal/views/views.go` | - -**Key insight on engine preferences:** `ParseSearchRequest` reads `engines` as a CSV form value (`r.FormValue("engines")`). The search forms in `index.html` and `results.html` will get a hidden `#engines-input` field that is kept in sync with localStorage. On submit, the engines preference is sent as a normal form field. HTMX `hx-include="this"` already includes the form element, so the hidden input is automatically included in the request. - ---- - -## Task 1: CSS — Popover, toggles, bottom sheet - -**Files:** -- Modify: `internal/views/static/css/kafka.css` - -- [ ] **Step 1: Add CSS for popover, triggers, toggles, bottom sheet** - -Append the following to `kafka.css`: - -```css -/* ============================================ - Settings Panel - ============================================ */ - -/* Header */ -.site-header { - display: flex; - align-items: center; - justify-content: space-between; - padding: 0.6rem 1rem; - background: var(--color-header-background); - border-bottom: 1px solid var(--color-header-border); -} -.site-title { - font-size: 1rem; - font-weight: 600; - color: var(--color-base-font); -} - -/* Gear trigger button */ -.settings-trigger { - background: none; - border: none; - font-size: 1.1rem; - cursor: pointer; - padding: 0.3rem 0.5rem; - border-radius: var(--radius); - color: var(--color-base-font); - opacity: 0.7; - transition: opacity 0.2s, background 0.2s; - line-height: 1; -} -.settings-trigger:hover, -.settings-trigger[aria-expanded="true"] { - opacity: 1; - background: var(--color-sidebar-background); -} - -/* Popover panel */ -.settings-popover { - position: absolute; - top: 100%; - right: 0; - width: 280px; - max-height: 420px; - overflow-y: auto; - background: var(--color-base-background); - border: 1px solid var(--color-sidebar-border); - border-radius: var(--radius); - box-shadow: 0 8px 24px rgba(0, 0, 0, 0.12); - z-index: 200; - display: none; - flex-direction: column; -} -.settings-popover[data-open="true"] { - display: flex; - animation: settings-slide-in 0.2s ease; -} -@keyframes settings-slide-in { - from { opacity: 0; transform: translateY(-8px); } - to { opacity: 1; transform: translateY(0); } -} - -.settings-popover-header { - display: flex; - justify-content: space-between; - align-items: center; - padding: 0.75rem 1rem; - border-bottom: 1px solid var(--color-sidebar-border); - font-weight: 600; - font-size: 0.9rem; - flex-shrink: 0; -} -.settings-popover-close { - background: none; - border: none; - font-size: 1.2rem; - cursor: pointer; - color: var(--color-base-font); - opacity: 0.6; - padding: 0 0.25rem; - line-height: 1; -} -.settings-popover-close:hover { opacity: 1; } - -.settings-popover-body { - padding: 0.8rem; - display: flex; - flex-direction: column; - gap: 1rem; -} - -.settings-section-title { - font-size: 0.7rem; - font-weight: 600; - text-transform: uppercase; - letter-spacing: 0.05em; - color: var(--color-suggestion); - margin-bottom: 0.5rem; -} - -/* Theme buttons */ -.theme-buttons { - display: flex; - gap: 0.4rem; -} -.theme-btn { - flex: 1; - padding: 0.35rem 0.5rem; - border: 1px solid var(--color-sidebar-border); - border-radius: var(--radius); - background: var(--color-btn-background); - color: var(--color-base-font); - cursor: pointer; - font-size: 0.75rem; - text-align: center; - transition: background 0.15s, border-color 0.15s; -} -.theme-btn:hover { background: var(--color-btn-hover); } -.theme-btn.active { - background: var(--color-link); - color: #fff; - border-color: var(--color-link); -} - -/* Engine toggles — 2-column grid */ -.engine-grid { - display: grid; - grid-template-columns: 1fr 1fr; - gap: 0.4rem; -} -.engine-toggle { - display: flex; - align-items: center; - gap: 0.4rem; - padding: 0.3rem 0.5rem; - border-radius: var(--radius); - background: var(--color-sidebar-background); - font-size: 0.78rem; - cursor: pointer; -} -.engine-toggle input[type="checkbox"] { - width: 15px; - height: 15px; - margin: 0; - cursor: pointer; - accent-color: var(--color-link); -} -.engine-toggle span { - flex: 1; - min-width: 0; - overflow: hidden; - text-overflow: ellipsis; - white-space: nowrap; -} - -/* Search defaults */ -.setting-row { - display: flex; - align-items: center; - justify-content: space-between; - gap: 0.5rem; - margin-top: 0.4rem; -} -.setting-row label { - font-size: 0.85rem; - flex: 1; -} -.setting-row select { - width: 110px; - padding: 0.3rem 0.4rem; - font-size: 0.8rem; - border: 1px solid var(--color-sidebar-border); - border-radius: var(--radius); - background: var(--color-base-background); - color: var(--color-base-font); - cursor: pointer; -} - -/* Mid-search notice */ -.settings-notice { - font-size: 0.72rem; - color: var(--color-suggestion); - margin-top: 0.3rem; - font-style: italic; -} - -/* Dark theme via data-theme attribute */ -html[data-theme="dark"] { - --color-base: #222; - --color-base-font: #dcdcdc; - --color-base-background: #2b2b2b; - --color-header-background: #333; - --color-header-border: #444; - --color-search-border: #555; - --color-search-focus: #5dade2; - --color-result-url: #8ab4f8; - --color-result-url-visited: #b39ddb; - --color-result-content: #b0b0b0; - --color-result-title: #8ab4f8; - --color-result-title-visited: #b39ddb; - --color-result-engine: #999; - --color-result-border: #3a3a3a; - --color-link: #5dade2; - --color-link-visited: #b39ddb; - --color-sidebar-background: #333; - --color-sidebar-border: #444; - --color-infobox-background: #333; - --color-infobox-border: #444; - --color-pagination-current: #5dade2; - --color-pagination-border: #444; - --color-error: #e74c3c; - --color-error-background: #3b1a1a; - --color-suggestion: #999; - --color-footer: #666; - --color-btn-background: #333; - --color-btn-border: #555; - --color-btn-hover: #444; -} - -/* Mobile: Bottom sheet + FAB trigger */ -@media (max-width: 768px) { - /* Hide desktop trigger, show FAB */ - .settings-trigger-desktop { - display: none; - } - .settings-trigger-mobile { - display: block; - } - .settings-popover { - position: fixed; - top: auto; - bottom: 0; - left: 0; - right: 0; - width: 100%; - max-height: 70vh; - border-radius: var(--radius) var(--radius) 0 0; - border-bottom: none; - } - /* FAB: fixed bottom-right button visible only on mobile */ - .settings-trigger-mobile { - display: block; - position: fixed; - bottom: 1.5rem; - right: 1.5rem; - width: 48px; - height: 48px; - border-radius: 50%; - background: var(--color-link); - color: #fff; - border: none; - box-shadow: 0 4px 12px rgba(0,0,0,0.2); - font-size: 1.2rem; - z-index: 199; - opacity: 1; - } -} -``` - -Note: The existing `:root` and `@media (prefers-color-scheme: dark)` blocks provide the "system" theme. `html[data-theme="dark"]` overrides only apply when the user explicitly picks dark mode. When `theme === 'system'`, the `data-theme` attribute is removed and the browser's `prefers-color-scheme` media query kicks in via the existing CSS. - -- [ ] **Step 2: Verify existing tests still pass** - -Run: `go test ./...` -Expected: all pass - -- [ ] **Step 3: Commit** - -```bash -git add internal/views/static/css/kafka.css -git commit -m "feat(settings): add popover, toggle, and bottom-sheet CSS" -``` - ---- - -## Task 2: JS — Settings logic - -**Files:** -- Create: `internal/views/static/js/settings.js` - -- [ ] **Step 1: Write the settings JS module** - -Create `internal/views/static/js/settings.js`: - -```javascript -'use strict'; - -var ALL_ENGINES = [ - 'wikipedia', 'arxiv', 'crossref', 'braveapi', - 'qwant', 'duckduckgo', 'github', 'reddit', 'bing' -]; - -var DEFAULT_PREFS = { - theme: 'system', - engines: ALL_ENGINES.slice(), - safeSearch: 'moderate', - format: 'html' -}; - -var STORAGE_KEY = 'kafka_prefs'; - -// ── Persistence ────────────────────────────────────────────────────────────── - -function loadPrefs() { - try { - var raw = localStorage.getItem(STORAGE_KEY); - if (!raw) return { theme: DEFAULT_PREFS.theme, engines: DEFAULT_PREFS.engines.slice(), safeSearch: DEFAULT_PREFS.safeSearch, format: DEFAULT_PREFS.format }; - var saved = JSON.parse(raw); - return { theme: saved.theme || DEFAULT_PREFS.theme, engines: saved.engines || DEFAULT_PREFS.engines.slice(), safeSearch: saved.safeSearch || DEFAULT_PREFS.safeSearch, format: saved.format || DEFAULT_PREFS.format }; - } catch (e) { - return { theme: DEFAULT_PREFS.theme, engines: DEFAULT_PREFS.engines.slice(), safeSearch: DEFAULT_PREFS.safeSearch, format: DEFAULT_PREFS.format }; - } -} - -function savePrefs(prefs) { - try { - localStorage.setItem(STORAGE_KEY, JSON.stringify({ theme: prefs.theme, engines: prefs.engines, safeSearch: prefs.safeSearch, format: prefs.format })); - } catch (e) { /* quota or private mode */ } -} - -// ── Theme application ──────────────────────────────────────────────────────── - -function applyTheme(theme) { - if (theme === 'system') { - document.documentElement.removeAttribute('data-theme'); - } else { - document.documentElement.setAttribute('data-theme', theme); - } -} - -// ── Engine input sync ───────────────────────────────────────────────────────── - -function syncEngineInput(prefs) { - var input = document.getElementById('engines-input'); - if (input) input.value = prefs.engines.join(','); -} - -// ── Panel open / close ──────────────────────────────────────────────────────── - -function closePanel() { - var panel = document.getElementById('settings-popover'); - var trigger = document.getElementById('settings-trigger'); - if (!panel) return; - panel.setAttribute('data-open', 'false'); - if (trigger) trigger.setAttribute('aria-expanded', 'false'); - if (trigger) trigger.focus(); -} - -function openPanel() { - var panel = document.getElementById('settings-popover'); - var trigger = document.getElementById('settings-trigger'); - if (!panel) return; - panel.setAttribute('data-open', 'true'); - if (trigger) trigger.setAttribute('aria-expanded', 'true'); - var focusable = panel.querySelector('button, input, select'); - if (focusable) focusable.focus(); -} - -// ── Escape key ─────────────────────────────────────────────────────────────── - -document.addEventListener('keydown', function(e) { - if (e.key !== 'Escape') return; - var panel = document.getElementById('settings-popover'); - if (!panel || panel.getAttribute('data-open') !== 'true') return; - closePanel(); -}); - -// ── Click outside ───────────────────────────────────────────────────────────── - -document.addEventListener('click', function(e) { - var panel = document.getElementById('settings-popover'); - var trigger = document.getElementById('settings-trigger'); - if (!panel || panel.getAttribute('data-open') !== 'true') return; - if (!panel.contains(e.target) && (!trigger || !trigger.contains(e.target))) { - closePanel(); - } -}); - -// ── Focus trap ──────────────────────────────────────────────────────────────── - -document.addEventListener('keydown', function(e) { - if (e.key !== 'Tab') return; - var panel = document.getElementById('settings-popover'); - if (!panel || panel.getAttribute('data-open') !== 'true') return; - var focusable = Array.prototype.slice.call(panel.querySelectorAll('button, input, select, [tabindex]:not([tabindex="-1"])')); - if (!focusable.length) return; - var first = focusable[0]; - var last = focusable[focusable.length - 1]; - if (e.shiftKey) { - if (document.activeElement === first) { e.preventDefault(); last.focus(); } - } else { - if (document.activeElement === last) { e.preventDefault(); first.focus(); } - } -}); - -// ── Render ──────────────────────────────────────────────────────────────────── - -function escapeHtml(str) { - return String(str).replace(/&/g, '&').replace(//g, '>'); -} - -function renderPanel(prefs) { - var panel = document.getElementById('settings-popover'); - if (!panel) return; - var body = panel.querySelector('.settings-popover-body'); - if (!body) return; - - var themeBtns = ''; - ['light', 'dark', 'system'].forEach(function(t) { - var icons = { light: '\u2600', dark: '\u263D', system: '\u2318' }; - var labels = { light: 'Light', dark: 'Dark', system: 'System' }; - var active = prefs.theme === t ? ' active' : ''; - themeBtns += ''; - }); - - var engineToggles = ''; - ALL_ENGINES.forEach(function(name) { - var checked = prefs.engines.indexOf(name) !== -1 ? ' checked' : ''; - engineToggles += ''; - }); - - var ssOptions = [ - { val: 'moderate', label: 'Moderate' }, - { val: 'strict', label: 'Strict' }, - { val: 'off', label: 'Off' } - ]; - var fmtOptions = [ - { val: 'html', label: 'HTML' }, - { val: 'json', label: 'JSON' }, - { val: 'csv', label: 'CSV' }, - { val: 'rss', label: 'RSS' } - ]; - var ssOptionsHtml = ''; - var fmtOptionsHtml = ''; - ssOptions.forEach(function(o) { - var sel = prefs.safeSearch === o.val ? ' selected' : ''; - ssOptionsHtml += ''; - }); - fmtOptions.forEach(function(o) { - var sel = prefs.format === o.val ? ' selected' : ''; - fmtOptionsHtml += ''; - }); - - body.innerHTML = - '
Engine changes apply to your next search.
' + - 'Select which engines to use for searches.
+Filter explicit content from results
+Fetch favicons for result URLs. "None" is most private.
+{{.Content}}
+{{.SafeContent}}
{{end}}