{{.SafeContent}}
+{{.Content}}
{{end}}Some search engines had errors
--
- {{range .UnresponsiveEngines}}
-
-
-
{{index . 0}}- {{index . 1}} -
- {{end}}
-
diff --git a/.forgejo/workflows/test.yml b/.forgejo/workflows/test.yml index 5f7efb4..bd05693 100644 --- a/.forgejo/workflows/test.yml +++ b/.forgejo/workflows/test.yml @@ -11,15 +11,12 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: https://github.com/actions/checkout@v5 + uses: https://github.com/actions/checkout@v4 - name: Set up Go uses: https://github.com/actions/setup-go@v5 with: go-version-file: go.mod - - name: Clean vendor - run: rm -rf vendor - - name: Test run: go test -race -v ./... diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml deleted file mode 100644 index 47cc920..0000000 --- a/.github/workflows/test.yml +++ /dev/null @@ -1,25 +0,0 @@ -name: Tests - -on: - pull_request: - branches: [main] - push: - branches: [main] - -jobs: - test: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v5 - - - name: Set up Go - uses: actions/setup-go@v5 - with: - go-version-file: go.mod - - - name: Clean vendor - run: rm -rf vendor - - - name: Test - run: go test -race -v ./... diff --git a/.gitignore b/.gitignore index 6cea500..a5388c7 100644 --- a/.gitignore +++ b/.gitignore @@ -1,11 +1,5 @@ node_modules/ .agent/ -internal/spa/dist/ -frontend/node_modules/ -frontend/dist/ -frontend/bun.lock -frontend/bun.lockb -frontend/package-lock.json *.exe *.exe~ *.dll diff --git a/CLAUDE.md b/CLAUDE.md index e136dd7..b7f254e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Project Overview -samsa is a privacy-respecting metasearch engine written in Go. It provides a SearXNG-compatible `/search` API and an HTML frontend (HTMX + Go templates). 9 engines are implemented natively in Go; unlisted engines can be proxied to an upstream metasearch instance. Responses from multiple engines are merged into a single JSON/CSV/RSS/HTML response. +kafka is a privacy-respecting metasearch engine written in Go. It provides a SearXNG-compatible `/search` API and an HTML frontend (HTMX + Go templates). 9 engines are implemented natively in Go; unlisted engines can be proxied to an upstream metasearch instance. Responses from multiple engines are merged into a single JSON/CSV/RSS/HTML response. ## Build & Run Commands @@ -22,7 +22,7 @@ go test -run TestWikipedia ./internal/engines/ go test -v ./internal/engines/ # Run the server (requires config.toml) -go run ./cmd/samsa -config config.toml +go run ./cmd/kafka -config config.toml ``` There is no Makefile. There is no linter configured. @@ -43,7 +43,7 @@ There is no Makefile. There is no linter configured. - `internal/cache` — Valkey/Redis-backed cache with SHA-256 cache keys. No-op if unconfigured. - `internal/middleware` — Three rate limiters (per-IP sliding window, burst+sustained, global) and CORS. All disabled by default. - `internal/views` — HTML templates and static files embedded via `//go:embed`. Renders full pages or HTMX fragments. Templates: `base.html`, `index.html`, `results.html`, `results_inner.html`, `result_item.html`. -- `cmd/samsa` — Entry point. Loads TOML config, seeds env vars for engine code, wires up middleware chain, starts HTTP server. +- `cmd/kafka` — Entry point. Loads TOML config, seeds env vars for engine code, wires up middleware chain, starts HTTP server. **Engine interface** (`internal/engines/engine.go`): ```go @@ -66,7 +66,7 @@ Config is loaded from `config.toml` (see `config.example.toml`). All fields can ## Conventions -- Module path: `github.com/metamorphosis-dev/samsa` +- Module path: `github.com/metamorphosis-dev/kafka` - Tests use shared mock helpers in `internal/engines/http_mock_test.go` (`roundTripperFunc`, `httpResponse`) - Engine implementations are single files under `internal/engines/` (e.g., `wikipedia.go`, `duckduckgo.go`) - Response merging de-duplicates by `engine|title|url` key; suggestions/corrections are merged as sets diff --git a/Dockerfile b/Dockerfile index 9f3443f..e21960f 100644 --- a/Dockerfile +++ b/Dockerfile @@ -21,7 +21,7 @@ RUN apk add --no-cache ca-certificates tzdata COPY --from=builder /kafka /usr/local/bin/kafka COPY config.example.toml /etc/kafka/config.example.toml -EXPOSE 5355 +EXPOSE 8080 ENTRYPOINT ["kafka"] CMD ["-config", "/etc/kafka/config.toml"] diff --git a/README.md b/README.md index 7427922..c03019e 100644 --- a/README.md +++ b/README.md @@ -1,23 +1,20 @@ -# samsa - -*samsa — named for Gregor Samsa, who woke to find himself transformed. You wanted results; you got a metasearch engine.* +# kafka A privacy-respecting, open metasearch engine written in Go. SearXNG-compatible API with an HTML frontend, designed to be fast, lightweight, and deployable anywhere. -**11 engines. No JavaScript required. No tracking. One binary.** +**9 engines. No JavaScript. No tracking. One binary.** ## Features - **SearXNG-compatible API** — drop-in replacement for existing integrations -- **11 search engines** — Wikipedia, arXiv, Crossref, Brave Search API, Brave (scraping), Qwant, DuckDuckGo, GitHub, Reddit, Bing, Google, YouTube -- **Stack Overflow** — bonus engine, not enabled by default -- **HTML frontend** — Go templates + HTMX with instant search, dark mode, responsive design +- **9 search engines** — Wikipedia, arXiv, Crossref, Brave, Qwant, DuckDuckGo, GitHub, Reddit, Bing +- **HTML frontend** — HTMX + Go templates with instant search, dark mode, responsive design - **Valkey cache** — optional Redis-compatible caching with configurable TTL - **Rate limiting** — three layers: per-IP, burst, and global (all disabled by default) - **CORS** — configurable origins for browser-based clients -- **OpenSearch** — browsers can add samsa as a search engine from the address bar +- **OpenSearch** — browsers can add kafka as a search engine from the address bar - **Graceful degradation** — individual engine failures don't kill the whole search -- **Docker** — multi-stage build, static binary, ~20MB runtime image +- **Docker** — multi-stage build, ~20MB runtime image - **NixOS** — native NixOS module with systemd service ## Quick Start @@ -25,17 +22,17 @@ A privacy-respecting, open metasearch engine written in Go. SearXNG-compatible A ### Binary ```bash -git clone https://git.ashisgreat.xyz/penal-colony/samsa.git -cd samsa -go build ./cmd/samsa -./samsa -config config.toml +git clone https://git.ashisgreat.xyz/penal-colony/gosearch.git +cd kafka +go build ./cmd/kafka +./kafka -config config.toml ``` ### Docker Compose ```bash cp config.example.toml config.toml -# Edit config.toml — set your Brave API key, YouTube API key, etc. +# Edit config.toml — set your Brave API key, etc. docker compose up -d ``` @@ -44,28 +41,28 @@ docker compose up -d Add to your flake inputs: ```nix -inputs.samsa.url = "git+https://git.ashisgreat.xyz/penal-colony/samsa.git"; +inputs.kafka.url = "git+https://git.ashisgreat.xyz/penal-colony/gosearch.git"; ``` Enable in your configuration: ```nix -imports = [ inputs.samsa.nixosModules.default ]; +imports = [ inputs.kafka.nixosModules.default ]; -services.samsa = { +services.kafka = { enable = true; openFirewall = true; baseUrl = "https://search.example.com"; - # config = "/etc/samsa/config.toml"; # default + # config = "/etc/kafka/config.toml"; # default }; ``` Write your config: ```bash -sudo mkdir -p /etc/samsa -sudo cp config.example.toml /etc/samsa/config.toml -sudo $EDITOR /etc/samsa/config.toml +sudo mkdir -p /etc/kafka +sudo cp config.example.toml /etc/kafka/config.toml +sudo $EDITOR /etc/kafka/config.toml ``` Deploy: @@ -79,7 +76,7 @@ sudo nixos-rebuild switch --flake .# ```bash nix develop go test ./... -go run ./cmd/samsa -config config.toml +go run ./cmd/kafka -config config.toml ``` ## Endpoints @@ -110,7 +107,7 @@ go run ./cmd/samsa -config config.toml ### Example ```bash -curl "http://localhost:5355/search?q=golang&format=json&engines=github,duckduckgo" +curl "http://localhost:8080/search?q=golang&format=json&engines=github,duckduckgo" ``` ### Response (JSON) @@ -143,8 +140,6 @@ Copy `config.example.toml` to `config.toml` and edit. All settings can also be o - **`[server]`** — port, timeout, public base URL for OpenSearch - **`[upstream]`** — optional upstream metasearch proxy for unported engines - **`[engines]`** — which engines run locally, engine-specific settings -- **`[engines.brave]`** — Brave Search API key -- **`[engines.youtube]`** — YouTube Data API v3 key - **`[cache]`** — Valkey/Redis address, password, TTL - **`[cors]`** — allowed origins and methods - **`[rate_limit]`** — per-IP sliding window (30 req/min default) @@ -155,14 +150,13 @@ Copy `config.example.toml` to `config.toml` and edit. All settings can also be o | Variable | Description | |---|---| -| `PORT` | Listen port (default: 5355) | +| `PORT` | Listen port (default: 8080) | | `BASE_URL` | Public URL for OpenSearch XML | | `UPSTREAM_SEARXNG_URL` | Upstream instance URL | | `LOCAL_PORTED_ENGINES` | Comma-separated local engine list | | `HTTP_TIMEOUT` | Upstream request timeout | | `BRAVE_API_KEY` | Brave Search API key | | `BRAVE_ACCESS_TOKEN` | Gate requests with token | -| `YOUTUBE_API_KEY` | YouTube Data API v3 key | | `VALKEY_ADDRESS` | Valkey/Redis address | | `VALKEY_PASSWORD` | Valkey/Redis password | | `VALKEY_CACHE_TTL` | Cache TTL | @@ -176,64 +170,55 @@ See `config.example.toml` for the full list including rate limiting and CORS var | Wikipedia | MediaWiki API | General knowledge | | arXiv | arXiv API | Academic papers | | Crossref | Crossref API | Academic metadata | -| Brave Search API | Brave API | General web (requires API key) | -| Brave | Brave Lite HTML | General web (no key needed) | +| Brave | Brave Search API | General web (requires API key) | | Qwant | Qwant Lite HTML | General web | | DuckDuckGo | DDG Lite HTML | General web | | GitHub | GitHub Search API v3 | Code and repositories | | Reddit | Reddit JSON API | Discussions | | Bing | Bing RSS | General web | -| Google | GSA User-Agent scraping | General web (no API key) | -| YouTube | YouTube Data API v3 | Videos (requires API key) | -| Stack Overflow | Stack Exchange API | Q&A (registered, not enabled by default) | Engines not listed in `engines.local_ported` are proxied to an upstream metasearch instance if `upstream.url` is configured. -### API Keys - -Brave Search API and YouTube Data API require keys. If omitted, those engines are silently skipped. Brave Lite (scraping) and Google (GSA UA scraping) work without keys. - ## Architecture ``` -┌───────────────────────────────────────┐ -│ HTTP Handler │ -│ /search / /opensearch.xml │ -├───────────────────────────────────────┤ -│ Middleware Chain │ -│ Global → Burst → Per-IP → CORS │ -├───────────────────────────────────────┤ -│ Search Service │ -│ Parallel engine execution │ -│ WaitGroup + graceful degradation │ -├───────────────────────────────────────┤ -│ Cache Layer │ -│ Valkey/Redis (optional; no-op if │ -│ unconfigured) │ -├───────────────────────────────────────┤ -│ Engines (×11 default) │ -│ Each runs in its own goroutine │ -│ Failures → unresponsive_engines │ -└───────────────────────────────────────┘ +┌─────────────────────────────────────┐ +│ HTTP Handler │ +│ /search / /opensearch.xml │ +├─────────────────────────────────────┤ +│ Middleware Chain │ +│ Global → Burst → Per-IP → CORS │ +├─────────────────────────────────────┤ +│ Search Service │ +│ Parallel engine execution │ +│ WaitGroup + graceful degradation │ +├─────────────────────────────────────┤ +│ Cache Layer │ +│ Valkey/Redis (optional, no-op if │ +│ unconfigured) │ +├─────────────────────────────────────┤ +│ Engines (×9) │ +│ Each runs in its own goroutine │ +│ Failures → unresponsive_engines │ +└─────────────────────────────────────┘ ``` ## Docker -The Dockerfile uses a multi-stage build with a static Go binary on alpine Linux: +The Dockerfile uses a multi-stage build: + +```dockerfile +# Build stage: golang:1.24-alpine +# Runtime stage: alpine:3.21 (~20MB) +# CGO_ENABLED=0 — static binary +``` ```bash -# Build: golang:1.24-alpine -# Runtime: alpine:3.21 (~20MB) -# CGO_ENABLED=0 — fully static docker compose up -d ``` Includes Valkey 8 with health checks out of the box. -## Contributing - -See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for a walkthrough of adding a new engine. The interface is two methods: `Name()` and `Search(context, request)`. - ## License -[AGPLv3](https://www.gnu.org/licenses/agpl-3.0.html) +MIT diff --git a/cmd/samsa/main.go b/cmd/kafka/main.go similarity index 82% rename from cmd/samsa/main.go rename to cmd/kafka/main.go index 199033b..cdc81b5 100644 --- a/cmd/samsa/main.go +++ b/cmd/kafka/main.go @@ -1,4 +1,4 @@ -// samsa — a privacy-respecting metasearch engine +// kafka — a privacy-respecting metasearch engine // Copyright (C) 2026-present metamorphosis-dev // // This program is free software: you can redistribute it and/or modify @@ -25,13 +25,13 @@ import ( "net/http" "os" - "github.com/metamorphosis-dev/samsa/internal/autocomplete" - "github.com/metamorphosis-dev/samsa/internal/cache" - "github.com/metamorphosis-dev/samsa/internal/config" - "github.com/metamorphosis-dev/samsa/internal/httpapi" - "github.com/metamorphosis-dev/samsa/internal/middleware" - "github.com/metamorphosis-dev/samsa/internal/search" - "github.com/metamorphosis-dev/samsa/internal/views" + "github.com/metamorphosis-dev/kafka/internal/autocomplete" + "github.com/metamorphosis-dev/kafka/internal/cache" + "github.com/metamorphosis-dev/kafka/internal/config" + "github.com/metamorphosis-dev/kafka/internal/httpapi" + "github.com/metamorphosis-dev/kafka/internal/middleware" + "github.com/metamorphosis-dev/kafka/internal/search" + "github.com/metamorphosis-dev/kafka/internal/views" ) func main() { @@ -77,20 +77,14 @@ func main() { acSvc := autocomplete.NewService(cfg.Upstream.URL, cfg.HTTPTimeout()) - h := httpapi.NewHandler(svc, acSvc.Suggestions, cfg.Server.SourceURL, searchCache) + h := httpapi.NewHandler(svc, acSvc.Suggestions, cfg.Server.SourceURL) mux := http.NewServeMux() - - // HTML template routes mux.HandleFunc("/", h.Index) - mux.HandleFunc("/search", h.Search) - mux.HandleFunc("/preferences", h.Preferences) - - // API routes mux.HandleFunc("/healthz", h.Healthz) + mux.HandleFunc("/search", h.Search) mux.HandleFunc("/autocompleter", h.Autocompleter) mux.HandleFunc("/opensearch.xml", h.OpenSearch(cfg.Server.BaseURL)) - mux.HandleFunc("/favicon/", h.Favicon) // Serve embedded static files (CSS, JS, images). staticFS, err := views.StaticFS() @@ -100,9 +94,8 @@ func main() { var subFS fs.FS = staticFS mux.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.FS(subFS)))) - // Apply middleware: global rate limit → burst rate limit → per-IP rate limit → CORS → security headers → handler. + // Apply middleware: global rate limit → burst rate limit → per-IP rate limit → CORS → handler. var handler http.Handler = mux - handler = middleware.SecurityHeaders(middleware.SecurityHeadersConfig{})(handler) handler = middleware.CORS(middleware.CORSConfig{ AllowedOrigins: cfg.CORS.AllowedOrigins, AllowedMethods: cfg.CORS.AllowedMethods, @@ -114,7 +107,6 @@ func main() { Requests: cfg.RateLimit.Requests, Window: cfg.RateLimitWindow(), CleanupInterval: cfg.RateLimitCleanupInterval(), - TrustedProxies: cfg.RateLimit.TrustedProxies, }, logger)(handler) handler = middleware.GlobalRateLimit(middleware.GlobalRateLimitConfig{ Requests: cfg.GlobalRateLimit.Requests, @@ -128,7 +120,7 @@ func main() { }, logger)(handler) addr := fmt.Sprintf(":%d", cfg.Server.Port) - logger.Info("samsa starting", + logger.Info("kafka starting", "addr", addr, "cache", searchCache.Enabled(), "rate_limit", cfg.RateLimit.Requests > 0, diff --git a/config.example.toml b/config.example.toml index 7fed53e..042bb63 100644 --- a/config.example.toml +++ b/config.example.toml @@ -1,22 +1,22 @@ -# samsa configuration +# kafka configuration # Copy to config.toml and adjust as needed. # Environment variables are used as fallbacks when a config field is empty/unset. [server] # Listen port (env: PORT) -port = 5355 +port = 8080 # HTTP timeout for engine and upstream calls (env: HTTP_TIMEOUT) http_timeout = "10s" # Public base URL for OpenSearch XML (env: BASE_URL) -# Set this so browsers can add samsa as a search engine. +# Set this so browsers can add kafka as a search engine. # Example: "https://search.example.com" base_url = "" # Link to the source code (shown in footer as "Source" link) -# Defaults to the upstream samsa repo if not set. -# Example: "https://git.example.com/my-samsa-fork" +# Defaults to the upstream kafka repo if not set. +# Example: "https://git.example.com/my-kafka-fork" source_url = "" [upstream] @@ -27,8 +27,7 @@ url = "" [engines] # Comma-separated list of engines to execute locally in Go (env: LOCAL_PORTED_ENGINES) # Engines not listed here will be proxied to the upstream instance. -# Include bing_images, ddg_images, qwant_images for image search when [upstream].url is empty. -local_ported = ["wikipedia", "wikidata", "arxiv", "crossref", "braveapi", "qwant", "duckduckgo", "github", "reddit", "bing", "google", "youtube", "bing_images", "ddg_images", "qwant_images"] +local_ported = ["wikipedia", "arxiv", "crossref", "braveapi", "qwant", "duckduckgo", "github", "reddit", "bing", "google", "youtube"] [engines.brave] # Brave Search API key (env: BRAVE_API_KEY) @@ -57,12 +56,6 @@ db = 0 # Cache TTL for search results (env: VALKEY_CACHE_TTL) default_ttl = "5m" -[cache.ttl_overrides] -# Per-engine TTL overrides (uncomment to use): -# wikipedia = "48h" -# reddit = "15m" -# braveapi = "2h" - [cors] # CORS configuration for browser-based clients. # Allowed origins: use "*" for all, or specific domains (env: CORS_ALLOWED_ORIGINS) diff --git a/docker-compose.yml b/docker-compose.yml index 538713f..98eae96 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -8,7 +8,7 @@ services: kafka: build: . ports: - - "5355:5355" + - "8080:8080" volumes: - ./config.toml:/etc/kafka/config.toml:ro depends_on: diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md deleted file mode 100644 index bada2e2..0000000 --- a/docs/CONTRIBUTING.md +++ /dev/null @@ -1,218 +0,0 @@ -# Contributing — Adding a New Engine - -This guide walks through adding a new search engine to samsa. The minimal engine needs only an HTTP client, a query, and a result parser. - ---- - -## 1. Create the engine file - -Place it in `internal/engines/`: - -``` -internal/engines/ - myengine.go ← your engine - myengine_test.go ← tests (required) -``` - -Name the struct after the engine, e.g. `WolframEngine` for "wolfram". The `Name()` method returns the engine key used throughout samsa. - -## 2. Implement the Engine interface - -```go -package engines - -import ( - "context" - "github.com/metamorphosis-dev/samsa/internal/contracts" -) - -type MyEngine struct { - client *http.Client -} - -func (e *MyEngine) Name() string { return "myengine" } - -func (e *MyEngine) Search(ctx context.Context, req contracts.SearchRequest) (contracts.SearchResponse, error) { - // ... -} -``` - -### The SearchRequest fields you'll use most: - -| Field | Type | Description | -|-------|------|-------------| -| `Query` | `string` | The search query | -| `Pageno` | `int` | Current page number (1-based) | -| `Safesearch` | `int` | 0=off, 1=moderate, 2=strict | -| `Language` | `string` | ISO language code (e.g. `"en"`) | - -### The SearchResponse to return: - -```go -contracts.SearchResponse{ - Query: req.Query, - NumberOfResults: len(results), - Results: results, // []MainResult - Answers: []map[string]any{}, - Corrections: []string{}, - Infoboxes: []map[string]any{}, - Suggestions: []string{}, - UnresponsiveEngines: [][2]string{}, -} -``` - -### Empty query — return early: - -```go -if strings.TrimSpace(req.Query) == "" { - return contracts.SearchResponse{Query: req.Query}, nil -} -``` - -### Engine unavailable / error — graceful degradation: - -```go -// Rate limited or blocked -return contracts.SearchResponse{ - Query: req.Query, - UnresponsiveEngines: [][2]string{{"myengine", "reason"}}, - Results: []contracts.MainResult{}, - // ... empty other fields -}, nil - -// Hard error — return it -return contracts.SearchResponse{}, fmt.Errorf("myengine upstream error: status %d", resp.StatusCode) -``` - -## 3. Build the result - -```go -urlPtr := "https://example.com/result" -result := contracts.MainResult{ - Title: "Result Title", - Content: "Snippet or description text", - URL: &urlPtr, // pointer to string, required - Engine: "myengine", - Category: "general", // or "it", "science", "videos", "images", "social media" - Score: 0, // used for relevance ranking during merge - Engines: []string{"myengine"}, -} -``` - -### Template field - -The template system checks for `"videos"` and `"images"`. Everything else renders via `result_item.html`. Set `Template` only if you have a custom template; omit it for the default result card. - -### Category field - -Controls which category tab the result appears under and which engines are triggered: - -| Category | Engines used | -|----------|-------------| -| `general` | google, bing, ddg, brave, braveapi, qwant, wikipedia | -| `it` | github, stackoverflow | -| `science` | arxiv, crossref | -| `videos` | youtube | -| `images` | bing_images, ddg_images, qwant_images | -| `social media` | reddit | - -## 4. Wire it into the factory - -In `internal/engines/factory.go`, add your engine to the map returned by `NewDefaultPortedEngines`: - -```go -"myengine": &MyEngine{client: client}, -``` - -If your engine needs an API key, read it from config or the environment (see `braveapi` or `youtube` in factory.go for the pattern). - -## 5. Register defaults - -In `internal/engines/planner.go`: - -**Add to `defaultPortedEngines`:** -```go -var defaultPortedEngines = []string{ - // ... existing ... - "myengine", -} -``` - -**Add to category mapping in `inferFromCategories`** (if applicable): -```go -case "general": - set["myengine"] = true -``` - -**Update the sort order map** so results maintain consistent ordering: -```go -order := map[string]int{ - // ... existing ... - "myengine": N, // pick a slot -} -``` - -## 6. Add tests - -At minimum, test: -- `Name()` returns the correct string -- Nil engine returns an error -- Empty query returns zero results -- Successful API response parses correctly -- Rate limit / error cases return `UnresponsiveEngines` with a reason - -Use `httptest.NewServer` to mock the upstream API. See `arxiv_test.go` or `reddit_test.go` for examples. - -## 7. Build and test - -```bash -go build ./... -go test ./internal/engines/ -run MyEngine -v -go test ./... -``` - -## Example: Adding an RSS-based engine - -If the engine provides an RSS feed, the parsing is straightforward: - -```go -type rssItem struct { - Title string `xml:"title"` - Link string `xml:"link"` - Description string `xml:"description"` -} - -type rssFeed struct { - Channel struct { - Items []rssItem `xml:"item"` - } `xml:"channel"` -} - -dec := xml.NewDecoder(resp.Body) -var feed rssFeed -dec.Decode(&feed) - -for _, item := range feed.Channel.Items { - urlPtr := item.Link - results = append(results, contracts.MainResult{ - Title: item.Title, - Content: stripHTML(item.Description), - URL: &urlPtr, - Engine: "myengine", - // ... - }) -} -``` - -## Checklist - -- [ ] Engine file created in `internal/engines/` -- [ ] `Engine` interface implemented (`Name()` + `Search()`) -- [ ] Empty query handled (return early, no error) -- [ ] Graceful degradation for errors and rate limits -- [ ] Results use `Category` to group with related engines -- [ ] Factory updated with new engine -- [ ] Planner updated (defaults + category mapping + sort order) -- [ ] Tests written covering main paths -- [ ] `go build ./...` succeeds -- [ ] `go test ./...` passes diff --git a/docs/superpowers/plans/2026-03-22-settings-ui.md b/docs/superpowers/plans/2026-03-22-settings-ui.md new file mode 100644 index 0000000..cf34df5 --- /dev/null +++ b/docs/superpowers/plans/2026-03-22-settings-ui.md @@ -0,0 +1,747 @@ +# Settings UI Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A preferences popover panel (top-right on desktop, bottom sheet on mobile) that lets users set theme, enabled engines, safe search, and default format. All changes auto-save to `localStorage` and apply immediately to the DOM. + +**Architecture:** Pure client-side JS + CSS added alongside existing templates. No Go changes. Settings persist via `localStorage` key `kafka_prefs`. Theme applies via `data-theme` attribute on ``. + +**Tech Stack:** Vanilla JS (no framework), existing `kafka.css` custom properties, HTMX for search. + +--- + +## File Map + +| Action | File | +|--------|------| +| Create | `internal/views/static/js/settings.js` | +| Modify | `internal/views/static/css/kafka.css` | +| Modify | `internal/views/templates/base.html` | +| Modify | `internal/views/templates/index.html` | +| Modify | `internal/views/templates/results.html` | +| Modify | `internal/views/views.go` | + +**Key insight on engine preferences:** `ParseSearchRequest` reads `engines` as a CSV form value (`r.FormValue("engines")`). The search forms in `index.html` and `results.html` will get a hidden `#engines-input` field that is kept in sync with localStorage. On submit, the engines preference is sent as a normal form field. HTMX `hx-include="this"` already includes the form element, so the hidden input is automatically included in the request. + +--- + +## Task 1: CSS — Popover, toggles, bottom sheet + +**Files:** +- Modify: `internal/views/static/css/kafka.css` + +- [ ] **Step 1: Add CSS for popover, triggers, toggles, bottom sheet** + +Append the following to `kafka.css`: + +```css +/* ============================================ + Settings Panel + ============================================ */ + +/* Header */ +.site-header { + display: flex; + align-items: center; + justify-content: space-between; + padding: 0.6rem 1rem; + background: var(--color-header-background); + border-bottom: 1px solid var(--color-header-border); +} +.site-title { + font-size: 1rem; + font-weight: 600; + color: var(--color-base-font); +} + +/* Gear trigger button */ +.settings-trigger { + background: none; + border: none; + font-size: 1.1rem; + cursor: pointer; + padding: 0.3rem 0.5rem; + border-radius: var(--radius); + color: var(--color-base-font); + opacity: 0.7; + transition: opacity 0.2s, background 0.2s; + line-height: 1; +} +.settings-trigger:hover, +.settings-trigger[aria-expanded="true"] { + opacity: 1; + background: var(--color-sidebar-background); +} + +/* Popover panel */ +.settings-popover { + position: absolute; + top: 100%; + right: 0; + width: 280px; + max-height: 420px; + overflow-y: auto; + background: var(--color-base-background); + border: 1px solid var(--color-sidebar-border); + border-radius: var(--radius); + box-shadow: 0 8px 24px rgba(0, 0, 0, 0.12); + z-index: 200; + display: none; + flex-direction: column; +} +.settings-popover[data-open="true"] { + display: flex; + animation: settings-slide-in 0.2s ease; +} +@keyframes settings-slide-in { + from { opacity: 0; transform: translateY(-8px); } + to { opacity: 1; transform: translateY(0); } +} + +.settings-popover-header { + display: flex; + justify-content: space-between; + align-items: center; + padding: 0.75rem 1rem; + border-bottom: 1px solid var(--color-sidebar-border); + font-weight: 600; + font-size: 0.9rem; + flex-shrink: 0; +} +.settings-popover-close { + background: none; + border: none; + font-size: 1.2rem; + cursor: pointer; + color: var(--color-base-font); + opacity: 0.6; + padding: 0 0.25rem; + line-height: 1; +} +.settings-popover-close:hover { opacity: 1; } + +.settings-popover-body { + padding: 0.8rem; + display: flex; + flex-direction: column; + gap: 1rem; +} + +.settings-section-title { + font-size: 0.7rem; + font-weight: 600; + text-transform: uppercase; + letter-spacing: 0.05em; + color: var(--color-suggestion); + margin-bottom: 0.5rem; +} + +/* Theme buttons */ +.theme-buttons { + display: flex; + gap: 0.4rem; +} +.theme-btn { + flex: 1; + padding: 0.35rem 0.5rem; + border: 1px solid var(--color-sidebar-border); + border-radius: var(--radius); + background: var(--color-btn-background); + color: var(--color-base-font); + cursor: pointer; + font-size: 0.75rem; + text-align: center; + transition: background 0.15s, border-color 0.15s; +} +.theme-btn:hover { background: var(--color-btn-hover); } +.theme-btn.active { + background: var(--color-link); + color: #fff; + border-color: var(--color-link); +} + +/* Engine toggles — 2-column grid */ +.engine-grid { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 0.4rem; +} +.engine-toggle { + display: flex; + align-items: center; + gap: 0.4rem; + padding: 0.3rem 0.5rem; + border-radius: var(--radius); + background: var(--color-sidebar-background); + font-size: 0.78rem; + cursor: pointer; +} +.engine-toggle input[type="checkbox"] { + width: 15px; + height: 15px; + margin: 0; + cursor: pointer; + accent-color: var(--color-link); +} +.engine-toggle span { + flex: 1; + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +/* Search defaults */ +.setting-row { + display: flex; + align-items: center; + justify-content: space-between; + gap: 0.5rem; + margin-top: 0.4rem; +} +.setting-row label { + font-size: 0.85rem; + flex: 1; +} +.setting-row select { + width: 110px; + padding: 0.3rem 0.4rem; + font-size: 0.8rem; + border: 1px solid var(--color-sidebar-border); + border-radius: var(--radius); + background: var(--color-base-background); + color: var(--color-base-font); + cursor: pointer; +} + +/* Mid-search notice */ +.settings-notice { + font-size: 0.72rem; + color: var(--color-suggestion); + margin-top: 0.3rem; + font-style: italic; +} + +/* Dark theme via data-theme attribute */ +html[data-theme="dark"] { + --color-base: #222; + --color-base-font: #dcdcdc; + --color-base-background: #2b2b2b; + --color-header-background: #333; + --color-header-border: #444; + --color-search-border: #555; + --color-search-focus: #5dade2; + --color-result-url: #8ab4f8; + --color-result-url-visited: #b39ddb; + --color-result-content: #b0b0b0; + --color-result-title: #8ab4f8; + --color-result-title-visited: #b39ddb; + --color-result-engine: #999; + --color-result-border: #3a3a3a; + --color-link: #5dade2; + --color-link-visited: #b39ddb; + --color-sidebar-background: #333; + --color-sidebar-border: #444; + --color-infobox-background: #333; + --color-infobox-border: #444; + --color-pagination-current: #5dade2; + --color-pagination-border: #444; + --color-error: #e74c3c; + --color-error-background: #3b1a1a; + --color-suggestion: #999; + --color-footer: #666; + --color-btn-background: #333; + --color-btn-border: #555; + --color-btn-hover: #444; +} + +/* Mobile: Bottom sheet + FAB trigger */ +@media (max-width: 768px) { + /* Hide desktop trigger, show FAB */ + .settings-trigger-desktop { + display: none; + } + .settings-trigger-mobile { + display: block; + } + .settings-popover { + position: fixed; + top: auto; + bottom: 0; + left: 0; + right: 0; + width: 100%; + max-height: 70vh; + border-radius: var(--radius) var(--radius) 0 0; + border-bottom: none; + } + /* FAB: fixed bottom-right button visible only on mobile */ + .settings-trigger-mobile { + display: block; + position: fixed; + bottom: 1.5rem; + right: 1.5rem; + width: 48px; + height: 48px; + border-radius: 50%; + background: var(--color-link); + color: #fff; + border: none; + box-shadow: 0 4px 12px rgba(0,0,0,0.2); + font-size: 1.2rem; + z-index: 199; + opacity: 1; + } +} +``` + +Note: The existing `:root` and `@media (prefers-color-scheme: dark)` blocks provide the "system" theme. `html[data-theme="dark"]` overrides only apply when the user explicitly picks dark mode. When `theme === 'system'`, the `data-theme` attribute is removed and the browser's `prefers-color-scheme` media query kicks in via the existing CSS. + +- [ ] **Step 2: Verify existing tests still pass** + +Run: `go test ./...` +Expected: all pass + +- [ ] **Step 3: Commit** + +```bash +git add internal/views/static/css/kafka.css +git commit -m "feat(settings): add popover, toggle, and bottom-sheet CSS" +``` + +--- + +## Task 2: JS — Settings logic + +**Files:** +- Create: `internal/views/static/js/settings.js` + +- [ ] **Step 1: Write the settings JS module** + +Create `internal/views/static/js/settings.js`: + +```javascript +'use strict'; + +var ALL_ENGINES = [ + 'wikipedia', 'arxiv', 'crossref', 'braveapi', + 'qwant', 'duckduckgo', 'github', 'reddit', 'bing' +]; + +var DEFAULT_PREFS = { + theme: 'system', + engines: ALL_ENGINES.slice(), + safeSearch: 'moderate', + format: 'html' +}; + +var STORAGE_KEY = 'kafka_prefs'; + +// ── Persistence ────────────────────────────────────────────────────────────── + +function loadPrefs() { + try { + var raw = localStorage.getItem(STORAGE_KEY); + if (!raw) return { theme: DEFAULT_PREFS.theme, engines: DEFAULT_PREFS.engines.slice(), safeSearch: DEFAULT_PREFS.safeSearch, format: DEFAULT_PREFS.format }; + var saved = JSON.parse(raw); + return { theme: saved.theme || DEFAULT_PREFS.theme, engines: saved.engines || DEFAULT_PREFS.engines.slice(), safeSearch: saved.safeSearch || DEFAULT_PREFS.safeSearch, format: saved.format || DEFAULT_PREFS.format }; + } catch (e) { + return { theme: DEFAULT_PREFS.theme, engines: DEFAULT_PREFS.engines.slice(), safeSearch: DEFAULT_PREFS.safeSearch, format: DEFAULT_PREFS.format }; + } +} + +function savePrefs(prefs) { + try { + localStorage.setItem(STORAGE_KEY, JSON.stringify({ theme: prefs.theme, engines: prefs.engines, safeSearch: prefs.safeSearch, format: prefs.format })); + } catch (e) { /* quota or private mode */ } +} + +// ── Theme application ──────────────────────────────────────────────────────── + +function applyTheme(theme) { + if (theme === 'system') { + document.documentElement.removeAttribute('data-theme'); + } else { + document.documentElement.setAttribute('data-theme', theme); + } +} + +// ── Engine input sync ───────────────────────────────────────────────────────── + +function syncEngineInput(prefs) { + var input = document.getElementById('engines-input'); + if (input) input.value = prefs.engines.join(','); +} + +// ── Panel open / close ──────────────────────────────────────────────────────── + +function closePanel() { + var panel = document.getElementById('settings-popover'); + var trigger = document.getElementById('settings-trigger'); + if (!panel) return; + panel.setAttribute('data-open', 'false'); + if (trigger) trigger.setAttribute('aria-expanded', 'false'); + if (trigger) trigger.focus(); +} + +function openPanel() { + var panel = document.getElementById('settings-popover'); + var trigger = document.getElementById('settings-trigger'); + if (!panel) return; + panel.setAttribute('data-open', 'true'); + if (trigger) trigger.setAttribute('aria-expanded', 'true'); + var focusable = panel.querySelector('button, input, select'); + if (focusable) focusable.focus(); +} + +// ── Escape key ─────────────────────────────────────────────────────────────── + +document.addEventListener('keydown', function(e) { + if (e.key !== 'Escape') return; + var panel = document.getElementById('settings-popover'); + if (!panel || panel.getAttribute('data-open') !== 'true') return; + closePanel(); +}); + +// ── Click outside ───────────────────────────────────────────────────────────── + +document.addEventListener('click', function(e) { + var panel = document.getElementById('settings-popover'); + var trigger = document.getElementById('settings-trigger'); + if (!panel || panel.getAttribute('data-open') !== 'true') return; + if (!panel.contains(e.target) && (!trigger || !trigger.contains(e.target))) { + closePanel(); + } +}); + +// ── Focus trap ──────────────────────────────────────────────────────────────── + +document.addEventListener('keydown', function(e) { + if (e.key !== 'Tab') return; + var panel = document.getElementById('settings-popover'); + if (!panel || panel.getAttribute('data-open') !== 'true') return; + var focusable = Array.prototype.slice.call(panel.querySelectorAll('button, input, select, [tabindex]:not([tabindex="-1"])')); + if (!focusable.length) return; + var first = focusable[0]; + var last = focusable[focusable.length - 1]; + if (e.shiftKey) { + if (document.activeElement === first) { e.preventDefault(); last.focus(); } + } else { + if (document.activeElement === last) { e.preventDefault(); first.focus(); } + } +}); + +// ── Render ──────────────────────────────────────────────────────────────────── + +function escapeHtml(str) { + return String(str).replace(/&/g, '&').replace(//g, '>'); +} + +function renderPanel(prefs) { + var panel = document.getElementById('settings-popover'); + if (!panel) return; + var body = panel.querySelector('.settings-popover-body'); + if (!body) return; + + var themeBtns = ''; + ['light', 'dark', 'system'].forEach(function(t) { + var icons = { light: '\u2600', dark: '\u263D', system: '\u2318' }; + var labels = { light: 'Light', dark: 'Dark', system: 'System' }; + var active = prefs.theme === t ? ' active' : ''; + themeBtns += ''; + }); + + var engineToggles = ''; + ALL_ENGINES.forEach(function(name) { + var checked = prefs.engines.indexOf(name) !== -1 ? ' checked' : ''; + engineToggles += ''; + }); + + var ssOptions = [ + { val: 'moderate', label: 'Moderate' }, + { val: 'strict', label: 'Strict' }, + { val: 'off', label: 'Off' } + ]; + var fmtOptions = [ + { val: 'html', label: 'HTML' }, + { val: 'json', label: 'JSON' }, + { val: 'csv', label: 'CSV' }, + { val: 'rss', label: 'RSS' } + ]; + var ssOptionsHtml = ''; + var fmtOptionsHtml = ''; + ssOptions.forEach(function(o) { + var sel = prefs.safeSearch === o.val ? ' selected' : ''; + ssOptionsHtml += ''; + }); + fmtOptions.forEach(function(o) { + var sel = prefs.format === o.val ? ' selected' : ''; + fmtOptionsHtml += ''; + }); + + body.innerHTML = + '
Engine changes apply to your next search.
' + + 'Private meta-search, powered by open source.
- - + +Select which engines to use for searches.
-Filter explicit content from results
-Fetch favicons for result URLs. "None" is most private.
-{{.SafeContent}}
+{{.Content}}
{{end}}{{index . 0}}
- {{index . 1}}
-