From 71b96598ed3ab40244490a63f170f9554d9cdb58 Mon Sep 17 00:00:00 2001 From: Franz Kafka Date: Mon, 23 Mar 2026 14:07:49 +0000 Subject: [PATCH 1/3] =?UTF-8?q?docs:=20refresh=20README=20=E2=80=94=2011?= =?UTF-8?q?=20engines,=20accurate=20clone=20URLs,=20API=20key=20clarity?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 55 +++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index f7fd2d4..6c021c1 100644 --- a/README.md +++ b/README.md @@ -1,20 +1,23 @@ # samsa +*samsa — named for Gregor Samsa, who woke to find himself transformed. You wanted results; you got a metasearch engine.* + A privacy-respecting, open metasearch engine written in Go. SearXNG-compatible API with an HTML frontend, designed to be fast, lightweight, and deployable anywhere. -**9 engines. No JavaScript. No tracking. One binary.** +**11 engines. No JavaScript required. No tracking. One binary.** ## Features - **SearXNG-compatible API** — drop-in replacement for existing integrations -- **9 search engines** — Wikipedia, arXiv, Crossref, Brave, Qwant, DuckDuckGo, GitHub, Reddit, Bing -- **HTML frontend** — HTMX + Go templates with instant search, dark mode, responsive design +- **11 search engines** — Wikipedia, arXiv, Crossref, Brave Search API, Brave (scraping), Qwant, DuckDuckGo, GitHub, Reddit, Bing, Google, YouTube +- **Stack Overflow** — bonus engine, not enabled by default +- **HTML frontend** — Go templates + HTMX with instant search, dark mode, responsive design - **Valkey cache** — optional Redis-compatible caching with configurable TTL - **Rate limiting** — three layers: per-IP, burst, and global (all disabled by default) - **CORS** — configurable origins for browser-based clients - **OpenSearch** — browsers can add samsa as a search engine from the address bar - **Graceful degradation** — individual engine failures don't kill the whole search -- **Docker** — multi-stage build, ~20MB runtime image +- **Docker** — multi-stage build, static binary, ~20MB runtime image - **NixOS** — native NixOS module with systemd service ## Quick Start @@ -22,7 +25,7 @@ A privacy-respecting, open metasearch engine written in Go. SearXNG-compatible A ### Binary ```bash -git clone https://git.ashisgreat.xyz/penal-colony/gosearch.git +git clone https://git.ashisgreat.xyz/penal-colony/samsa.git cd samsa go build ./cmd/samsa ./samsa -config config.toml @@ -32,7 +35,7 @@ go build ./cmd/samsa ```bash cp config.example.toml config.toml -# Edit config.toml — set your Brave API key, etc. +# Edit config.toml — set your Brave API key, YouTube API key, etc. docker compose up -d ``` @@ -41,7 +44,7 @@ docker compose up -d Add to your flake inputs: ```nix -inputs.samsa.url = "git+https://git.ashisgreat.xyz/penal-colony/gosearch.git"; +inputs.samsa.url = "git+https://git.ashisgreat.xyz/penal-colony/samsa.git"; ``` Enable in your configuration: @@ -140,6 +143,8 @@ Copy `config.example.toml` to `config.toml` and edit. All settings can also be o - **`[server]`** — port, timeout, public base URL for OpenSearch - **`[upstream]`** — optional upstream metasearch proxy for unported engines - **`[engines]`** — which engines run locally, engine-specific settings +- **`[engines.brave]`** — Brave Search API key +- **`[engines.youtube]`** — YouTube Data API v3 key - **`[cache]`** — Valkey/Redis address, password, TTL - **`[cors]`** — allowed origins and methods - **`[rate_limit]`** — per-IP sliding window (30 req/min default) @@ -157,6 +162,7 @@ Copy `config.example.toml` to `config.toml` and edit. All settings can also be o | `HTTP_TIMEOUT` | Upstream request timeout | | `BRAVE_API_KEY` | Brave Search API key | | `BRAVE_ACCESS_TOKEN` | Gate requests with token | +| `YOUTUBE_API_KEY` | YouTube Data API v3 key | | `VALKEY_ADDRESS` | Valkey/Redis address | | `VALKEY_PASSWORD` | Valkey/Redis password | | `VALKEY_CACHE_TTL` | Cache TTL | @@ -170,15 +176,23 @@ See `config.example.toml` for the full list including rate limiting and CORS var | Wikipedia | MediaWiki API | General knowledge | | arXiv | arXiv API | Academic papers | | Crossref | Crossref API | Academic metadata | -| Brave | Brave Search API | General web (requires API key) | +| Brave Search API | Brave API | General web (requires API key) | +| Brave | Brave Lite HTML | General web (no key needed) | | Qwant | Qwant Lite HTML | General web | | DuckDuckGo | DDG Lite HTML | General web | | GitHub | GitHub Search API v3 | Code and repositories | | Reddit | Reddit JSON API | Discussions | | Bing | Bing RSS | General web | +| Google | GSA User-Agent scraping | General web (no API key) | +| YouTube | YouTube Data API v3 | Videos (requires API key) | +| Stack Overflow | Stack Exchange API | Q&A (registered, not enabled by default) | Engines not listed in `engines.local_ported` are proxied to an upstream metasearch instance if `upstream.url` is configured. +### API Keys + +Brave Search API and YouTube Data API require keys. If omitted, those engines are silently skipped. Brave Lite (scraping) and Google (GSA UA scraping) work without keys. + ## Architecture ``` @@ -186,7 +200,7 @@ Engines not listed in `engines.local_ported` are proxied to an upstream metasear │ HTTP Handler │ │ /search / /opensearch.xml │ ├─────────────────────────────────────┤ -│ Middleware Chain │ +│ Middleware Chain │ │ Global → Burst → Per-IP → CORS │ ├─────────────────────────────────────┤ │ Search Service │ @@ -194,31 +208,32 @@ Engines not listed in `engines.local_ported` are proxied to an upstream metasear │ WaitGroup + graceful degradation │ ├─────────────────────────────────────┤ │ Cache Layer │ -│ Valkey/Redis (optional, no-op if │ +│ Valkey/Redis (optional; no-op if │ │ unconfigured) │ ├─────────────────────────────────────┤ -│ Engines (×9) │ -│ Each runs in its own goroutine │ -│ Failures → unresponsive_engines │ +│ Engines (×11 default) │ +│ Each runs in its own goroutine │ +│ Failures → unresponsive_engines │ └─────────────────────────────────────┘ ``` ## Docker -The Dockerfile uses a multi-stage build: - -```dockerfile -# Build stage: golang:1.24-alpine -# Runtime stage: alpine:3.21 (~20MB) -# CGO_ENABLED=0 — static binary -``` +The Dockerfile uses a multi-stage build with a static Go binary on alpine Linux: ```bash +# Build: golang:1.24-alpine +# Runtime: alpine:3.21 (~20MB) +# CGO_ENABLED=0 — fully static docker compose up -d ``` Includes Valkey 8 with health checks out of the box. +## Contributing + +See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for a walkthrough of adding a new engine. The interface is two methods: `Name()` and `Search(context, request)`. + ## License [AGPLv3](https://www.gnu.org/licenses/agpl-3.0.html) From aaac1f8f4b06d1c2182d2a36eaf38d488d2322ac Mon Sep 17 00:00:00 2001 From: Franz Kafka Date: Mon, 23 Mar 2026 14:17:41 +0000 Subject: [PATCH 2/3] docs: fix ASCII architecture diagram alignment --- README.md | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 6c021c1..19e0642 100644 --- a/README.md +++ b/README.md @@ -196,25 +196,25 @@ Brave Search API and YouTube Data API require keys. If omitted, those engines ar ## Architecture ``` -┌─────────────────────────────────────┐ -│ HTTP Handler │ -│ /search / /opensearch.xml │ -├─────────────────────────────────────┤ -│ Middleware Chain │ -│ Global → Burst → Per-IP → CORS │ -├─────────────────────────────────────┤ -│ Search Service │ -│ Parallel engine execution │ -│ WaitGroup + graceful degradation │ -├─────────────────────────────────────┤ -│ Cache Layer │ -│ Valkey/Redis (optional; no-op if │ -│ unconfigured) │ -├─────────────────────────────────────┤ -│ Engines (×11 default) │ -│ Each runs in its own goroutine │ -│ Failures → unresponsive_engines │ -└─────────────────────────────────────┘ +┌───────────────────────────────────────┐ +│ HTTP Handler │ +│ /search / /opensearch.xml │ +├───────────────────────────────────────┤ +│ Middleware Chain │ +│ Global → Burst → Per-IP → CORS │ +├───────────────────────────────────────┤ +│ Search Service │ +│ Parallel engine execution │ +│ WaitGroup + graceful degradation │ +├───────────────────────────────────────┤ +│ Cache Layer │ +│ Valkey/Redis (optional; no-op if │ +│ unconfigured) │ +├───────────────────────────────────────┤ +│ Engines (×11 default) │ +│ Each runs in its own goroutine │ +│ Failures → unresponsive_engines │ +└───────────────────────────────────────┘ ``` ## Docker From 540a127f7bbf3b8d5c532e171c86b152fcc36b7b Mon Sep 17 00:00:00 2001 From: Franz Kafka Date: Mon, 23 Mar 2026 15:19:29 +0100 Subject: [PATCH 3/3] Update README.md --- README.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 19e0642..e4c146d 100644 --- a/README.md +++ b/README.md @@ -197,23 +197,23 @@ Brave Search API and YouTube Data API require keys. If omitted, those engines ar ``` ┌───────────────────────────────────────┐ -│ HTTP Handler │ -│ /search / /opensearch.xml │ +│ HTTP Handler │ +│ /search / /opensearch.xml │ ├───────────────────────────────────────┤ -│ Middleware Chain │ -│ Global → Burst → Per-IP → CORS │ +│ Middleware Chain │ +│ Global → Burst → Per-IP → CORS │ ├───────────────────────────────────────┤ -│ Search Service │ -│ Parallel engine execution │ -│ WaitGroup + graceful degradation │ +│ Search Service │ +│ Parallel engine execution │ +│ WaitGroup + graceful degradation │ ├───────────────────────────────────────┤ -│ Cache Layer │ -│ Valkey/Redis (optional; no-op if │ -│ unconfigured) │ +│ Cache Layer │ +│ Valkey/Redis (optional; no-op if │ +│ unconfigured) │ ├───────────────────────────────────────┤ -│ Engines (×11 default) │ -│ Each runs in its own goroutine │ -│ Failures → unresponsive_engines │ +│ Engines (×11 default) │ +│ Each runs in its own goroutine │ +│ Failures → unresponsive_engines │ └───────────────────────────────────────┘ ```