Merge branch 'main' of https://github.com/metamorphosis-dev/samsa

README.md

@@ -1,20 +1,23 @@
 # samsa
 
+*samsa — named for Gregor Samsa, who woke to find himself transformed. You wanted results; you got a metasearch engine.*
+
 A privacy-respecting, open metasearch engine written in Go. SearXNG-compatible API with an HTML frontend, designed to be fast, lightweight, and deployable anywhere.
 
-**9 engines. No JavaScript. No tracking. One binary.**
+**11 engines. No JavaScript required. No tracking. One binary.**
 
 ## Features
 
 - **SearXNG-compatible API** — drop-in replacement for existing integrations
-- **9 search engines** — Wikipedia, arXiv, Crossref, Brave, Qwant, DuckDuckGo, GitHub, Reddit, Bing
-- **HTML frontend** — HTMX + Go templates with instant search, dark mode, responsive design
+- **11 search engines** — Wikipedia, arXiv, Crossref, Brave Search API, Brave (scraping), Qwant, DuckDuckGo, GitHub, Reddit, Bing, Google, YouTube
+- **Stack Overflow** — bonus engine, not enabled by default
+- **HTML frontend** — Go templates + HTMX with instant search, dark mode, responsive design
 - **Valkey cache** — optional Redis-compatible caching with configurable TTL
 - **Rate limiting** — three layers: per-IP, burst, and global (all disabled by default)
 - **CORS** — configurable origins for browser-based clients
 - **OpenSearch** — browsers can add samsa as a search engine from the address bar
 - **Graceful degradation** — individual engine failures don't kill the whole search
-- **Docker** — multi-stage build, ~20MB runtime image
+- **Docker** — multi-stage build, static binary, ~20MB runtime image
 - **NixOS** — native NixOS module with systemd service
 
 ## Quick Start
@@ -22,7 +25,7 @@ A privacy-respecting, open metasearch engine written in Go. SearXNG-compatible A
 ### Binary
 
 ```bash
-git clone https://git.ashisgreat.xyz/penal-colony/gosearch.git
+git clone https://git.ashisgreat.xyz/penal-colony/samsa.git
 cd samsa
 go build ./cmd/samsa
 ./samsa -config config.toml
@@ -32,7 +35,7 @@ go build ./cmd/samsa
 
 ```bash
 cp config.example.toml config.toml
-# Edit config.toml — set your Brave API key, etc.
+# Edit config.toml — set your Brave API key, YouTube API key, etc.
 docker compose up -d
 ```
 
@@ -41,7 +44,7 @@ docker compose up -d
 Add to your flake inputs:
 
 ```nix
-inputs.samsa.url = "git+https://git.ashisgreat.xyz/penal-colony/gosearch.git";
+inputs.samsa.url = "git+https://git.ashisgreat.xyz/penal-colony/samsa.git";
 ```
 
 Enable in your configuration:
@@ -140,6 +143,8 @@ Copy `config.example.toml` to `config.toml` and edit. All settings can also be o
 - **`[server]`** — port, timeout, public base URL for OpenSearch
 - **`[upstream]`** — optional upstream metasearch proxy for unported engines
 - **`[engines]`** — which engines run locally, engine-specific settings
+- **`[engines.brave]`** — Brave Search API key
+- **`[engines.youtube]`** — YouTube Data API v3 key
 - **`[cache]`** — Valkey/Redis address, password, TTL
 - **`[cors]`** — allowed origins and methods
 - **`[rate_limit]`** — per-IP sliding window (30 req/min default)
@@ -157,6 +162,7 @@ Copy `config.example.toml` to `config.toml` and edit. All settings can also be o
 | `HTTP_TIMEOUT` | Upstream request timeout |
 | `BRAVE_API_KEY` | Brave Search API key |
 | `BRAVE_ACCESS_TOKEN` | Gate requests with token |
+| `YOUTUBE_API_KEY` | YouTube Data API v3 key |
 | `VALKEY_ADDRESS` | Valkey/Redis address |
 | `VALKEY_PASSWORD` | Valkey/Redis password |
 | `VALKEY_CACHE_TTL` | Cache TTL |
@@ -170,55 +176,64 @@ See `config.example.toml` for the full list including rate limiting and CORS var
 | Wikipedia | MediaWiki API | General knowledge |
 | arXiv | arXiv API | Academic papers |
 | Crossref | Crossref API | Academic metadata |
-| Brave | Brave Search API | General web (requires API key) |
+| Brave Search API | Brave API | General web (requires API key) |
+| Brave | Brave Lite HTML | General web (no key needed) |
 | Qwant | Qwant Lite HTML | General web |
 | DuckDuckGo | DDG Lite HTML | General web |
 | GitHub | GitHub Search API v3 | Code and repositories |
 | Reddit | Reddit JSON API | Discussions |
 | Bing | Bing RSS | General web |
+| Google | GSA User-Agent scraping | General web (no API key) |
+| YouTube | YouTube Data API v3 | Videos (requires API key) |
+| Stack Overflow | Stack Exchange API | Q&A (registered, not enabled by default) |
 
 Engines not listed in `engines.local_ported` are proxied to an upstream metasearch instance if `upstream.url` is configured.
 
+### API Keys
+
+Brave Search API and YouTube Data API require keys. If omitted, those engines are silently skipped. Brave Lite (scraping) and Google (GSA UA scraping) work without keys.
+
 ## Architecture
 
 ```
-┌─────────────────────────────────────┐
+┌───────────────────────────────────────┐
 │             HTTP Handler                     │
 │      /search  /  /opensearch.xml             │
-├─────────────────────────────────────┤
+├───────────────────────────────────────┤
 │            Middleware Chain                  │
 │   Global → Burst → Per-IP → CORS           │
-├─────────────────────────────────────┤
+├───────────────────────────────────────┤
 │            Search Service                    │
 │     Parallel engine execution                │
 │   WaitGroup + graceful degradation           │
-├─────────────────────────────────────┤
+├───────────────────────────────────────┤
 │             Cache Layer                      │
-│  Valkey/Redis (optional, no-op if   │
+│  Valkey/Redis (optional; no-op if            │
 │              unconfigured)                   │
-├─────────────────────────────────────┤
-│          Engines (×9)               │
+├───────────────────────────────────────┤
+│        Engines (×11 default)                 │
 │    Each runs in its own goroutine            │
 │   Failures → unresponsive_engines           │
-└─────────────────────────────────────┘
+└───────────────────────────────────────┘
 ```
 
 ## Docker
 
-The Dockerfile uses a multi-stage build:
-
-```dockerfile
-# Build stage: golang:1.24-alpine
-# Runtime stage: alpine:3.21 (~20MB)
-# CGO_ENABLED=0 — static binary
-```
+The Dockerfile uses a multi-stage build with a static Go binary on alpine Linux:
 
 ```bash
+# Build: golang:1.24-alpine
+# Runtime: alpine:3.21 (~20MB)
+# CGO_ENABLED=0 — fully static
 docker compose up -d
 ```
 
 Includes Valkey 8 with health checks out of the box.
 
+## Contributing
+
+See [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) for a walkthrough of adding a new engine. The interface is two methods: `Name()` and `Search(context, request)`.
+
 ## License
 
 [AGPLv3](https://www.gnu.org/licenses/agpl-3.0.html)