docs: add gosearch integration design spec
Add comprehensive design document for integrating gosearch (kafka) metasearch engine into the NixOS VPS configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
bd3295e918
commit
ea372d056e
1 changed files with 210 additions and 0 deletions
210
docs/superpowers/specs/2026-03-21-gosearch-integration-design.md
Normal file
210
docs/superpowers/specs/2026-03-21-gosearch-integration-design.md
Normal file
|
|
@ -0,0 +1,210 @@
|
|||
# gosearch Integration Design
|
||||
|
||||
**Date:** 2026-03-21
|
||||
**Status:** Approved
|
||||
**Author:** Claude Code
|
||||
|
||||
## Overview
|
||||
|
||||
Integrate [gosearch](https://git.ashisgreat.xyz/penal-colony/gosearch) (kafka), a privacy-respecting Go-based metasearch engine, into the NixOS VPS configuration. The service will run as a hardened systemd service, connect to the existing Valkey cache, and be exposed via Nginx reverse proxy.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Deploy gosearch using its native NixOS module
|
||||
- Expose on `search2.ashisgreat.xyz`
|
||||
- Connect to existing Valkey instance (shared with SearXNG)
|
||||
- Enable Brave Search with API key
|
||||
- Run on port `8889` (localhost only)
|
||||
- Harden the service following systemd security best practices
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ User Request Flow │
|
||||
│ Browser → search2.ashisgreat.xyz → Nginx → localhost:8889 │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ systemd kafka service │
|
||||
│ - User: kafka | Port: 8889 | Hardened systemd │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Configuration Layer │
|
||||
│ /etc/kafka/config.toml (generated via sops template) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Dependencies │
|
||||
│ - Valkey: 127.0.0.1:6379, DB index 1 │
|
||||
│ - Brave API: from SOPS │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Implementation
|
||||
|
||||
### 1. Add Flake Input
|
||||
|
||||
```nix
|
||||
# flake.nix
|
||||
inputs.gosearch.url = "git+ssh://forgejo@git.ashisgreat.xyz:2222/penal-colony/gosearch.git";
|
||||
```
|
||||
|
||||
### 2. Import Module
|
||||
|
||||
```nix
|
||||
# configuration.nix
|
||||
imports = [
|
||||
...
|
||||
inputs.gosearch.nixosModules.default
|
||||
];
|
||||
```
|
||||
|
||||
### 3. Service Configuration
|
||||
|
||||
```nix
|
||||
# configuration.nix
|
||||
services.kafka = {
|
||||
enable = true;
|
||||
port = 8889;
|
||||
baseUrl = "https://search2.ashisgreat.xyz";
|
||||
config = config.sops.templates."kafka-config.toml".path;
|
||||
};
|
||||
```
|
||||
|
||||
### 4. SOPS Template
|
||||
|
||||
```nix
|
||||
# configuration.nix
|
||||
sops.templates."kafka-config.toml" = {
|
||||
content = ''
|
||||
[server]
|
||||
port = 8889
|
||||
http_timeout = "10s"
|
||||
base_url = "https://search2.ashisgreat.xyz"
|
||||
|
||||
[engines]
|
||||
local_ported = ["wikipedia", "arxiv", "crossref", "braveapi", "qwant", "duckduckgo", "github", "reddit", "bing"]
|
||||
|
||||
[engines.brave]
|
||||
api_key = "${config.sops.placeholder.openclaw_brave_api_key}"
|
||||
|
||||
[cache]
|
||||
address = "127.0.0.1:6379"
|
||||
db = 1
|
||||
default_ttl = "5m"
|
||||
|
||||
[cors]
|
||||
allowed_origins = ["*"]
|
||||
|
||||
[rate_limit]
|
||||
requests = 30
|
||||
window = "1m"
|
||||
cleanup_interval = "5m"
|
||||
|
||||
[global_rate_limit]
|
||||
requests = 0
|
||||
window = "1m"
|
||||
|
||||
[burst_rate_limit]
|
||||
burst = 0
|
||||
burst_window = "5s"
|
||||
sustained = 0
|
||||
sustained_window = "1m"
|
||||
'';
|
||||
};
|
||||
```
|
||||
|
||||
### 5. Systemd Hardening
|
||||
|
||||
```nix
|
||||
# configuration.nix
|
||||
systemd.services.kafka.serviceConfig = {
|
||||
# Capability bounds
|
||||
CapabilityBoundingSet = [ "" ];
|
||||
AmbientCapabilities = [ "" ];
|
||||
|
||||
# Filesystem
|
||||
ProtectSystem = "strict";
|
||||
ProtectHome = true;
|
||||
ReadWritePaths = [ "/var/lib/kafka" ];
|
||||
PrivateTmp = true;
|
||||
|
||||
# Network
|
||||
PrivateDevices = true;
|
||||
RestrictAddressFamilies = [ "AF_INET" "AF_INET6" "AF_UNIX" ];
|
||||
|
||||
# Process isolation
|
||||
ProtectProc = "invisible";
|
||||
ProcSubset = "pid";
|
||||
NoNewPrivileges = true;
|
||||
ProtectClock = true;
|
||||
ProtectHostname = true;
|
||||
|
||||
# System call filtering
|
||||
SystemCallFilter = [ "@system-service" "~@privileged" ];
|
||||
SystemCallArchitectures = "native";
|
||||
|
||||
# Memory
|
||||
MemoryDenyWriteExecute = true;
|
||||
RestrictRealtime = true;
|
||||
RestrictSUIDSGID = true;
|
||||
|
||||
# Resource limits
|
||||
RestrictNamespaces = true;
|
||||
LockPersonality = true;
|
||||
RemoveIPC = true;
|
||||
};
|
||||
```
|
||||
|
||||
### 6. Nginx Configuration
|
||||
|
||||
Add to `myModules.nginx.domains`:
|
||||
|
||||
```nix
|
||||
"search2.ashisgreat.xyz" = {
|
||||
port = 8889;
|
||||
};
|
||||
```
|
||||
|
||||
### 7. Valkey Host Port Forward
|
||||
|
||||
Since gosearch runs as a systemd service (not in podman network), add a host port forward for Valkey:
|
||||
|
||||
```nix
|
||||
# modules/searxng.nix - modify valkey container
|
||||
virtualisation.oci-containers.containers."searxng-valkey" = {
|
||||
...
|
||||
ports = [ "127.0.0.1:6379:6379" ];
|
||||
};
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Network Exposure**: Service binds to localhost only; Nginx is the only external access
|
||||
2. **Secrets Management**: Brave API key from existing SOPS secret
|
||||
3. **Cache Isolation**: Uses DB index 1 (SearXNG uses 0)
|
||||
4. **Process Isolation**: Comprehensive systemd hardening applied
|
||||
5. **Supply Chain**: Binary built from source via Nix flakes
|
||||
|
||||
## Files to Modify
|
||||
|
||||
1. `flake.nix` - Add gosearch input
|
||||
2. `configuration.nix` - Import module, configure service, add SOPS template, add Nginx domain, add systemd hardening
|
||||
3. `modules/searxng.nix` - Add Valkey host port forward
|
||||
4. `secrets/secrets.yaml` - No changes needed (reusing existing Brave key)
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
- [ ] Configuration builds: `nixos-rebuild build`
|
||||
- [ ] Service starts: `systemctl status kafka`
|
||||
- [ ] Service is hardened: `systemctl show kafka | grep -i protect`
|
||||
- [ ] Web interface loads: `curl -I http://localhost:8889`
|
||||
- [ ] Nginx proxy works: `curl -I https://search2.ashisgreat.xyz`
|
||||
- [ ] Valkey connection works: check service logs
|
||||
- [ ] Search returns results
|
||||
- [ ] Health check: `curl http://localhost:8889/healthz`
|
||||
Loading…
Add table
Add a link
Reference in a new issue