Add comprehensive design document for integrating gosearch (kafka) metasearch engine into the NixOS VPS configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
6.7 KiB
6.7 KiB
gosearch Integration Design
Date: 2026-03-21 Status: Approved Author: Claude Code
Overview
Integrate gosearch (kafka), a privacy-respecting Go-based metasearch engine, into the NixOS VPS configuration. The service will run as a hardened systemd service, connect to the existing Valkey cache, and be exposed via Nginx reverse proxy.
Requirements
- Deploy gosearch using its native NixOS module
- Expose on
search2.ashisgreat.xyz - Connect to existing Valkey instance (shared with SearXNG)
- Enable Brave Search with API key
- Run on port
8889(localhost only) - Harden the service following systemd security best practices
Architecture
┌─────────────────────────────────────────────────────────────┐
│ User Request Flow │
│ Browser → search2.ashisgreat.xyz → Nginx → localhost:8889 │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ systemd kafka service │
│ - User: kafka | Port: 8889 | Hardened systemd │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Configuration Layer │
│ /etc/kafka/config.toml (generated via sops template) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Dependencies │
│ - Valkey: 127.0.0.1:6379, DB index 1 │
│ - Brave API: from SOPS │
└─────────────────────────────────────────────────────────────┘
Implementation
1. Add Flake Input
# flake.nix
inputs.gosearch.url = "git+ssh://forgejo@git.ashisgreat.xyz:2222/penal-colony/gosearch.git";
2. Import Module
# configuration.nix
imports = [
...
inputs.gosearch.nixosModules.default
];
3. Service Configuration
# configuration.nix
services.kafka = {
enable = true;
port = 8889;
baseUrl = "https://search2.ashisgreat.xyz";
config = config.sops.templates."kafka-config.toml".path;
};
4. SOPS Template
# configuration.nix
sops.templates."kafka-config.toml" = {
content = ''
[server]
port = 8889
http_timeout = "10s"
base_url = "https://search2.ashisgreat.xyz"
[engines]
local_ported = ["wikipedia", "arxiv", "crossref", "braveapi", "qwant", "duckduckgo", "github", "reddit", "bing"]
[engines.brave]
api_key = "${config.sops.placeholder.openclaw_brave_api_key}"
[cache]
address = "127.0.0.1:6379"
db = 1
default_ttl = "5m"
[cors]
allowed_origins = ["*"]
[rate_limit]
requests = 30
window = "1m"
cleanup_interval = "5m"
[global_rate_limit]
requests = 0
window = "1m"
[burst_rate_limit]
burst = 0
burst_window = "5s"
sustained = 0
sustained_window = "1m"
'';
};
5. Systemd Hardening
# configuration.nix
systemd.services.kafka.serviceConfig = {
# Capability bounds
CapabilityBoundingSet = [ "" ];
AmbientCapabilities = [ "" ];
# Filesystem
ProtectSystem = "strict";
ProtectHome = true;
ReadWritePaths = [ "/var/lib/kafka" ];
PrivateTmp = true;
# Network
PrivateDevices = true;
RestrictAddressFamilies = [ "AF_INET" "AF_INET6" "AF_UNIX" ];
# Process isolation
ProtectProc = "invisible";
ProcSubset = "pid";
NoNewPrivileges = true;
ProtectClock = true;
ProtectHostname = true;
# System call filtering
SystemCallFilter = [ "@system-service" "~@privileged" ];
SystemCallArchitectures = "native";
# Memory
MemoryDenyWriteExecute = true;
RestrictRealtime = true;
RestrictSUIDSGID = true;
# Resource limits
RestrictNamespaces = true;
LockPersonality = true;
RemoveIPC = true;
};
6. Nginx Configuration
Add to myModules.nginx.domains:
"search2.ashisgreat.xyz" = {
port = 8889;
};
7. Valkey Host Port Forward
Since gosearch runs as a systemd service (not in podman network), add a host port forward for Valkey:
# modules/searxng.nix - modify valkey container
virtualisation.oci-containers.containers."searxng-valkey" = {
...
ports = [ "127.0.0.1:6379:6379" ];
};
Security Considerations
- Network Exposure: Service binds to localhost only; Nginx is the only external access
- Secrets Management: Brave API key from existing SOPS secret
- Cache Isolation: Uses DB index 1 (SearXNG uses 0)
- Process Isolation: Comprehensive systemd hardening applied
- Supply Chain: Binary built from source via Nix flakes
Files to Modify
flake.nix- Add gosearch inputconfiguration.nix- Import module, configure service, add SOPS template, add Nginx domain, add systemd hardeningmodules/searxng.nix- Add Valkey host port forwardsecrets/secrets.yaml- No changes needed (reusing existing Brave key)
Testing Checklist
- Configuration builds:
nixos-rebuild build - Service starts:
systemctl status kafka - Service is hardened:
systemctl show kafka | grep -i protect - Web interface loads:
curl -I http://localhost:8889 - Nginx proxy works:
curl -I https://search2.ashisgreat.xyz - Valkey connection works: check service logs
- Search returns results
- Health check:
curl http://localhost:8889/healthz