docs: add gosearch integration design spec

Add comprehensive design document for integrating gosearch (kafka)
metasearch engine into the NixOS VPS configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
ashisgreat22 2026-03-21 21:42:54 +00:00
parent bd3295e918
commit ea372d056e

View file

@ -0,0 +1,210 @@
# gosearch Integration Design
**Date:** 2026-03-21
**Status:** Approved
**Author:** Claude Code
## Overview
Integrate [gosearch](https://git.ashisgreat.xyz/penal-colony/gosearch) (kafka), a privacy-respecting Go-based metasearch engine, into the NixOS VPS configuration. The service will run as a hardened systemd service, connect to the existing Valkey cache, and be exposed via Nginx reverse proxy.
## Requirements
- Deploy gosearch using its native NixOS module
- Expose on `search2.ashisgreat.xyz`
- Connect to existing Valkey instance (shared with SearXNG)
- Enable Brave Search with API key
- Run on port `8889` (localhost only)
- Harden the service following systemd security best practices
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ User Request Flow │
│ Browser → search2.ashisgreat.xyz → Nginx → localhost:8889 │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ systemd kafka service │
│ - User: kafka | Port: 8889 | Hardened systemd │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Configuration Layer │
│ /etc/kafka/config.toml (generated via sops template) │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Dependencies │
│ - Valkey: 127.0.0.1:6379, DB index 1 │
│ - Brave API: from SOPS │
└─────────────────────────────────────────────────────────────┘
```
## Implementation
### 1. Add Flake Input
```nix
# flake.nix
inputs.gosearch.url = "git+ssh://forgejo@git.ashisgreat.xyz:2222/penal-colony/gosearch.git";
```
### 2. Import Module
```nix
# configuration.nix
imports = [
...
inputs.gosearch.nixosModules.default
];
```
### 3. Service Configuration
```nix
# configuration.nix
services.kafka = {
enable = true;
port = 8889;
baseUrl = "https://search2.ashisgreat.xyz";
config = config.sops.templates."kafka-config.toml".path;
};
```
### 4. SOPS Template
```nix
# configuration.nix
sops.templates."kafka-config.toml" = {
content = ''
[server]
port = 8889
http_timeout = "10s"
base_url = "https://search2.ashisgreat.xyz"
[engines]
local_ported = ["wikipedia", "arxiv", "crossref", "braveapi", "qwant", "duckduckgo", "github", "reddit", "bing"]
[engines.brave]
api_key = "${config.sops.placeholder.openclaw_brave_api_key}"
[cache]
address = "127.0.0.1:6379"
db = 1
default_ttl = "5m"
[cors]
allowed_origins = ["*"]
[rate_limit]
requests = 30
window = "1m"
cleanup_interval = "5m"
[global_rate_limit]
requests = 0
window = "1m"
[burst_rate_limit]
burst = 0
burst_window = "5s"
sustained = 0
sustained_window = "1m"
'';
};
```
### 5. Systemd Hardening
```nix
# configuration.nix
systemd.services.kafka.serviceConfig = {
# Capability bounds
CapabilityBoundingSet = [ "" ];
AmbientCapabilities = [ "" ];
# Filesystem
ProtectSystem = "strict";
ProtectHome = true;
ReadWritePaths = [ "/var/lib/kafka" ];
PrivateTmp = true;
# Network
PrivateDevices = true;
RestrictAddressFamilies = [ "AF_INET" "AF_INET6" "AF_UNIX" ];
# Process isolation
ProtectProc = "invisible";
ProcSubset = "pid";
NoNewPrivileges = true;
ProtectClock = true;
ProtectHostname = true;
# System call filtering
SystemCallFilter = [ "@system-service" "~@privileged" ];
SystemCallArchitectures = "native";
# Memory
MemoryDenyWriteExecute = true;
RestrictRealtime = true;
RestrictSUIDSGID = true;
# Resource limits
RestrictNamespaces = true;
LockPersonality = true;
RemoveIPC = true;
};
```
### 6. Nginx Configuration
Add to `myModules.nginx.domains`:
```nix
"search2.ashisgreat.xyz" = {
port = 8889;
};
```
### 7. Valkey Host Port Forward
Since gosearch runs as a systemd service (not in podman network), add a host port forward for Valkey:
```nix
# modules/searxng.nix - modify valkey container
virtualisation.oci-containers.containers."searxng-valkey" = {
...
ports = [ "127.0.0.1:6379:6379" ];
};
```
## Security Considerations
1. **Network Exposure**: Service binds to localhost only; Nginx is the only external access
2. **Secrets Management**: Brave API key from existing SOPS secret
3. **Cache Isolation**: Uses DB index 1 (SearXNG uses 0)
4. **Process Isolation**: Comprehensive systemd hardening applied
5. **Supply Chain**: Binary built from source via Nix flakes
## Files to Modify
1. `flake.nix` - Add gosearch input
2. `configuration.nix` - Import module, configure service, add SOPS template, add Nginx domain, add systemd hardening
3. `modules/searxng.nix` - Add Valkey host port forward
4. `secrets/secrets.yaml` - No changes needed (reusing existing Brave key)
## Testing Checklist
- [ ] Configuration builds: `nixos-rebuild build`
- [ ] Service starts: `systemctl status kafka`
- [ ] Service is hardened: `systemctl show kafka | grep -i protect`
- [ ] Web interface loads: `curl -I http://localhost:8889`
- [ ] Nginx proxy works: `curl -I https://search2.ashisgreat.xyz`
- [ ] Valkey connection works: check service logs
- [ ] Search returns results
- [ ] Health check: `curl http://localhost:8889/healthz`