nixos-vps/docs/superpowers/plans/2026-03-21-gosearch-integration.md

gosearch Integration Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Integrate gosearch (kafka) metasearch engine into the NixOS VPS configuration as a hardened systemd service with Nginx reverse proxy and Valkey caching.

Architecture: Add gosearch as a flake input, import its native NixOS module, configure service to run on localhost:8889, connect to shared Valkey cache (DB index 1), and expose via Nginx at search2.ashisgreat.xyz.

Tech Stack: NixOS flakes, systemd, Podman (for Valkey), Nginx, SOPS for secrets

---

File Structure

Files to create:

• None (all modifications to existing files)

Files to modify:

File	Purpose
flake.nix	Add gosearch flake input
configuration.nix	Import kafka module, configure service, add SOPS template, add systemd hardening, add nginx domain
modules/searxng.nix	Add Valkey host port forward for gosearch access

---

Task 1: Add gosearch as Flake Input

Files:

• Modify: flake.nix

• Step 1: Add gosearch input to flake.nix

The gosearch input goes in the inputs attribute set. Place it alphabetically after existing inputs.

inputs = {
  nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  sops-nix.url = "github:Mic92/sops-nix";
  gosearch.url = "git+ssh://forgejo@git.ashisgreat.xyz:2222/penal-colony/gosearch.git";
  openclaw-superpowers = {
    url = "github:ArchieIndian/openclaw-superpowers";
    flake = false;
  };
};

• Step 2: Verify flake update works

Run: nix flake update --commit-lockfile Expected: Lockfile updates successfully, no errors

Note: The gosearch flake has a placeholder vendorHash = "" in its flake.nix. During the first nixos-rebuild build, Nix will fail with the correct hash. This is expected — you'll need to copy that hash into the gosearch flake and run nix flake update again. Alternatively, if gosearch's repository has been updated with the correct hash, this step will work immediately.

• Step 3: Commit

git add flake.nix flake.lock
git commit -m "feat(gosearch): add gosearch flake input"

---

Task 2: Import kafka Module and Configure Basic Service

Files:

• Modify: configuration.nix

• Step 1: Import the kafka module

Add to the imports array in configuration.nix (around line 3, after ./hardware-configuration.nix):

imports = [
  ./hardware-configuration.nix
  inputs.gosearch.nixosModules.default  # Add this line
];

• Step 2: Add basic service configuration

Add after the OpenWeb UI configuration block (around line 240, before # === Backups ===):

# === gosearch (kafka) Metasearch Engine ===
services.kafka = {
  enable = true;
  port = 8889;
  baseUrl = "https://search2.ashisgreat.xyz";
  config = config.sops.templates."kafka-config.toml".path;
};

• Step 3: Test configuration build

Run: nixos-rebuild build --flake .#nixos Expected: Build succeeds (may have warnings about missing config file, that's expected at this stage)

• Step 4: Commit

git add configuration.nix
git commit -m "feat(gosearch): import kafka module and configure basic service"

---

Task 3: Create SOPS Config Template

Files:

• Modify: configuration.nix

• Step 1: Add kafka-config.toml SOPS template

Add the SOPS template after the OpenClaw SOPS templates block (around line 175):

# gosearch/kafka config template
sops.templates."kafka-config.toml" = {
  content = ''
    [server]
    port = 8889
    http_timeout = "10s"
    base_url = "https://search2.ashisgreat.xyz"

    [engines]
    local_ported = ["wikipedia", "arxiv", "crossref", "braveapi", "qwant", "duckduckgo", "github", "reddit", "bing"]

    [engines.brave]
    api_key = "${config.sops.placeholder.openclaw_brave_api_key}"

    [cache]
    address = "127.0.0.1:6379"
    db = 1
    default_ttl = "5m"

    [cors]
    allowed_origins = ["*"]

    [rate_limit]
    requests = 30
    window = "1m"
    cleanup_interval = "5m"

    [global_rate_limit]
    requests = 0
    window = "1m"

    [burst_rate_limit]
    burst = 0
    burst_window = "5s"
    sustained = 0
    sustained_window = "1m"
  '';
};

• Step 2: Test configuration build

Run: nixos-rebuild build --flake .#nixos Expected: Build succeeds

• Step 3: Commit

git add configuration.nix
git commit -m "feat(gosearch): add SOPS config template for kafka"

---

Task 4: Add Systemd Hardening

Files:

• Modify: configuration.nix

• Step 1: Add systemd service hardening

Add the hardening configuration after the kafka service block (added in Task 2):

# Hardening for kafka service
systemd.services.kafka.serviceConfig = {
  # Capability bounds
  CapabilityBoundingSet = [ "" ];
  AmbientCapabilities = [ "" ];

  # Filesystem
  ProtectSystem = "strict";
  ProtectHome = true;
  ReadWritePaths = [ "/var/lib/kafka" ];
  PrivateTmp = true;

  # Network
  PrivateDevices = true;
  RestrictAddressFamilies = [ "AF_INET" "AF_INET6" "AF_UNIX" ];

  # Process isolation
  ProtectProc = "invisible";
  ProcSubset = "pid";
  NoNewPrivileges = true;
  ProtectClock = true;
  ProtectHostname = true;

  # System call filtering
  SystemCallFilter = [ "@system-service" "~@privileged" ];
  SystemCallArchitectures = "native";

  # Memory
  MemoryDenyWriteExecute = true;
  RestrictRealtime = true;
  RestrictSUIDSGID = true;

  # Resource limits
  RestrictNamespaces = true;
  LockPersonality = true;
  RemoveIPC = true;
};

• Step 2: Test configuration build

Run: nixos-rebuild build --flake .#nixos Expected: Build succeeds

• Step 3: Commit

git add configuration.nix
git commit -m "feat(gosearch): add systemd hardening for kafka service"

---

Task 5: Add Nginx Domain Configuration

Files:

• Modify: configuration.nix

• Step 1: Add search2 domain to nginx configuration

Find the myModules.nginx.domains section (around line 127) and add the new domain after the search.ashisgreat.xyz entry:

myModules.nginx = {
  enable = true;
  email = "info@ashisgreat.xyz";
  rateLimit = {
    enable = true;
    requests = 10;
    burst = 20;
  };
  domains = {
    "search.ashisgreat.xyz" = {
      port = 8888;
      contentSecurityPolicy = null;
      rateLimit.requests = 20;
      rateLimit.burst = 40;
    };
    "search2.ashisgreat.xyz" = {
      port = 8889;
    };
  };
};

• Step 2: Test configuration build

Run: nixos-rebuild build --flake .#nixos Expected: Build succeeds

• Step 3: Commit

git add configuration.nix
git commit -m "feat(gosearch): add search2 domain to nginx"

---

Task 6: Add Valkey Host Port Forward

Files:

• Modify: modules/searxng.nix

• Step 1: Add port mapping to searxng-valkey container

Find the searxng-valkey container definition in modules/searxng.nix (around line 94) and add the ports option:

virtualisation.oci-containers.containers."searxng-valkey" = {
  image = "docker.io/valkey/valkey:alpine";
  cmd = [ "valkey-server" "--save" "" "--appendonly" "no" ];
  extraOptions = [
    "--network=searxng-net"
    "--network-alias=valkey"
  ];
  ports = [ "127.0.0.1:6379:6379" ];  # Add this line for gosearch access
};

• Step 2: Test configuration build

Run: nixos-rebuild build --flake .#nixos Expected: Build succeeds

• Step 3: Commit

git add modules/searxng.nix
git commit -m "feat(gosearch): add Valkey host port forward for local access"

---

Task 7: Deploy and Verify

Files:

• None (deployment and testing)

• Step 1: Dry-run rebuild to verify configuration

Run: nixos-rebuild build --flake .#nixos Expected: Build completes successfully with no errors

• Step 2: Apply configuration

Run: sudo nixos-rebuild switch --flake .#nixos Expected: Configuration applies successfully, services start

• Step 3: Verify kafka service is running

Run: systemctl status kafka Expected: Service shows as active (running)

• Step 4: Verify systemd hardening is applied

Run: systemctl show kafka | grep -E 'ProtectSystem|ProtectHome|PrivateTmp|NoNewPrivileges' Expected: All hardening options show yes

• Step 5: Verify web interface responds locally

Run: curl -I http://localhost:8889 Expected: HTTP 200 OK response

• Step 6: Verify health endpoint

Run: curl http://localhost:8889/healthz Expected: OK response

• Step 7: Verify nginx proxy works

Run: curl -I https://search2.ashisgreat.xyz Expected: HTTP 200 OK response via nginx

• Step 8: Test search functionality

Run: curl -s "https://search2.ashisgreat.xyz/search?q=nixos&format=json" | jq -e '.results | length > 0' Expected: Output is true (indicating search returned non-empty results)

Alternative (without jq): Run: curl -s "https://search2.ashisgreat.xyz/search?q=nixos&format=json" | grep -o '"results":\s*\[' | wc -l Expected: Output is 1 or more (results array is present)

• Step 9: Check kafka logs for Valkey connection

Run: journalctl -u kafka -n 50 Expected: No errors about Valkey connection, cache operations visible

• Step 10: Commit final deployment

git add -A
git commit -m "feat(gosearch): complete gosearch integration - service running and verified"

---

Rollback Procedure (if issues occur)

If deployment fails:

# Rollback to previous generation
sudo nixos-rebuild switch --rollback

# Or list generations and select a specific one
sudo nixos-rebuild list-generations
sudo nixos-rebuild switch --profile /nix/var/nix/profiles/system --rollback

---

Verification Checklist

After completing all tasks:

• nixos-rebuild build succeeds without errors
• systemctl status kafka shows active (running)
• systemctl show kafka | grep ProtectSystem returns ProtectSystem=yes
• curl -I http://localhost:8889 returns HTTP 200
• curl -I https://search2.ashisgreat.xyz returns HTTP 200
• curl "https://search2.ashisgreat.xyz/search?q=nixos&format=json" returns results
• journalctl -u kafka shows no Valkey connection errors
• curl http://localhost:8889/healthz returns OK