# gosearch Integration Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Integrate gosearch (kafka) metasearch engine into the NixOS VPS configuration as a hardened systemd service with Nginx reverse proxy and Valkey caching.

**Architecture:** Add gosearch as a flake input, import its native NixOS module, configure service to run on localhost:8889, connect to shared Valkey cache (DB index 1), and expose via Nginx at search2.ashisgreat.xyz.

**Tech Stack:** NixOS flakes, systemd, Podman (for Valkey), Nginx, SOPS for secrets

---

## File Structure

**Files to create:**
- None (all modifications to existing files)

**Files to modify:**
| File | Purpose |
|------|---------|
| `flake.nix` | Add gosearch flake input |
| `configuration.nix` | Import kafka module, configure service, add SOPS template, add systemd hardening, add nginx domain |
| `modules/searxng.nix` | Add Valkey host port forward for gosearch access |

---

## Task 1: Add gosearch as Flake Input

**Files:**
- Modify: `flake.nix`

- [ ] **Step 1: Add gosearch input to flake.nix**

The gosearch input goes in the `inputs` attribute set. Place it alphabetically after existing inputs.

```nix
inputs = {
  nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  sops-nix.url = "github:Mic92/sops-nix";
  gosearch.url = "git+ssh://forgejo@git.ashisgreat.xyz:2222/penal-colony/gosearch.git";
  openclaw-superpowers = {
    url = "github:ArchieIndian/openclaw-superpowers";
    flake = false;
  };
};
```

- [ ] **Step 2: Verify flake update works**

Run: `nix flake update --commit-lockfile`
Expected: Lockfile updates successfully, no errors

**Note:** The gosearch flake has a placeholder `vendorHash = ""` in its flake.nix. During the first `nixos-rebuild build`, Nix will fail with the correct hash. This is expected — you'll need to copy that hash into the gosearch flake and run `nix flake update` again. Alternatively, if gosearch's repository has been updated with the correct hash, this step will work immediately.

- [ ] **Step 3: Commit**

```bash
git add flake.nix flake.lock
git commit -m "feat(gosearch): add gosearch flake input"
```

---

## Task 2: Import kafka Module and Configure Basic Service

**Files:**
- Modify: `configuration.nix`

- [ ] **Step 1: Import the kafka module**

Add to the `imports` array in `configuration.nix` (around line 3, after `./hardware-configuration.nix`):

```nix
imports = [
  ./hardware-configuration.nix
  inputs.gosearch.nixosModules.default  # Add this line
];
```

- [ ] **Step 2: Add basic service configuration**

Add after the OpenWeb UI configuration block (around line 240, before `# === Backups ===`):

```nix
# === gosearch (kafka) Metasearch Engine ===
services.kafka = {
  enable = true;
  port = 8889;
  baseUrl = "https://search2.ashisgreat.xyz";
  config = config.sops.templates."kafka-config.toml".path;
};
```

- [ ] **Step 3: Test configuration build**

Run: `nixos-rebuild build --flake .#nixos`
Expected: Build succeeds (may have warnings about missing config file, that's expected at this stage)

- [ ] **Step 4: Commit**

```bash
git add configuration.nix
git commit -m "feat(gosearch): import kafka module and configure basic service"
```

---

## Task 3: Create SOPS Config Template

**Files:**
- Modify: `configuration.nix`

- [ ] **Step 1: Add kafka-config.toml SOPS template**

Add the SOPS template after the OpenClaw SOPS templates block (around line 175):

```nix
# gosearch/kafka config template
sops.templates."kafka-config.toml" = {
  content = ''
    [server]
    port = 8889
    http_timeout = "10s"
    base_url = "https://search2.ashisgreat.xyz"

    [engines]
    local_ported = ["wikipedia", "arxiv", "crossref", "braveapi", "qwant", "duckduckgo", "github", "reddit", "bing"]

    [engines.brave]
    api_key = "${config.sops.placeholder.openclaw_brave_api_key}"

    [cache]
    address = "127.0.0.1:6379"
    db = 1
    default_ttl = "5m"

    [cors]
    allowed_origins = ["*"]

    [rate_limit]
    requests = 30
    window = "1m"
    cleanup_interval = "5m"

    [global_rate_limit]
    requests = 0
    window = "1m"

    [burst_rate_limit]
    burst = 0
    burst_window = "5s"
    sustained = 0
    sustained_window = "1m"
  '';
};
```

- [ ] **Step 2: Test configuration build**

Run: `nixos-rebuild build --flake .#nixos`
Expected: Build succeeds

- [ ] **Step 3: Commit**

```bash
git add configuration.nix
git commit -m "feat(gosearch): add SOPS config template for kafka"
```

---

## Task 4: Add Systemd Hardening

**Files:**
- Modify: `configuration.nix`

- [ ] **Step 1: Add systemd service hardening**

Add the hardening configuration after the kafka service block (added in Task 2):

```nix
# Hardening for kafka service
systemd.services.kafka.serviceConfig = {
  # Capability bounds
  CapabilityBoundingSet = [ "" ];
  AmbientCapabilities = [ "" ];

  # Filesystem
  ProtectSystem = "strict";
  ProtectHome = true;
  ReadWritePaths = [ "/var/lib/kafka" ];
  PrivateTmp = true;

  # Network
  PrivateDevices = true;
  RestrictAddressFamilies = [ "AF_INET" "AF_INET6" "AF_UNIX" ];

  # Process isolation
  ProtectProc = "invisible";
  ProcSubset = "pid";
  NoNewPrivileges = true;
  ProtectClock = true;
  ProtectHostname = true;

  # System call filtering
  SystemCallFilter = [ "@system-service" "~@privileged" ];
  SystemCallArchitectures = "native";

  # Memory
  MemoryDenyWriteExecute = true;
  RestrictRealtime = true;
  RestrictSUIDSGID = true;

  # Resource limits
  RestrictNamespaces = true;
  LockPersonality = true;
  RemoveIPC = true;
};
```

- [ ] **Step 2: Test configuration build**

Run: `nixos-rebuild build --flake .#nixos`
Expected: Build succeeds

- [ ] **Step 3: Commit**

```bash
git add configuration.nix
git commit -m "feat(gosearch): add systemd hardening for kafka service"
```

---

## Task 5: Add Nginx Domain Configuration

**Files:**
- Modify: `configuration.nix`

- [ ] **Step 1: Add search2 domain to nginx configuration**

Find the `myModules.nginx.domains` section (around line 127) and add the new domain after the search.ashisgreat.xyz entry:

```nix
myModules.nginx = {
  enable = true;
  email = "info@ashisgreat.xyz";
  rateLimit = {
    enable = true;
    requests = 10;
    burst = 20;
  };
  domains = {
    "search.ashisgreat.xyz" = {
      port = 8888;
      contentSecurityPolicy = null;
      rateLimit.requests = 20;
      rateLimit.burst = 40;
    };
    "search2.ashisgreat.xyz" = {
      port = 8889;
    };
  };
};
```

- [ ] **Step 2: Test configuration build**

Run: `nixos-rebuild build --flake .#nixos`
Expected: Build succeeds

- [ ] **Step 3: Commit**

```bash
git add configuration.nix
git commit -m "feat(gosearch): add search2 domain to nginx"
```

---

## Task 6: Add Valkey Host Port Forward

**Files:**
- Modify: `modules/searxng.nix`

- [ ] **Step 1: Add port mapping to searxng-valkey container**

Find the `searxng-valkey` container definition in `modules/searxng.nix` (around line 94) and add the `ports` option:

```nix
virtualisation.oci-containers.containers."searxng-valkey" = {
  image = "docker.io/valkey/valkey:alpine";
  cmd = [ "valkey-server" "--save" "" "--appendonly" "no" ];
  extraOptions = [
    "--network=searxng-net"
    "--network-alias=valkey"
  ];
  ports = [ "127.0.0.1:6379:6379" ];  # Add this line for gosearch access
};
```

- [ ] **Step 2: Test configuration build**

Run: `nixos-rebuild build --flake .#nixos`
Expected: Build succeeds

- [ ] **Step 3: Commit**

```bash
git add modules/searxng.nix
git commit -m "feat(gosearch): add Valkey host port forward for local access"
```

---

## Task 7: Deploy and Verify

**Files:**
- None (deployment and testing)

- [ ] **Step 1: Dry-run rebuild to verify configuration**

Run: `nixos-rebuild build --flake .#nixos`
Expected: Build completes successfully with no errors

- [ ] **Step 2: Apply configuration**

Run: `sudo nixos-rebuild switch --flake .#nixos`
Expected: Configuration applies successfully, services start

- [ ] **Step 3: Verify kafka service is running**

Run: `systemctl status kafka`
Expected: Service shows as `active (running)`

- [ ] **Step 4: Verify systemd hardening is applied**

Run: `systemctl show kafka | grep -E 'ProtectSystem|ProtectHome|PrivateTmp|NoNewPrivileges'`
Expected: All hardening options show `yes`

- [ ] **Step 5: Verify web interface responds locally**

Run: `curl -I http://localhost:8889`
Expected: HTTP 200 OK response

- [ ] **Step 6: Verify health endpoint**

Run: `curl http://localhost:8889/healthz`
Expected: `OK` response

- [ ] **Step 7: Verify nginx proxy works**

Run: `curl -I https://search2.ashisgreat.xyz`
Expected: HTTP 200 OK response via nginx

- [ ] **Step 8: Test search functionality**

Run: `curl -s "https://search2.ashisgreat.xyz/search?q=nixos&format=json" | jq -e '.results | length > 0'`
Expected: Output is `true` (indicating search returned non-empty results)

**Alternative (without jq):** Run: `curl -s "https://search2.ashisgreat.xyz/search?q=nixos&format=json" | grep -o '"results":\s*\[' | wc -l`
Expected: Output is `1` or more (results array is present)

- [ ] **Step 9: Check kafka logs for Valkey connection**

Run: `journalctl -u kafka -n 50`
Expected: No errors about Valkey connection, cache operations visible

- [ ] **Step 10: Commit final deployment**

```bash
git add -A
git commit -m "feat(gosearch): complete gosearch integration - service running and verified"
```

---

## Rollback Procedure (if issues occur)

If deployment fails:

```bash
# Rollback to previous generation
sudo nixos-rebuild switch --rollback

# Or list generations and select a specific one
sudo nixos-rebuild list-generations
sudo nixos-rebuild switch --profile /nix/var/nix/profiles/system --rollback
```

---

## Verification Checklist

After completing all tasks:

- [ ] `nixos-rebuild build` succeeds without errors
- [ ] `systemctl status kafka` shows active (running)
- [ ] `systemctl show kafka | grep ProtectSystem` returns `ProtectSystem=yes`
- [ ] `curl -I http://localhost:8889` returns HTTP 200
- [ ] `curl -I https://search2.ashisgreat.xyz` returns HTTP 200
- [ ] `curl "https://search2.ashisgreat.xyz/search?q=nixos&format=json"` returns results
- [ ] `journalctl -u kafka` shows no Valkey connection errors
- [ ] `curl http://localhost:8889/healthz` returns `OK`