Commit graph

61 commits

Author SHA1 Message Date
f7cece9648 feat: complete UI redesign — modern, clean search interface
- New CSS: complete design system with CSS variables, modern color palette
- Homepage: full-viewport hero with centered search, logo, tagline
- Result cards: rounded, shadowed, with favicons via Google Favicon API
- Layout: sidebar + results grid, responsive
- Typography: proper font stack, variable weights
- Settings panel: polished popover with animations
- Autocomplete: modern dropdown with keyboard nav
- Dark mode: full color palette via data-theme attribute
- Favicon: clean search icon SVG
2026-03-22 08:06:31 +00:00
f1436310eb fix: regexp.DotAll flag in google engine and Metadata field removal
Some checks failed
Build and Push Docker Image / build-and-push (push) Failing after 7s
Mirror to GitHub / mirror (push) Failing after 3s
Tests / test (push) Successful in 21s
- google.go: use inline (?s) flag instead of regexp.DotAll second arg
- youtube.go: remove Metadata field (not in MainResult contract)
- config_test.go: fix expected engine count from 9 to 11 (google+youtube)
2026-03-22 02:54:12 +00:00
b499db68f7 fix: use explicit if/else template dispatch instead of dynamic name
html/template requires template names to be string literals, not field
accesses. Use {{if eq .Template "videos"}} to branch and call the
appropriate template by literal name.
2026-03-22 02:46:28 +00:00
f0a65e2b8c fix: compute TemplateName in ResultView instead of using dynamic template function
Go html/template doesn't support function calls as template names in
{{template (func .Arg) .}}. Instead, precompute TemplateName in
FromResponse and use {{template .TemplateName .}} in the template.
2026-03-22 02:44:50 +00:00
7d3c82214b fix: move replace directive outside require block 2026-03-22 02:40:12 +00:00
8a4a606dd6 fix: replace x/net v0.52.0 (requires go 1.25) with v0.33.0 (go 1.21)
Use replace directive to force Go to use v0.33.0 instead of the
transitively-pulled v0.52.0, which requires Go 1.25.
2026-03-22 02:39:10 +00:00
841526276e fix: update goquery v1.9.0 checksum in go.sum 2026-03-22 02:33:01 +00:00
a03945b0e4 fix: downgrade goquery to v1.9.0 (v1.12.0 requires Go 1.25)
goquery v1.12.0 has a go.mod requirement of Go 1.25, which is not
released yet. Downgrade to v1.9.0 which works with Go 1.24.
2026-03-22 02:27:03 +00:00
281c327f60 fix: correct go.mod to 1.24 (go 1.25 does not exist)
Also use go-version-file in CI so go.mod and workflow stay in sync.
2026-03-22 02:19:12 +00:00
4a6559be62 fix: add Thumbnail field and video result template
MainResult: add Thumbnail field (used by YouTube, images, etc.)
video_item.html: new partial for video results with thumbnail display
views.go: add templateForResult func + video_item.html to template parse
results_inner.html: dispatch to video_item when Template="videos"
kafka.css: add .video-result flex layout with thumbnail styling
2026-03-22 02:06:41 +00:00
af23a63a73 remove settings-design-playground.html 2026-03-22 02:03:30 +00:00
a9ea99c104 Merge branch 'feat/youtube-engine' 2026-03-22 02:02:32 +00:00
b7f8a87edb feat: complete settings UI — popover, auto-save, theme, engines, mobile bottom-sheet
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:00:12 +01:00
277db9463e feat(settings): add hidden engines input to search forms 2026-03-22 03:00:12 +01:00
84777211f8 feat(settings): add gear trigger and panel markup to base template
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:00:12 +01:00
8e53a8b11d fix(settings): rewrite renderPanel to use panel body innerHTML, not document.body
- Replace document.body.innerHTML with panel.querySelector('.settings-popover-body').innerHTML
- Use theme buttons (.theme-btn) with icons instead of radio buttons
- Use .engine-toggle class for engine checkboxes in 2-column grid
- Include settings-notice paragraph for engine changes
- Use dropdowns for safe search and format with proper ids

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:00:12 +01:00
1906723859 fix(settings): re-render panel when last engine unchecked to enforce minimum
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:00:12 +01:00
2785b84939 feat(settings): add JS module for localStorage preferences and panel
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:00:12 +01:00
4fe78c69ce fix(settings): add html[data-theme=light] explicit light mode reset 2026-03-22 03:00:12 +01:00
11480dacdf feat(settings): add popover, toggle, and bottom-sheet CSS 2026-03-22 03:00:12 +01:00
4c52c77460 docs: fix plan issues — mobile FAB, RSS format, init wiring
Reviewed-by: plan-document-reviewer

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:00:12 +01:00
2c9d1c3543 docs: add settings UI implementation plan
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:00:12 +01:00
bfed53ae33 docs: add settings UI design spec
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 03:00:12 +01:00
a7f594b7fa feat: add YouTube engine with config file and env support
YouTube Data API v3 engine:
- Add YouTubeConfig to EnginesConfig with api_key field
- Add YOUTUBE_API_KEY env override
- Thread *config.Config through search service to factory
- Factory falls back to env vars if config fields are empty
- Update config.example.toml with youtube section

Also update default local_ported to include google and youtube.
2026-03-22 01:57:13 +00:00
1689cab9bd feat: add YouTube engine via Data API v3
Uses the official YouTube Data API v3. Requires YOUTUBE_API_KEY
environment variable (free from Google Cloud Console).

Returns video results with title, description, channel, publish
date, and thumbnail URL. Falls back gracefully if no API key.
2026-03-22 01:53:19 +00:00
31fdd5e06f Merge branch 'feat/google-engine', remote-tracking branch 'origin/main' 2026-03-22 01:35:20 +00:00
4be9cf2725 feat: add Google engine using GSA User-Agent scraping
SearXNG approach: use Google Search Appliance (GSA) User-Agent
pool — these are whitelisted enterprise identifiers Google trusts.

Key techniques:
- GSA User-Agent (iPhone OS + GSA/ version) instead of Chrome desktop
- CONSENT=YES+ cookie to bypass EU consent wall
- Parse /url?q= redirector URLs (unquote + strip &sa= params)
- div.MjjYud class for result containers (SearXNG selector)
- data-sncf divs for snippets
- detect sorry.google.com blocks
- Suggestions from ouy7Mc class cards
2026-03-22 01:29:46 +00:00
fcd9be16df refactor: remove SearXNG references and rename binary to kafka
- Rename cmd/searxng-go to cmd/kafka
- Remove all SearXNG references from source comments while keeping
  "SearXNG-compatible API" in user-facing docs
- Update binary paths in README, CLAUDE.md, and Dockerfile
- Update log message to "kafka starting"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:47:03 +01:00
4482cb4dde docs: update CLAUDE.md with autocomplete package and endpoint
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:27:10 +01:00
a2f8077669 feat: add autocomplete dropdown UI with keyboard nav
- Inline JS in base.html: debounced fetch from /autocompleter on keyup
- Keyboard nav: arrows to navigate, Enter to select, Esc to close
- Highlight matching prefix in suggestions
- Click to select and submit
- Dropdown positioned absolutely below search input
- Dark mode compatible via existing CSS variables
2026-03-22 00:20:43 +00:00
7a2ca8672e docs: add MIT license
Some checks failed
Build and Push Docker Image / build-and-push (push) Failing after 5m42s
Mirror to GitHub / mirror (push) Failing after 3s
Tests / test (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:18:22 +01:00
33e1e0c3eb chore: ignore result binary
Some checks failed
Mirror to GitHub / mirror (push) Has been cancelled
Build and Push Docker Image / build-and-push (push) Has been cancelled
Tests / test (push) Failing after 17s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:07:12 +01:00
9b280ad606 feat: add /autocompleter endpoint for search suggestions
Some checks failed
Mirror to GitHub / mirror (push) Waiting to run
Tests / test (push) Waiting to run
Build and Push Docker Image / build-and-push (push) Has been cancelled
Proxies to upstream SearXNG /autocompleter if configured, otherwise
falls back to Wikipedia OpenSearch API. Returns a JSON array of
suggestion strings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 01:06:25 +01:00
90810cb934 fix(flake): set vendorHash and skip tests in build
Some checks failed
Build and Push Docker Image / build-and-push (push) Has started running
Mirror to GitHub / mirror (push) Failing after 4s
Tests / test (push) Has been cancelled
vendorHash was empty, causing build failures. The hash was
obtained by running the build once and using the error output.

Tests are skipped because they require network access; CI runs
them via the test workflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 00:47:10 +01:00
0d3f3c19d7 fix: add missing engines to defaultPortedEngines
duckduckgo, github, reddit, and bing were registered in factory.go
and config.go but missing from planner.go, so they were silently
skipped when LOCAL_PORTED_ENGINES was not set.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 00:13:57 +01:00
b85a06cf11 ci: add test workflow
Runs go test -race -v ./... on push and PRs to main.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-22 00:11:25 +01:00
6001979d7f docs: add CLAUDE.md for Claude Code onboarding
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 23:52:23 +01:00
52395c7cd0 fix(ci): use GHCR_TOKEN for GitHub mirror
Some checks failed
Build and Push Docker Image / build-and-push (push) Failing after 9m27s
Mirror to GitHub / mirror (push) Failing after 3s
Reuse the same token secret for both Docker push and git mirror.
2026-03-21 21:05:43 +00:00
8a7b840b69 fix(ci): use raw git push for GitHub mirror
ad-m/github-push-action chokes on secret substitution in act_runner.
Use plain git commands instead — no third-party action needed.
2026-03-21 21:04:31 +00:00
6346fb7155 chore: update Go module path to github.com/metamorphosis-dev/kafka
Module path now matches the GitHub mirror location.
All internal imports updated across 35+ files.
2026-03-21 19:42:01 +00:00
9a6a40c817 fix(ci): rename GITHUB_TOKEN to GHCR_TOKEN for Forgejo compat
Forgejo reserves GITHUB_TOKEN as a protected secret name.
Use GHCR_USERNAME + GHCR_TOKEN instead.
2026-03-21 19:31:20 +00:00
fed5fa469e fix(ci): use full GitHub URLs for actions
data.forgejo.org is unreachable from the runner. Using https://github.com/
prefix for all action uses to fetch directly from GitHub instead of the
Forgejo marketplace mirror.
2026-03-21 19:26:45 +00:00
c761805b56 ci: add Docker build + push workflow
Forgejo Actions workflow that builds and pushes Docker images on:
- Push to main → tagged 'latest' + short SHA
- Tags matching 'v*' → semver tags (1.0.0, 1.0, latest)
- PRs → build only (no push)

Pushes to both GHCR and Docker Hub with GitHub Actions cache.

Required secrets:
- DOCKERHUB_USERNAME: Docker Hub username
- DOCKERHUB_PASSWORD: Docker Hub access token
- GITHUB_TOKEN: automatically provided by Forgejo (or set manually for GHCR)

Note: Requires a Forgejo Actions runner to be registered. See:
https://forgejo.org/docs/latest/user/actions/#runner-setup
2026-03-21 19:24:04 +00:00
e5295fa69d chore: rename project from gosearch to kafka
A search engine named after a man who proved answers don't exist.

Renamed everywhere user-facing:
- Brand name, UI titles, OpenSearch description, CSS filename
- Docker service name, NixOS module (services.kafka)
- Cache key prefix (kafka:), User-Agent strings (kafka/0.1)
- README, config.example.toml, flake.nix descriptions

Kept unchanged (internal):
- Go module path: github.com/ashie/gosearch
- Git repository URL: git.ashisgreat.xyz/penal-colony/gosearch
- Binary entrypoint: cmd/searxng-go
2026-03-21 19:20:47 +00:00
be7ba66a09 docs: complete README rewrite
- Updated to reflect all current features (9 engines, HTMX frontend, Valkey cache, 3-layer rate limiting, CORS, OpenSearch)
- Added quick start for binary, Docker Compose, and NixOS
- Documented all endpoints, API parameters, and response format
- Configuration reference with environment variable table
- Engine table with source and notes
- ASCII architecture diagram
- Docker and NixOS deployment sections
2026-03-21 18:51:14 +00:00
13040268d6 feat: add global and burst rate limiters
Three layers of rate limiting, all disabled by default, opt-in via config:

1. Per-IP (existing): 30 req/min per IP
2. Global: server-wide limit across all IPs
   - Lock-free atomic counter for minimal overhead
   - Returns 503 when exceeded
   - Prevents pool exhaustion from distributed attacks
3. Burst: per-IP burst + sustained windows
   - Blocks rapid-fire abuse within seconds
   - Returns 429 with X-RateLimit-Reason header
   - Example: 5 req/5s burst, 60 req/min sustained

Config:
[global_rate_limit]
requests = 0  # disabled by default
window = "1m"

[burst_rate_limit]
burst = 0  # disabled by default
burst_window = "5s"
sustained = 0
sustained_window = "1m"

Env overrides: GLOBAL_RATE_LIMIT_REQUESTS, GLOBAL_RATE_LIMIT_WINDOW,
BURST_RATE_LIMIT_BURST, BURST_RATE_LIMIT_BURST_WINDOW,
BURST_RATE_LIMIT_SUSTAINED, BURST_RATE_LIMIT_SUSTAINED_WINDOW

Full test coverage: concurrent lock-free test, window expiry, disabled states,
IP isolation, burst vs sustained distinction.
2026-03-21 18:35:31 +00:00
91ab76758c feat: add NixOS module for native deployment
- flake.nix now exports packages, NixOS module, and dev shell
- NixOS module: services.gosearch
  - Configurable port, base URL, user/group, state dir
  - Creates system user automatically
  - Runs as systemd service with auto-restart
  - Optional firewall opening
- To deploy on your NixOS VPS:
  1. Get the vendor hash: nix build .#packages.x86_64-linux.default (copy the hash)
  2. Add to your flake inputs and imports
  3. Enable in configuration.nix
2026-03-21 17:42:05 +00:00
4ec600f6c0 feat: add OpenSearch XML endpoint
- Serve /opensearch.xml with configurable base URL
- Browsers can now add gosearch as a search engine from the address bar
- Configurable via [server] base_url or BASE_URL env var
- XML template embedded in the binary via go:embed
- Added base_url to config.example.toml
2026-03-21 17:40:05 +00:00
3caf702c4f Merge pull request 'feat: add DuckDuckGo, GitHub, Reddit, and Bing engines' (#1) from feat/more-engines into main
Reviewed-on: penal-colony/gosearch#1
2026-03-21 17:35:53 +00:00
a8ab29b23a fix: fix DDG and Bing parsers — verified with live tests
DuckDuckGo:
- Fixed parser to handle single-quoted class attributes (class='result-link')
- Decode DDG tracking URLs (uddg= parameter) to extract real URLs
- Match snippet extraction to actual DDG Lite HTML structure (</td> terminator)

Bing:
- Switched from HTML scraping (blocked by JS detection) to RSS endpoint
  (?format=rss) which returns parseable XML
- Added JSON API response parsing as fallback
- Returns graceful unresponsive_engines entry when blocked

Live test results:
- DuckDuckGo: 9 results 
- GitHub: 10 results (14,768 total) 
- Bing: 10 results via RSS 
- Reddit: skipped (403 from sandbox, needs browser-like context)
2026-03-21 16:57:02 +00:00