From 99df060f517df831e8c842527877a5440089fc67 Mon Sep 17 00:00:00 2001 From: Seth Freiberg Date: Fri, 13 Mar 2026 05:47:23 +0000 Subject: [PATCH] Reorganize docs by project and archive legacy context files --- README.md | 34 ++++++++----------- archive/README.md | 5 +++ .../ARCHITECTURE.md | 0 .../legacy-context-2026-03-13/CONTEXT.md | 0 .../legacy-context-2026-03-13}/OPERATIONS.md | 0 .../legacy-context-2026-03-13/api_README.md | 0 .../searchxng_README.md | 0 projects/searchxng/CONTEXT.md | 23 +++++++++++++ projects/searchxng/README.md | 10 ++++++ projects/sethsearch-api/CONTEXT.md | 27 +++++++++++++++ projects/sethsearch-api/README.md | 17 ++++++++++ projects/shared/ARCHITECTURE.md | 20 +++++++++++ projects/shared/OPERATIONS.md | 16 +++++++++ 13 files changed, 132 insertions(+), 20 deletions(-) create mode 100644 archive/README.md rename {context => archive/legacy-context-2026-03-13}/ARCHITECTURE.md (100%) rename CONTEXT.md => archive/legacy-context-2026-03-13/CONTEXT.md (100%) rename {context => archive/legacy-context-2026-03-13}/OPERATIONS.md (100%) rename api/README.md => archive/legacy-context-2026-03-13/api_README.md (100%) rename searchxng/README.md => archive/legacy-context-2026-03-13/searchxng_README.md (100%) create mode 100644 projects/searchxng/CONTEXT.md create mode 100644 projects/searchxng/README.md create mode 100644 projects/sethsearch-api/CONTEXT.md create mode 100644 projects/sethsearch-api/README.md create mode 100644 projects/shared/ARCHITECTURE.md create mode 100644 projects/shared/OPERATIONS.md diff --git a/README.md b/README.md index 84e00fd..08ba81d 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,22 @@ # SearchXNG Project -Unified workspace for: +This repo is now organized by project area. -- `searchxng/`: SearXNG settings, engine wiring, and instance branding context. -- `api/`: SethSearch API context, indexing rules, and source integration details. -- `context/`: operational runbooks and architecture docs. +## Layout -## Stack +- `projects/searchxng/`: SearXNG integration, engine wiring, and service context. +- `projects/sethsearch-api/`: SethSearch indexer/API behavior, source rules, and maintenance. +- `projects/shared/`: cross-cutting architecture and operations runbooks. +- `archive/legacy-context-2026-03-13/`: archived first-pass context docs. -- SearXNG instance: `searxng.sethpc.xyz` / `search.sethpc.xyz` (CT 119 on `pve173`) -- SethSearch API: `sethsearch.sethpc.xyz` (CT 620 on `pve173`) -- Reverse proxy: Caddy CT 600 (`pve241`) +## Live stack -## Key behavior +- SearXNG: `https://searxng.sethpc.xyz` and `https://search.sethpc.xyz` (CT 119, `pve173`) +- SethSearch API: `https://sethsearch.sethpc.xyz` (CT 620, `pve173`) +- Caddy: CT 600 (`pve241`) -- `sethsearch` engine (general): searches homelab/site/content sources with strict FreshRSS and strict Emby matching. -- `sethflix` engine (videos): Emby-only search for media discovery. -- LibreTranslate integrated via `translate.sethpc.xyz`. +## Quick commands -## Quick Ops - -- Restart SethSearch API: - - `ssh pve173 "pct exec 620 -- systemctl restart sethsearch"` -- Restart SearXNG: - - `ssh pve173 "pct exec 119 -- systemctl restart searxng"` -- Validate Caddy: - - `ssh pve241 "pct exec 600 -- caddy validate --config /etc/caddy/Caddyfile"` +- Restart SethSearch: `ssh pve173 "pct exec 620 -- systemctl restart sethsearch"` +- Restart SearXNG: `ssh pve173 "pct exec 119 -- systemctl restart searxng"` +- Validate Caddy: `ssh pve241 "pct exec 600 -- caddy validate --config /etc/caddy/Caddyfile"` diff --git a/archive/README.md b/archive/README.md new file mode 100644 index 0000000..be7d6e0 --- /dev/null +++ b/archive/README.md @@ -0,0 +1,5 @@ +# Archive + +This folder stores superseded context docs. + +- `legacy-context-2026-03-13/`: initial ungrouped context/runbook files moved during project-based reorganization. diff --git a/context/ARCHITECTURE.md b/archive/legacy-context-2026-03-13/ARCHITECTURE.md similarity index 100% rename from context/ARCHITECTURE.md rename to archive/legacy-context-2026-03-13/ARCHITECTURE.md diff --git a/CONTEXT.md b/archive/legacy-context-2026-03-13/CONTEXT.md similarity index 100% rename from CONTEXT.md rename to archive/legacy-context-2026-03-13/CONTEXT.md diff --git a/context/OPERATIONS.md b/archive/legacy-context-2026-03-13/OPERATIONS.md similarity index 100% rename from context/OPERATIONS.md rename to archive/legacy-context-2026-03-13/OPERATIONS.md diff --git a/api/README.md b/archive/legacy-context-2026-03-13/api_README.md similarity index 100% rename from api/README.md rename to archive/legacy-context-2026-03-13/api_README.md diff --git a/searchxng/README.md b/archive/legacy-context-2026-03-13/searchxng_README.md similarity index 100% rename from searchxng/README.md rename to archive/legacy-context-2026-03-13/searchxng_README.md diff --git a/projects/searchxng/CONTEXT.md b/projects/searchxng/CONTEXT.md new file mode 100644 index 0000000..d24f635 --- /dev/null +++ b/projects/searchxng/CONTEXT.md @@ -0,0 +1,23 @@ +# Project: SearchXNG (SearXNG Layer) + +## Purpose + +Run and tune the SearXNG front-end search experience and route custom engines to SethSearch. + +## Deployment + +- Container: CT `119` on `pve173` +- Service URL: `https://searxng.sethpc.xyz`, `https://search.sethpc.xyz` +- Config file: `/etc/searxng/settings.yml` + +## Active custom engines + +- `sethsearch` (`shortcut: ss`, category: `general`, weight: `5.0`) +- `sethflix` (`shortcut: sfx`, category: `videos`, weight: `5.0`) +- `libretranslate` (`shortcut: lt`, category: `translate`) + +## Notes + +- Use HTTPS URLs for `json_engine` targets. +- Keep engine names lowercase to avoid startup warnings. +- This layer owns weighting and user-facing engine categories. diff --git a/projects/searchxng/README.md b/projects/searchxng/README.md new file mode 100644 index 0000000..169d1a0 --- /dev/null +++ b/projects/searchxng/README.md @@ -0,0 +1,10 @@ +# SearchXNG Project Docs + +- Runtime config: `/etc/searxng/settings.yml` +- Primary task: expose and weight custom engines for homelab and media search. +- Engine endpoint target: `https://sethsearch.sethpc.xyz/search` + +## Validation + +- Engine list check: `curl -s "http://192.168.0.177:8888/config"` +- Restart service: `ssh pve173 "pct exec 119 -- systemctl restart searxng"` diff --git a/projects/sethsearch-api/CONTEXT.md b/projects/sethsearch-api/CONTEXT.md new file mode 100644 index 0000000..a0d3a20 --- /dev/null +++ b/projects/sethsearch-api/CONTEXT.md @@ -0,0 +1,27 @@ +# Project: SethSearch API + +## Purpose + +Provide a unified `json_engine` backend for SearXNG with source-aware matching and weighting. + +## Deployment + +- Container: CT `620` on `pve173` +- Service URL: `https://sethsearch.sethpc.xyz` +- Service name: `sethsearch` +- Code path: `/opt/sethsearch/sethsearch.py` +- Config path: `/opt/sethsearch/config.json` + +## Source groups + +- `source=general`: sites, gitea, wikijs, wordpress, freshrss, emby (strict emby/freshrss) +- `source=sethflix`: emby-only (liberal media matching) + +## Indexed sources + +- Caddy domains (`sites`) +- Gitea public repos (`gitea`) +- Wiki.js public pages/fallback (`wikijs`) +- WordPress public pages/posts (`wordpress`) +- Emby media catalog (`emby`) +- FreshRSS feed items (`freshrss`) diff --git a/projects/sethsearch-api/README.md b/projects/sethsearch-api/README.md new file mode 100644 index 0000000..8cb1f09 --- /dev/null +++ b/projects/sethsearch-api/README.md @@ -0,0 +1,17 @@ +# SethSearch API Docs + +- Health: `GET /health` +- Search: `GET /search?q=&source=&limit=` +- Stats: `GET /stats` +- Manual sync: `POST /sync` + +## Operations + +- Restart: `ssh pve173 "pct exec 620 -- systemctl restart sethsearch"` +- Logs: `ssh pve173 "pct exec 620 -- journalctl -u sethsearch -n 100 --no-pager"` + +## Matching behavior + +- FreshRSS: strict + lower source weight. +- Emby in general: strict. +- Emby in sethflix: liberal and title-boosted. diff --git a/projects/shared/ARCHITECTURE.md b/projects/shared/ARCHITECTURE.md new file mode 100644 index 0000000..3469d03 --- /dev/null +++ b/projects/shared/ARCHITECTURE.md @@ -0,0 +1,20 @@ +# Shared Architecture + +## Request flow + +1. User searches on SearXNG. +2. Custom engines query SethSearch API (`json_engine`). +3. SethSearch reads SQLite FTS5 index and applies source-specific scoring. +4. SearXNG merges with other web engines and renders response. + +## Data model + +- DB: `/opt/sethsearch/articles.db` +- Main table: `documents` +- FTS table: `documents_fts` + +## Reliability pattern + +- Source sync jobs run independently. +- Failures in one source do not block other source updates. +- systemd restart policy keeps service resilient. diff --git a/projects/shared/OPERATIONS.md b/projects/shared/OPERATIONS.md new file mode 100644 index 0000000..da799d7 --- /dev/null +++ b/projects/shared/OPERATIONS.md @@ -0,0 +1,16 @@ +# Shared Operations + +## SearXNG checks + +- Status: `ssh pve173 "pct exec 119 -- systemctl status searxng --no-pager"` +- Logs: `ssh pve173 "pct exec 119 -- journalctl -u searxng -n 100 --no-pager"` + +## SethSearch checks + +- Status: `ssh pve173 "pct exec 620 -- systemctl status sethsearch --no-pager"` +- Logs: `ssh pve173 "pct exec 620 -- journalctl -u sethsearch -n 100 --no-pager"` +- Stats: `curl -s "https://sethsearch.sethpc.xyz/stats"` + +## Caddy checks + +- Validate: `ssh pve241 "pct exec 600 -- caddy validate --config /etc/caddy/Caddyfile"`