chore: baseline existing CI-V suite before RepeaterBook integration
This commit is contained in:
@@ -0,0 +1,201 @@
|
||||
# RepeaterBook integration for the ID-5100 web panel — design
|
||||
|
||||
- **Date:** 2026-06-14
|
||||
- **Project:** `/home/claude/bin/id5100`
|
||||
- **Status:** Approved (design); ready for implementation plan
|
||||
- **Builds on:** existing `civ → radio → {cli, server+panel}` layering (see `README.md`, `DECISIONS.md`)
|
||||
|
||||
## Goal
|
||||
|
||||
Let the web panel browse nearby amateur repeaters from RepeaterBook and, on a single
|
||||
tap, configure the radio to operate through one: set **output frequency + duplex
|
||||
offset + CTCSS tone**. Repeater data is sourced from a locally cached, normalized
|
||||
`repeaters.json` so the panel never calls RepeaterBook directly.
|
||||
|
||||
## Decisions that shaped this design
|
||||
|
||||
1. **Full configuration, not freq-only.** Tapping a repeater sets freq + offset + tone
|
||||
(option 2 from brainstorming). This requires new CI-V commands not yet in the codebase.
|
||||
2. **Cached data, not live per-request.** RepeaterBook's terms ask for infrequent bulk
|
||||
pulls. A scheduled fetch writes `repeaters.json`; the server reads the cache.
|
||||
3. **Build radio-side + UI now; wire the live data source in parallel.** The RepeaterBook
|
||||
API is now gated behind an approval-required `X-RB-App-Token` (bare request → `401
|
||||
auth_missing`). We build against a hand-seeded `repeaters.json` immediately and request
|
||||
the token concurrently. (Decision 3+1 from brainstorming.)
|
||||
4. **QTH:** 38.8423 N, 77.2684 W (Annandale/Fairfax, VA). Scope = Virginia (FIPS 51) +
|
||||
DC (11) + Maryland (24), analog FM + D-STAR, sorted nearest-first by haversine distance.
|
||||
5. **Module split (option B):** focused new modules alongside the existing layers, not a
|
||||
monolithic `server.py` and not a separate microservice.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
repeaters.json ──read──▶ repeaters.py ──▶ server.py /api/repeaters ──▶ panel.html card
|
||||
▲ (load/normalize/ │ tap row
|
||||
│ written by distance/search) ▼
|
||||
rb_fetch.py (API client, server.py /api/tune-repeater ─▶ radio.py
|
||||
reads token from .env) (freq→offset→tone, per-step) (CI-V)
|
||||
│
|
||||
civ.py codec
|
||||
```
|
||||
|
||||
### Module layout
|
||||
|
||||
| File | Role | Pure? / Tested by |
|
||||
|------|------|-------------------|
|
||||
| `civ.py` *(extend)* | Command constants + BCD helpers for offset freq and CTCSS tone | pure → `test_civ.py` |
|
||||
| `radio.py` *(extend)* | `set_duplex()`, `set_offset()`, `set_tone()`, `set_tone_on()` + getters | live-radio verified |
|
||||
| `repeaters.py` *(new)* | Load/normalize `repeaters.json`, haversine distance from QTH, search + nearest-first sort | pure → `test_repeaters.py` |
|
||||
| `rb_fetch.py` *(new)* | RepeaterBook API client; reads token, pulls VA+DC+MD, normalizes → `repeaters.json` | network-isolated; inert without token |
|
||||
| `server.py` *(extend)* | `GET /api/repeaters`, `POST /api/tune-repeater` | live |
|
||||
| `panel.html` *(extend)* | "Repeaters" card (search + nearest-first list + per-step tune status) | headless screenshot |
|
||||
| `repeaters.json` | Cached, normalized data (hand-seeded now, API-filled later) | — |
|
||||
| `rb-fetch.service` / `rb-fetch.timer` *(new, inert)* | systemd user units to refresh cache on a cadence once the token exists | — |
|
||||
|
||||
## Data: normalized record schema
|
||||
|
||||
`rb_fetch.py`, any future CSV ingest, and the hand-seed all emit the **same** internal
|
||||
record, so nothing downstream depends on RepeaterBook's field names:
|
||||
|
||||
```json
|
||||
{
|
||||
"callsign": "W4ABC",
|
||||
"location": "Fairfax, VA",
|
||||
"output_mhz": 147.300,
|
||||
"offset_mhz": 0.600,
|
||||
"duplex": "+", // "+", "-", or "simplex"
|
||||
"tone_hz": 131.8, // null if none
|
||||
"tone_mode": "ctcss", // "ctcss" | "dcs" | "none"
|
||||
"mode": "FM", // "FM" | "DV"
|
||||
"lat": 38.85,
|
||||
"long": -77.30,
|
||||
"notes": ""
|
||||
}
|
||||
```
|
||||
|
||||
`distance_mi` is **computed at serve time** in `repeaters.py` from the QTH constant
|
||||
(not stored). QTH lives as a module constant `QTH = (38.8423, -77.2684)`.
|
||||
|
||||
### Normalizer (RepeaterBook → internal)
|
||||
|
||||
`rb_fetch.py` maps RepeaterBook fields (`Frequency`, `Input Freq`, `PL`/`CTCSS Uplink`,
|
||||
`Callsign`, `Lat`/`Long`, `D-Star`, …) to the schema above. Duplex direction is derived
|
||||
from `sign(input - output)`; `offset_mhz = abs(input - output)`. If lat/long are absent
|
||||
in the live API response, records still load but sort by county/city grouping instead of
|
||||
distance (graceful fallback — confirm field presence once the token lands).
|
||||
|
||||
## New CI-V capability (radio.py + civ.py)
|
||||
|
||||
**All command numbers below are candidate values from the generic ICOM CI-V table and are
|
||||
UNTRUSTED until they ACK on the live ID-5100** — same discipline as the existing
|
||||
`DV=0x17` / `FM=0x05` verification.
|
||||
|
||||
| Capability | Candidate command | Encoding |
|
||||
|-----------|-------------------|----------|
|
||||
| Duplex direction | `0x0F` | `0x10` simplex / `0x11` DUP− / `0x12` DUP+ |
|
||||
| Offset amount | `0x0D` | BCD offset frequency (e.g. 0.600 MHz, 5.000 MHz) |
|
||||
| Repeater TX tone freq | `0x1B 0x00` | BCD tone (1318 → 131.8 Hz) |
|
||||
| Tone encode on/off | `0x16 0x42` | `0x00` off / `0x01` on |
|
||||
|
||||
**Verification method (during implementation, live radio):** read the current value,
|
||||
write it back unchanged (no-op), confirm ACK (`0xFB`). Only after a command ACKs is it
|
||||
marked trusted; until then `tune-repeater` reports that step as failed rather than
|
||||
pretending success. The offset-freq BCD order (LE vs BE) and tone BCD order are pinned by
|
||||
unit tests once observed, mirroring the existing freq(LE)/level(BE) split.
|
||||
|
||||
## API endpoints (server.py)
|
||||
|
||||
- `GET /api/repeaters?q=<text>&limit=25`
|
||||
Reads `repeaters.json` via `repeaters.py`, computes distance from QTH, filters by
|
||||
case-insensitive substring match on callsign/location, returns nearest-first.
|
||||
Response: `{"ok": true, "repeaters": [ {…record…, "distance_mi": 4.2}, … ]}`.
|
||||
|
||||
- `POST /api/tune-repeater` (body = one normalized record)
|
||||
Applies steps **in fixed order: freq → offset/duplex → tone**. Returns per-step results
|
||||
so a half-configured radio is visible, not hidden:
|
||||
```json
|
||||
{"ok": true, "steps": {"freq": true, "duplex": true, "offset": true,
|
||||
"tone_freq": false, "tone_on": false},
|
||||
"errors": {"tone_freq": "radio rejected cmd 0x1B"}}
|
||||
```
|
||||
`ok` is true if at least `freq` succeeded; individual failures surface in `steps`.
|
||||
|
||||
Existing endpoints (`/api/status`, `/api/freq`, `/api/mode`, `/api/vol`, `/api/sql`)
|
||||
are unchanged. The serial `lock` in `server.py` continues to serialize all radio access.
|
||||
|
||||
## UI: "Repeaters" card in panel.html
|
||||
|
||||
A new `.card` matching the existing amber/VFD aesthetic, placed after the Levels card:
|
||||
|
||||
- **Search input** (debounced) → `GET /api/repeaters?q=`.
|
||||
- **Nearest-first list**, each row: `W4ABC · Fairfax VA · 147.300 − · 131.8 · 4.2 mi`.
|
||||
DV rows carry a small `DV` tag so the call-routing limitation isn't surprising.
|
||||
- **Tap a row** → `POST /api/tune-repeater` with that record.
|
||||
- **Per-step status** rendered on the row after tuning: `freq ✓ offset ✓ tone ✗`.
|
||||
No single green/red — the operator sees exactly what landed.
|
||||
|
||||
No new fonts or third-party JS; reuses existing styles and the `post()`/`fetch` helpers.
|
||||
|
||||
## Data refresh (path 1, runs in parallel, inert until token)
|
||||
|
||||
- `rb_fetch.py` reads `RB_APP_TOKEN` from `id5100/.env` (gitignored, per homelab
|
||||
convention), sends the required descriptive `User-Agent`
|
||||
(`id5100-panel/1.0 (homelab; seth@sethfreiberg.com)`), pulls VA+DC+MD, writes
|
||||
`repeaters.json`. Honors `429` with backoff.
|
||||
- `rb-fetch.timer` (systemd **user** unit) runs it on a cadence (default: daily). Both
|
||||
units ship now but stay disabled/inert until the token exists.
|
||||
|
||||
## Limitations (on the record)
|
||||
|
||||
1. **Tune is not atomic.** If offset succeeds but tone NAKs, the radio is half-set.
|
||||
Freq-first ordering guarantees the most important step always lands; per-step UI makes
|
||||
the state visible. Acceptable for a manual operating aid.
|
||||
2. **DV repeaters: freq/offset only.** D-STAR call routing (URCALL/RPT1/RPT2) is the
|
||||
RS-MS1A data-jack protocol, explicitly out of scope (see `DECISIONS.md`). DV rows are
|
||||
labeled; tapping parks the radio on the right freq/offset, routing is set on the head.
|
||||
3. **Proximity depends on lat/long surviving in the API response** (docs dropped them
|
||||
from the documented field list). Hand-seed includes them; live-API absence falls back
|
||||
to county/city grouping. Confirmable when the token lands.
|
||||
4. **New CI-V commands unverified** until they ACK on the live radio (see above).
|
||||
|
||||
## Testing
|
||||
|
||||
- `test_civ.py` *(extend)*: BCD encode/decode for offset freq and tone; round-trips.
|
||||
- `test_repeaters.py` *(new)*: haversine distance correctness, nearest-first sort,
|
||||
substring search, normalizer mapping (RepeaterBook fields → schema), duplex-sign and
|
||||
offset-magnitude derivation, lat/long-absent fallback. Pure, no network, no radio.
|
||||
- Live-radio integration (manual, during implementation): verify each new CI-V command
|
||||
ACKs via read-then-write-back; end-to-end `tune-repeater` against a known local machine.
|
||||
|
||||
## Revision 2026-06-14 (post-CSV pivot)
|
||||
|
||||
Seth supplied a RepeaterBook **CSV proximity export** (`repeaterbook/RB_2606141409.csv`,
|
||||
1258 rows, DC-metro radius). This supersedes the API/token data path:
|
||||
|
||||
- **Decision 3 changed → CSV-only.** The RepeaterBook API client (`rb_fetch.py`) and the
|
||||
systemd refresh timer are **dropped** (YAGNI). Refresh = re-drop a new CSV and re-run a
|
||||
one-line ingest. The token path can be revived later if hands-off refresh is ever wanted.
|
||||
- **No lat/long in the CSV.** Distance/haversine and the `QTH` constant are **dropped**.
|
||||
The export is RepeaterBook's own proximity sort (row 2 = QTH, distance fans outward), so
|
||||
ingest preserves row order as a `rank` integer and `search()` ranks by `rank`. The panel
|
||||
shows location, not a distance figure.
|
||||
- **CSV columns differ from the API** → a CSV normalizer (`normalize_csv`) replaces the
|
||||
API-JSON `normalize_rb`. Columns: `Output Freq, Input Freq, Offset, Uplink Tone,
|
||||
Downlink Tone, Call, Location, County, State, Modes, Digital Access`. Offset magnitude =
|
||||
`abs(Input−Output)`; the `Offset` column gives only the +/− sign. **Uplink Tone** = TX
|
||||
tone we set.
|
||||
- **Mode/band filter:** keep rows whose `Modes` contains `FM` or `DSTAR`, within 2m
|
||||
(144–148) or 70cm (420–450). ~1072 of 1258 rows survive. Mode label: `FM` if `FM`
|
||||
present (operable as analog immediately), else `DV` for pure `DSTAR` (freq/offset only;
|
||||
call routing on the head).
|
||||
- Schema keeps `lat`/`long` as nullable (future-proof, unused now) and adds `rank`.
|
||||
|
||||
The actionable plan (`docs/superpowers/plans/2026-06-14-repeaterbook-integration.md`) was
|
||||
rewritten to match this revision.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- D-STAR DR routing / callsign / messaging (RS-MS1A) — separate effort.
|
||||
- DMR/NXDN/P25/YSF repeater configuration — the ID-5100 is FM + D-STAR only.
|
||||
- Automated authenticated CSV scraping of the RepeaterBook website.
|
||||
- systemd unit for `server.py` itself (tracked separately in the handoff's next-steps).
|
||||
Reference in New Issue
Block a user