# RepeaterBook integration for the ID-5100 web panel — design

- **Date:** 2026-06-14
- **Project:** `/home/claude/bin/id5100`
- **Status:** Approved (design); ready for implementation plan
- **Builds on:** existing `civ → radio → {cli, server+panel}` layering (see `README.md`, `DECISIONS.md`)

## Goal

Let the web panel browse nearby amateur repeaters from RepeaterBook and, on a single
tap, configure the radio to operate through one: set **output frequency + duplex
offset + CTCSS tone**. Repeater data is sourced from a locally cached, normalized
`repeaters.json` so the panel never calls RepeaterBook directly.

## Decisions that shaped this design

1. **Full configuration, not freq-only.** Tapping a repeater sets freq + offset + tone
   (option 2 from brainstorming). This requires new CI-V commands not yet in the codebase.
2. **Cached data, not live per-request.** RepeaterBook's terms ask for infrequent bulk
   pulls. A scheduled fetch writes `repeaters.json`; the server reads the cache.
3. **Build radio-side + UI now; wire the live data source in parallel.** The RepeaterBook
   API is now gated behind an approval-required `X-RB-App-Token` (bare request → `401
   auth_missing`). We build against a hand-seeded `repeaters.json` immediately and request
   the token concurrently. (Decision 3+1 from brainstorming.)
4. **QTH:** 38.8423 N, 77.2684 W (Annandale/Fairfax, VA). Scope = Virginia (FIPS 51) +
   DC (11) + Maryland (24), analog FM + D-STAR, sorted nearest-first by haversine distance.
5. **Module split (option B):** focused new modules alongside the existing layers, not a
   monolithic `server.py` and not a separate microservice.

## Architecture

```
repeaters.json  ──read──▶ repeaters.py ──▶ server.py /api/repeaters ──▶ panel.html card
   ▲                       (load/normalize/                                   │ tap row
   │ written by            distance/search)                                   ▼
rb_fetch.py (API client,                         server.py /api/tune-repeater ─▶ radio.py
 reads token from .env)                            (freq→offset→tone, per-step)   (CI-V)
                                                                                  │
                                                                            civ.py codec
```

### Module layout

| File | Role | Pure? / Tested by |
|------|------|-------------------|
| `civ.py` *(extend)* | Command constants + BCD helpers for offset freq and CTCSS tone | pure → `test_civ.py` |
| `radio.py` *(extend)* | `set_duplex()`, `set_offset()`, `set_tone()`, `set_tone_on()` + getters | live-radio verified |
| `repeaters.py` *(new)* | Load/normalize `repeaters.json`, haversine distance from QTH, search + nearest-first sort | pure → `test_repeaters.py` |
| `rb_fetch.py` *(new)* | RepeaterBook API client; reads token, pulls VA+DC+MD, normalizes → `repeaters.json` | network-isolated; inert without token |
| `server.py` *(extend)* | `GET /api/repeaters`, `POST /api/tune-repeater` | live |
| `panel.html` *(extend)* | "Repeaters" card (search + nearest-first list + per-step tune status) | headless screenshot |
| `repeaters.json` | Cached, normalized data (hand-seeded now, API-filled later) | — |
| `rb-fetch.service` / `rb-fetch.timer` *(new, inert)* | systemd user units to refresh cache on a cadence once the token exists | — |

## Data: normalized record schema

`rb_fetch.py`, any future CSV ingest, and the hand-seed all emit the **same** internal
record, so nothing downstream depends on RepeaterBook's field names:

```json
{
  "callsign": "W4ABC",
  "location": "Fairfax, VA",
  "output_mhz": 147.300,
  "offset_mhz": 0.600,
  "duplex": "+",          // "+", "-", or "simplex"
  "tone_hz": 131.8,        // null if none
  "tone_mode": "ctcss",    // "ctcss" | "dcs" | "none"
  "mode": "FM",            // "FM" | "DV"
  "lat": 38.85,
  "long": -77.30,
  "notes": ""
}
```

`distance_mi` is **computed at serve time** in `repeaters.py` from the QTH constant
(not stored). QTH lives as a module constant `QTH = (38.8423, -77.2684)`.

### Normalizer (RepeaterBook → internal)

`rb_fetch.py` maps RepeaterBook fields (`Frequency`, `Input Freq`, `PL`/`CTCSS Uplink`,
`Callsign`, `Lat`/`Long`, `D-Star`, …) to the schema above. Duplex direction is derived
from `sign(input - output)`; `offset_mhz = abs(input - output)`. If lat/long are absent
in the live API response, records still load but sort by county/city grouping instead of
distance (graceful fallback — confirm field presence once the token lands).

## New CI-V capability (radio.py + civ.py)

**All command numbers below are candidate values from the generic ICOM CI-V table and are
UNTRUSTED until they ACK on the live ID-5100** — same discipline as the existing
`DV=0x17` / `FM=0x05` verification.

| Capability | Candidate command | Encoding |
|-----------|-------------------|----------|
| Duplex direction | `0x0F` | `0x10` simplex / `0x11` DUP− / `0x12` DUP+ |
| Offset amount | `0x0D` | BCD offset frequency (e.g. 0.600 MHz, 5.000 MHz) |
| Repeater TX tone freq | `0x1B 0x00` | BCD tone (1318 → 131.8 Hz) |
| Tone encode on/off | `0x16 0x42` | `0x00` off / `0x01` on |

**Verification method (during implementation, live radio):** read the current value,
write it back unchanged (no-op), confirm ACK (`0xFB`). Only after a command ACKs is it
marked trusted; until then `tune-repeater` reports that step as failed rather than
pretending success. The offset-freq BCD order (LE vs BE) and tone BCD order are pinned by
unit tests once observed, mirroring the existing freq(LE)/level(BE) split.

## API endpoints (server.py)

- `GET /api/repeaters?q=<text>&limit=25`
  Reads `repeaters.json` via `repeaters.py`, computes distance from QTH, filters by
  case-insensitive substring match on callsign/location, returns nearest-first.
  Response: `{"ok": true, "repeaters": [ {…record…, "distance_mi": 4.2}, … ]}`.

- `POST /api/tune-repeater` (body = one normalized record)
  Applies steps **in fixed order: freq → offset/duplex → tone**. Returns per-step results
  so a half-configured radio is visible, not hidden:
  ```json
  {"ok": true, "steps": {"freq": true, "duplex": true, "offset": true,
                          "tone_freq": false, "tone_on": false},
   "errors": {"tone_freq": "radio rejected cmd 0x1B"}}
  ```
  `ok` is true if at least `freq` succeeded; individual failures surface in `steps`.

Existing endpoints (`/api/status`, `/api/freq`, `/api/mode`, `/api/vol`, `/api/sql`)
are unchanged. The serial `lock` in `server.py` continues to serialize all radio access.

## UI: "Repeaters" card in panel.html

A new `.card` matching the existing amber/VFD aesthetic, placed after the Levels card:

- **Search input** (debounced) → `GET /api/repeaters?q=`.
- **Nearest-first list**, each row: `W4ABC · Fairfax VA · 147.300 − · 131.8 · 4.2 mi`.
  DV rows carry a small `DV` tag so the call-routing limitation isn't surprising.
- **Tap a row** → `POST /api/tune-repeater` with that record.
- **Per-step status** rendered on the row after tuning: `freq ✓ offset ✓ tone ✗`.
  No single green/red — the operator sees exactly what landed.

No new fonts or third-party JS; reuses existing styles and the `post()`/`fetch` helpers.

## Data refresh (path 1, runs in parallel, inert until token)

- `rb_fetch.py` reads `RB_APP_TOKEN` from `id5100/.env` (gitignored, per homelab
  convention), sends the required descriptive `User-Agent`
  (`id5100-panel/1.0 (homelab; seth@sethfreiberg.com)`), pulls VA+DC+MD, writes
  `repeaters.json`. Honors `429` with backoff.
- `rb-fetch.timer` (systemd **user** unit) runs it on a cadence (default: daily). Both
  units ship now but stay disabled/inert until the token exists.

## Limitations (on the record)

1. **Tune is not atomic.** If offset succeeds but tone NAKs, the radio is half-set.
   Freq-first ordering guarantees the most important step always lands; per-step UI makes
   the state visible. Acceptable for a manual operating aid.
2. **DV repeaters: freq/offset only.** D-STAR call routing (URCALL/RPT1/RPT2) is the
   RS-MS1A data-jack protocol, explicitly out of scope (see `DECISIONS.md`). DV rows are
   labeled; tapping parks the radio on the right freq/offset, routing is set on the head.
3. **Proximity depends on lat/long surviving in the API response** (docs dropped them
   from the documented field list). Hand-seed includes them; live-API absence falls back
   to county/city grouping. Confirmable when the token lands.
4. **New CI-V commands unverified** until they ACK on the live radio (see above).

## Testing

- `test_civ.py` *(extend)*: BCD encode/decode for offset freq and tone; round-trips.
- `test_repeaters.py` *(new)*: haversine distance correctness, nearest-first sort,
  substring search, normalizer mapping (RepeaterBook fields → schema), duplex-sign and
  offset-magnitude derivation, lat/long-absent fallback. Pure, no network, no radio.
- Live-radio integration (manual, during implementation): verify each new CI-V command
  ACKs via read-then-write-back; end-to-end `tune-repeater` against a known local machine.

## Revision 2026-06-14 (post-CSV pivot)

Seth supplied a RepeaterBook **CSV proximity export** (`repeaterbook/RB_2606141409.csv`,
1258 rows, DC-metro radius). This supersedes the API/token data path:

- **Decision 3 changed → CSV-only.** The RepeaterBook API client (`rb_fetch.py`) and the
  systemd refresh timer are **dropped** (YAGNI). Refresh = re-drop a new CSV and re-run a
  one-line ingest. The token path can be revived later if hands-off refresh is ever wanted.
- **No lat/long in the CSV.** Distance/haversine and the `QTH` constant are **dropped**.
  The export is RepeaterBook's own proximity sort (row 2 = QTH, distance fans outward), so
  ingest preserves row order as a `rank` integer and `search()` ranks by `rank`. The panel
  shows location, not a distance figure.
- **CSV columns differ from the API** → a CSV normalizer (`normalize_csv`) replaces the
  API-JSON `normalize_rb`. Columns: `Output Freq, Input Freq, Offset, Uplink Tone,
  Downlink Tone, Call, Location, County, State, Modes, Digital Access`. Offset magnitude =
  `abs(Input−Output)`; the `Offset` column gives only the +/− sign. **Uplink Tone** = TX
  tone we set.
- **Mode/band filter:** keep rows whose `Modes` contains `FM` or `DSTAR`, within 2m
  (144–148) or 70cm (420–450). ~1072 of 1258 rows survive. Mode label: `FM` if `FM`
  present (operable as analog immediately), else `DV` for pure `DSTAR` (freq/offset only;
  call routing on the head).
- Schema keeps `lat`/`long` as nullable (future-proof, unused now) and adds `rank`.

The actionable plan (`docs/superpowers/plans/2026-06-14-repeaterbook-integration.md`) was
rewritten to match this revision.

## Out of scope

- D-STAR DR routing / callsign / messaging (RS-MS1A) — separate effort.
- DMR/NXDN/P25/YSF repeater configuration — the ID-5100 is FM + D-STAR only.
- Automated authenticated CSV scraping of the RepeaterBook website.
- systemd unit for `server.py` itself (tracked separately in the handoff's next-steps).