Seth/clar200-unlock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e220d2501cccf1f2147917be3e87ee5a9d87468f
clar200-unlock/.claude/handoffs/2026-06-14-215802-f3-unlock-and-tank-redundancy.md
T

7.3 KiB
Raw Blame History

Handoff: F3 unlock of CLAR200 drives + tank redundancy restore

Session Metadata

  • Created: 2026-06-14 21:58:02
  • Project: /home/claude/bin/clar200-unlock
  • Branch: main
  • Session duration: ~1 session (tank diagnosis → project bootstrap)

Recent Commits (for context)

  • 947d5cf init: F3 unlock project — ESP32-C3 bridge + procedure for CLAR200 Exos X18

Handoff Chain

  • Continues from: 2026-06-14-bootstrap.md
    • Previous title: Handoff — 2026-06-14 — clar200-unlock bootstrap
  • Supersedes: None (bootstrap is the inline build-time handoff; this is the session-close handoff. Both current; read bootstrap for the blow-by-blow.)

Review the previous handoff for full context before filling this one.

Current State Summary

Two threads, one root cause. tank (pve173, 549 TB ZFS) is DEGRADED: mirror-35 is a 3-way mirror effectively running on ONE healthy disk, and ex-mirror-38 is a bare single-disk vdev — both need 14.6 TB+ replacements, of which none are free on pve173. Seth re-added an EMC-locked 16 TB drive (SN ZRS015V7) to fill the gap, but it's the same CLAR200-locked Exos X18 a prior session couldn't unlock. I confirmed the lock is firmware-level (no host fix), then bootstrapped the clar200-unlock project: an ESP32-C3 USB-serial bridge

  • full F3-terminal procedure to attempt an out-of-band reformat. Firmware and docs are written, committed, and pushed. Nothing flashed, no drive benched yet — stopped at "ready, awaiting bench hardware + go-ahead."

Codebase Understanding

Architecture Overview

  • Recovery rides the drive's F3 service UART, bypassing the SCSI layer (every SCSI cmd incl. READ CAPACITY returns Hardware Error on these EMC drives).
  • Bridge = ESP32-C3 (native USB-CDC host side) ↔ UART1 GPIO4(TX)/GPIO5(RX) to the drive. Live ~N baud menu avoids reflashing during baud sweep.
  • Build flag CDCOnBoot=cdc is mandatory (makes Serial == USB-CDC).

Critical Files

File Purpose Relevance
PROCEDURE.md bench→wire→flash→F3→format→verify runbook primary — work from here
firmware/clar200_bridge.ino ESP32-C3 bridge sketch flash to the C3
firmware/flash.sh build+flash (installs arduino-cli+esp32 core) run first
docs/f3-command-reference.md F3 dialect, confirmed vs 7200.11-only consult before any format cmd
~/bin/EMC_CLAR200_DRIVE_FIX_NOTES.md exhaustive prior SCSI-side failure log why host tools can't work

Key Patterns Discovered

  • These C3 boards: onboard LED GPIO8 (active-low); safe free UART pins GPIO4/5 (avoid strapping 2/8/9, console 20/21, USB 18/19).
  • esptool already on steel141; python is python3 only (no python alias).

Work Completed

Tasks Finished

  • Diagnosed tank mirror-35 (3-way: 1 dead + 1 dying + 1 healthy; 3.68 TB live data)
  • Confirmed CLAR200 drive lock is firmware-level (new HBA + bus reset both fail)
  • Identified the attached cable as ESP32-C3 (/dev/ttyACM0, MAC 58:8c:81:ac:f0:4c)
  • Built clar200-unlock project: firmware, flash script, procedure, F3 ref, docs
  • Created + pushed Gitea repo; wrote bootstrap + this handoff

Files Modified

File Changes Rationale
(new project ~/bin/clar200-unlock/*) created all bootstrap; see commit 947d5cf

Decisions Made

Decision Options Considered Rationale
F3 UART path SCSI tools, firmware DL, F3, RMA only F3 bypasses the locked SCSI layer
ESP32-C3 as bridge buy USB-TTL vs use attached C3 C3 already here, 3.3 V native-safe
Bench, not in-JBOD in-chassis vs bench never risk live tank during F3 work
Discover F3 cmds via ? copy 7200.11 recipes vs map live Exos dialect differs; blind m0, risky

Pending Work

Immediate Next Steps

  1. Flash the bridge: cd ~/bin/clar200-unlock/firmware && ./flash.sh /dev/ttyACM0 (installs arduino-cli + esp32 core first run). Then loopback-test (jumper GPIO4↔GPIO5, confirm echo in screen /dev/ttyACM0 115200).
  2. Bench drive ZRS015V7: pull from pve173 JBOD, give standalone SAS power.
  3. Find F3 pinout (multimeter), wire TX/RX/GND, get boot banner + F3 T>.
  4. Map format cmd via ?, format to 512, verify sg_readcap/smartctl.
  5. If healthy → tank: zpool replace tank 4450364206979169854 <newdisk> (replace BEFORE detaching the dying leg), then re-mirror ex-mirror-38.
  6. Repeat whole procedure for ZR511KWK.

Blockers/Open Questions

  • Does Seth have standalone SAS power (SFF-8482→SATA-power / SAS dock), jumper wires, and a multimeter on the bench? (gates Step 2–3)
  • RMA to ServerPartDeals first vs. attempt F3 first? (leaning: F3 first)
  • Real Exos X18 F3 format token is unknown until the live ? session

Deferred Items

  • BSY isolation/tape trick — drive isn't BSY-hung; contingency only.
  • Further SCSI-level attempts — exhausted, rejected.
  • zpool scrub tank — do AFTER mirror-35 is on healthy disks (scrub hammers the dying sdg); minor CKSUM counts on mirror-10/18/19/20/30 noted.

Context for Resuming Agent

Important Context

The drive ZRS015V7 is on pve173 at /dev/sg121 + /dev/sg122 (dual SAS paths, 7:0:43:0 and 8:0:11:0), model "SS160520 CLAR200", fake serial "Maxtor 5A320J0…RAMB1TV0". The ESP32-C3 is on steel141 /dev/ttyACM0 — a DIFFERENT machine — which is fine because F3 work is bench-based; the USB side plugs into whatever host runs screen, and the GPIO side wires to the benched drive. tank mirror-35's dying leg wwn-0x5000c500c37f4eff (/dev/sdg, 30k pending sectors) is the ONLY redundancy besides the one healthy leg — do NOT detach it until a real replacement disk has resilvered in. The dead leg to replace is GUID 4450364206979169854.

Assumptions Made

  • The C3 board has onboard LED on GPIO8 (harmless if not).
  • 38400 8N1 is the F3 baud first guess (sweep if garbled).
  • Drive boots clean (answers INQUIRY) so F3 T> reachable without isolation trick.

Potential Gotchas

  • python → use python3 on steel141.
  • USB-CDC baud is virtual; the drive baud is set with the firmware's ~N menu, not the host terminal baud.
  • Never wire ESP32 VCC/5V to the drive — TX/RX/GND only; drive self-powered.
  • flash.sh first run downloads ~1 GB esp32 core.

Environment State

Tools/Services Used

  • pve173 (tank host): zpool, smartctl, sg_*, sg_reset via ssh pve173.
  • steel141: esptool (installed), ESP32-C3 on /dev/ttyACM0. arduino-cli NOT yet installed (flash.sh installs it).
  • Gitea: repo https://git.sethpc.xyz/Seth/clar200-unlock (pushed).

Active Processes

  • None started/left running. No flashing done. No zpool mutations performed (diagnosis was read-only except a harmless SCSI bus reset on the dead drive).

Environment Variables

  • HOMELAB_PASSWORD (used indirectly via SSH aliases) — name only.

Related Resources

  • Repo: https://git.sethpc.xyz/Seth/clar200-unlock
  • ~/bin/EMC_CLAR200_DRIVE_FIX_NOTES.md — prior exhaustive attempt
  • ~/bin/CLAUDE.md — storage section (tank, drive gotchas, ZL2PNZ99/ZL21R9ZF)
  • Bootstrap handoff: ./2026-06-14-bootstrap.md

Security Reminder: Before finalizing, run validate_handoff.py to check for accidental secret exposure.

Reference in New Issue View Git Blame Copy Permalink