Files

7.8 KiB
Raw Permalink Blame History

F3 Unlock Procedure — EMC CLAR200 Exos X18 via ESP32-C3

Step-by-step to reach the Seagate F3 service terminal on a CLAR200-locked Exos X18 (ST16000NM004J, EMC fw ES2264) and reformat it to 512-byte sectors.

Read first: ~/bin/EMC_CLAR200_DRIVE_FIX_NOTES.md (why every host-side tool fails) and docs/f3-command-reference.md (F3 dialect + honesty about what's uncertain on Exos). This file is the physical + operational runbook.


⚠️ Safety rules (non-negotiable)

  1. Bench only. Never in the pve173 JBOD. The target drive lives one bay away from the live 549 TB tank. Pull it and work on a bench so a slipped probe, a shorted pad, or a power glitch can't touch a pool disk.
  2. 3.3 V only. The ESP32-C3 is 3.3 V native — correct for the F3 header. Never put 5 V on the diagnostic pins. Never connect the ESP32's VCC/3V3/5V to the drive — power the drive from its own SATA/SAS power. You share only TX, RX, GND.
  3. Log everything. Run the terminal under screen -L (or tmux + pipe). F3 sessions are unrepeatable forensics; capture every byte.
  4. Escalation brake. If the terminal is silent or garbled, the answer is re-check wiring / sweep baud / re-check ground — never "try a more aggressive format command." Destructive F3 commands are a last, deliberate step, not a flailing one.
  5. One drive at a time. Do ZRS015V7 end-to-end, document it, then ZR511KWK.

What you need

Item Have? Notes
ESP32-C3 on steel141 /dev/ttyACM0 native USB-CDC, 3.3 V, MAC 58:8c:81:ac:f0:4c
3× jumper wires (F-F) ? drive header → ESP32 GPIO4/GPIO5/GND
Drive standalone power ? SFF-8482 → SATA-power adapter, a SAS dock, or PSU 5V+12V to the SAS power segment. F3 needs only power — no SAS data link required.
Multimeter ? to find TX/RX/GND pads (continuity + idle-voltage)
Insulating card / Kapton tape ? contingency only — for the BSY isolation trick if the terminal won't respond
Bench host running screen steel141 or a laptop wherever the ESP32 USB plugs in

Step 1 — Flash the bridge (do this now, at the desk)

The ESP32-C3 is already on steel141. From the repo:

cd ~/bin/clar200-unlock/firmware
./flash.sh /dev/ttyACM0        # installs arduino-cli + esp32 core on first run, then builds + uploads

Sanity-check the bridge before wiring to a drive: open it and loop GPIO4↔GPIO5 with a jumper — anything you type should echo back.

screen /dev/ttyACM0 115200     # Ctrl-A k  to quit screen
# jumper GPIO4<->GPIO5: typing shows characters; remove jumper: echo stops

USB-CDC baud is virtual (115200 is fine). You set the drive baud live inside the session with the escape menu: ~1=9600 ~2=38400 ~3=115200 ~4=460800.

Step 2 — Bench the drive

  1. Power down / pull ZRS015V7 from the pve173 JBOD. Record the bay so it goes back if this fails.
  2. On the bench, give it power only (SATA/SAS power segment). No HBA/data cable needed for F3.
  3. Leave it powered off until the ESP32 is wired and the terminal is open (Step 4), so you catch the boot banner.

Step 3 — Find the F3 header pinout (multimeter)

3.5" Seagate SAS drives expose a small diagnostic UART — usually a 3- or 4-pad group or a tiny header near the SAS connector / on the PCB edge. Pads are not labeled. Identify them:

  • GND: continuity (beep) to a drive mounting screw hole / SAS connector shell.
  • TX (drive→host): with the drive powered, idles high ≈ 3.3 V to GND and twitches during boot. This is the pad you wire to ESP32 GPIO5 (DRV_RX).
  • RX (drive→in): the remaining signal pad, usually sits near 0 V / floating. Wire to ESP32 GPIO4 (DRV_TX).
  • A 4th pad is often N/C or a 3.3 V rail — do not wire a power rail.

Photograph the PCB and pad group; save under docs/photos/ for the next drive.

If you can't disambiguate TX/RX by voltage, they're swappable safely at 3.3 V — if you get nothing, swap GPIO4↔GPIO5 and retry. Wrong-way TX/RX cannot damage anything here.

Step 4 — Wire + open the terminal

drive GND  ---- ESP32 GND
drive TX   ---> ESP32 GPIO5 (DRV_RX)
drive RX   <--- ESP32 GPIO4 (DRV_TX)
screen -L /dev/ttyACM0 115200      # -L logs to screenlog.0

Power on the drive. You should see a boot banner (Seagate F3 firmware spew).

  • Banner readable → baud is right (try ~2=38400 first; if garbage, sweep ~1/~3/~4 until the banner is clean ASCII).
  • Nothing at any baud → re-check GND first, then swap TX/RX (~ won't help a wiring fault). Only if still dead consider the BSY isolation trick (docs/f3-command-reference.md §BSY).

Step 5 — Get the F3 T> prompt

Once you see boot text (or a blank but live line):

  • Press Ctrl-Z. Expect the prompt: F3 T> (T = top level).
  • If Ctrl-Z does nothing, try Enter, then Ctrl-Z again; some firmware needs the boot to finish first.

Capture the banner — it often states firmware rev and sometimes the current sector size / "format corrupt" state, which tells you what you're fixing.

Step 6 — Diagnose state, THEN format (drive-specific — go slow)

This is where the Exos X18 F3 dialect diverges from old 7200.11 recipes. Do not paste m0,6,2,...-style commands blind. Map the drive first.

  1. At F3 T> type ? (and at each level you enter) to list available commands. Record the menus. Identify the format / translator level and the read-capacity / identify command.
  2. Read current geometry/sector size. Confirm the hypothesis: locked at 520 B sectors (EMC T10-DIF) and/or format-corrupt (block size 0).
  3. Only once you've located the documented format command for this firmware, issue a format/reinitialize to 512-byte sectors. On Seagate F3 this is the format-unit / translator-regen at the format level; the exact token set comes from Step 6.1's ? output, not from memory.
  4. Formats can run hours on a 16 TB drive and may print little. Don't interrupt. Keep power stable.

See docs/f3-command-reference.md for the candidate command families and exactly which ones are confirmed vs. 7200.11-era-only.

Step 7 — Verify back on a real SAS port

After format completes and the drive is power-cycled:

  1. Reinstall in the pve173 JBOD (or attach via the bench HBA).
  2. smartctl -i /dev/sgX — identity should now be the real Seagate ST16000NM004J, not "SATA-SAM SS160520 CLAR200", with the true serial.
  3. sg_readcap -l /dev/sgX — must report 512-byte logical sectors and full ~16 TB capacity (was Hardware Error before).
  4. Short self-test: smartctl -t short, then check pending/reallocated sectors.

Step 8 — Put it to work in tank

Healthy 14.6 TB+ disk → fix the redundancy holes (see ~/bin/CLAUDE.md storage notes and the tank diagnosis in this project's first handoff):

  • mirror-35 (dead ZL2PNZ99 leg + dying c37f4eff): zpool replace tank 4450364206979169854 <newdisk>, let it resilver, then replace/detach the dying leg. Replace before detach.
  • ex-mirror-38 single-disk vdev (wwn-0x5000c500c382f4ac-part1): zpool attach a mirror partner to restore redundancy.

Then repeat the whole procedure for ZR511KWK.


If F3 fails

Not every EMC-locked drive yields to the terminal (the firmware may refuse the format at the F3 level too, or the SA modules may be EMC-proprietary). Stop and fall back — do not escalate:

  1. RMA / refund — ServerPartDeals. They shipped drives still in CLAR200 locked state = not decommissioned. Lead with the evidence in EMC_CLAR200_DRIVE_FIX_NOTES.md. Lowest effort, arguably their fault.
  2. Decommission in a real EMC VNX/CLARiiON array if you can borrow one.
  3. PC-3000 SAS (professional) — overkill for $/TB drives.