From 043a9aec99aa743a91b74b4170c5cca0d47ca068 Mon Sep 17 00:00:00 2001 From: Mortdecai Date: Tue, 16 Jun 2026 23:05:02 -0400 Subject: [PATCH] =?UTF-8?q?docs:=20session=20handoff=20=E2=80=94=20gemma4:?= =?UTF-8?q?12b=20research=20+=20smoke=20test=20+=20steel141=20Ollama=200.3?= =?UTF-8?q?0.8=20upgrade?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 (1M context) --- ...16-230331-gemma4-12b-research-smoketest.md | 133 ++++++++++++++++++ 1 file changed, 133 insertions(+) create mode 100644 .claude/handoffs/2026-06-16-230331-gemma4-12b-research-smoketest.md diff --git a/.claude/handoffs/2026-06-16-230331-gemma4-12b-research-smoketest.md b/.claude/handoffs/2026-06-16-230331-gemma4-12b-research-smoketest.md new file mode 100644 index 0000000..cea0665 --- /dev/null +++ b/.claude/handoffs/2026-06-16-230331-gemma4-12b-research-smoketest.md @@ -0,0 +1,133 @@ +# Handoff: gemma4:12b Research + Smoke Test (and steel141 Ollama 0.20.4→0.30.8 upgrade) + +## Session Metadata +- Created: 2026-06-16 23:03:31 +- Project: /home/claude/bin/gemma4-research +- Branch: master +- Session duration: ~1 hour + +### Recent Commits (for context) + - 2336fb4 docs: add gemma4:12b variant + smoke-test results + - f11aa31 docs: session handoff — Gemma 4 tool-calling May-2026 update + bot audit + - 438a96f docs: May 2026 tool-calling update + agentic retrieval landscape + - 0f82cd7 docs: session handoff — GPU bakeoff (3090 Ti vs Strix Halo) + - 91842f3 docs: scrub PII/IPs from gpu-bakeoff + +## Handoff Chain + +- **Continues from**: [2026-05-25-103131-gemma4-tool-calling-may-update.md](./2026-05-25-103131-gemma4-tool-calling-may-update.md) + - Previous title: Gemma 4 Tool-Calling May-2026 Update + Bot Audit +- **Supersedes**: None + +## Current State Summary + +Researched and smoke-tested the new `gemma4:12b` model (released 2026-06-03). The model required Ollama ≥ 0.30.5, which neither homelab daemon met (steel141 0.20.4, pve197 0.20.2) — both were hard-gated with HTTP 412. Seth chose to upgrade steel141's Ollama. Upgraded 0.20.4 → 0.30.8 via the official install script, verified service/override/models all intact, pulled `gemma4:12b`, and ran a 4-capability smoke test (text/vision/tools pass; audio listed but untested). Findings committed to `CORPUS_ollama_variants.md` and pushed. Work is **complete** — only optional follow-ups remain. + +## Codebase Understanding + +### Architecture Overview + +This is a documentation/research corpus repo (no application code). Findings about Gemma 4 models live in `CORPUS_*.md` files indexed by `README.md`. Smoke tests hit the local Ollama HTTP API at `http://localhost:11434` (steel141, GPU-backed: GTX 1660 + RTX 3090 Ti). + +### Critical Files + +| File | Purpose | Relevance | +|------|---------|-----------| +| `CORPUS_ollama_variants.md` | Model variant table + capabilities | **Edited this session** — added 12b row + 2 callout notes | +| `docs/reference/mort-bakeoff-2026-04-18.md` | Pinned 0.20.4 thinking-in-context finding | **Now stale** — pin invalidated by the upgrade | +| `README.md` | Corpus index | Unchanged; points to all CORPUS docs | +| `.backup/` | Safety backups | Contains old ollama binary + override + ps state from this session | + +### Key Patterns Discovered + +- Ollama registry existence check without downloading: `curl -sI https://registry.ollama.ai/v2/library//manifests/` → HTTP 200 means pullable. `tags/list` 404s for library repos (use per-tag manifest HEAD instead). +- Throughput intuition: dense 12B (70 tok/s) is slower per-token than the 26B **MoE** (~134 tok/s) because MoE activates only ~3.8B params/token. Smaller ≠ faster when comparing dense vs MoE. + +## Work Completed + +### Tasks Finished + +- [x] Verified `gemma4:12b` is a real release (2026-06-03, Apache-2.0) via registry manifest + web search +- [x] Gathered specs: 11.9B dense, 256K ctx, Q4_K_M, encoder-free multimodal, 52.4M CLIP projector, requires Ollama 0.30.5 +- [x] Asked Seth how to unblock (he chose: upgrade steel141 Ollama) +- [x] Backed up old binary/override/loaded-model state to `.backup/` +- [x] Upgraded steel141 Ollama 0.20.4 → 0.30.8 (official install script); verified override + 28 models intact +- [x] Pulled `gemma4:12b` (7.6 GB, 80s) +- [x] Smoke test: text reasoning ✅, vision ✅, tools ✅ — all 70 tok/s, 100% GPU on 3090 Ti +- [x] Updated `CORPUS_ollama_variants.md`, committed (2336fb4), pushed to Gitea + +### Files Modified + +| File | Changes | Rationale | +|------|---------|-----------| +| `CORPUS_ollama_variants.md` | Added 12b row, throughput note, 0.20.4-pin-invalidation warning, audio-unverified note | Persist research + smoke-test results | + +### Decisions Made + +| Decision | Options Considered | Rationale | +|----------|-------------------|-----------| +| Upgrade steel141 Ollama to pull 12b | (a) upgrade steel141, (b) isolated llama.cpp+GGUF, (c) upgrade pve197 | Seth chose (a) — it's the real production path; isolated build was more work and wouldn't exercise the Ollama path he'd deploy on | +| Upgrade via official install script | manual binary swap | Script preserves systemd drop-in override + `~/.ollama`; cleanest | + +## Pending Work + +## Immediate Next Steps + +1. **(Optional) Re-verify the mort-bakeoff finding on 0.30.8.** The "thinking does NOT accumulate in context on Ollama **0.20.4**" claim (`docs/reference/mort-bakeoff-2026-04-18.md`) was version-pinned and is now unverified. Harness at `scripts/mort-bakeoff/`. +2. **(Optional) Test 12b audio through the Ollama API.** `ollama show` lists audio capability, but it was NOT tested. Historically Ollama rejected audio for E-series despite advertising it. Send an audio payload and confirm. +3. **(Optional) Pull `gemma4:12b-mtp`** and benchmark the Multi-Token-Prediction speculative-decode speedup (Google claims 1.4–2.2×). + +### Blockers/Open Questions + +- [ ] Does the 26b/31b "think:false silent-stop" behavior (`docs/reference/bakeoff-2026-04-18.md`) change on 0.30.8? Untested. +- [ ] pve197 still on 0.20.2 — kept as a 0.20.x reference point. Decide whether to upgrade it too (mort-bot/pve services hit it). + +### Deferred Items + +- Did not re-run any bakeoffs (scope) — just flagged the pin invalidation in the corpus. + +## Context for Resuming Agent + +## Important Context + +The headline event this session is NOT just the 12b model — it's that **steel141's Ollama jumped 0.20.4 → 0.30.8**, a 10-minor-version leap. Several pinned findings in `docs/reference/*.md` were measured on 0.20.4 and may no longer hold. If anyone relies on those docs, they need re-verification on 0.30.8. The corpus now carries an explicit ⚠️ warning about this in `CORPUS_ollama_variants.md`. + +### Assumptions Made + +- The systemd drop-in override (`OLLAMA_HOST`, `OLLAMA_KEEP_ALIVE=30m`) survives the install script — **verified true** via `systemctl show ollama -p Environment`. +- The model symlink `/usr/share/ollama/.ollama/models → /mnt/ai_data/live-models` survives — verified (install script only touches `/usr/local/bin` + base unit). + +### Potential Gotchas + +- `python` is not on PATH on steel141 — use `python3`. +- Registry `tags/list` endpoint returns 404 for `library/*` repos; use per-tag manifest HEAD to check existence. +- gemma4:26b was loaded in VRAM at upgrade time; the restart dropped it (expected). KEEP_ALIVE re-applies on next use. + +## Environment State + +### Tools/Services Used + +- steel141 Ollama daemon, now **v0.30.8** (was 0.20.4), `http://localhost:11434` (also 0.0.0.0 via override) +- 3090 Ti for inference (12b runs 100% GPU, 8.4 GB VRAM) +- `hf` CLI available at `~/.local/bin/hf` (unused this session — was a fallback option) + +### Active Processes + +- `ollama serve` running under systemd (`systemctl is-active ollama` = active) +- No background jobs left from this session (the failed early pull was on the OLD binary; superseded) + +### Environment Variables + +- `OLLAMA_HOST`, `OLLAMA_KEEP_ALIVE` (set in `/etc/systemd/system/ollama.service.d/override.conf`; backed up to `.backup/ollama-override-*.conf`) + +## Related Resources + +- `CORPUS_ollama_variants.md` (updated) — the 12b row + warnings +- `docs/reference/mort-bakeoff-2026-04-18.md` — the now-stale 0.20.4 pin +- `.backup/ollama-bin-0.20.4-1781646458` — old binary if rollback ever needed +- Smoke-test image: `.backup/vision-smoketest.png` +- Web refs: blog.google/.../introducing-gemma-4-12b, developers.googleblog.com/gemma-4-12b-the-developer-guide, apxml.com/models/gemma-4-12b + +--- + +**Security Reminder**: Before finalizing, run `validate_handoff.py` to check for accidental secret exposure.