gemma4-research/.claude/handoffs/2026-06-16-230331-gemma4-12b-research-smoketest.md

# Handoff: gemma4:12b Research + Smoke Test (and steel141 Ollama 0.20.4→0.30.8 upgrade)

## Session Metadata
- Created: 2026-06-16 23:03:31
- Project: /home/claude/bin/gemma4-research
- Branch: master
- Session duration: ~1 hour

### Recent Commits (for context)
  - 2336fb4 docs: add gemma4:12b variant + smoke-test results
  - f11aa31 docs: session handoff — Gemma 4 tool-calling May-2026 update + bot audit
  - 438a96f docs: May 2026 tool-calling update + agentic retrieval landscape
  - 0f82cd7 docs: session handoff — GPU bakeoff (3090 Ti vs Strix Halo)
  - 91842f3 docs: scrub PII/IPs from gpu-bakeoff

## Handoff Chain

- **Continues from**: [2026-05-25-103131-gemma4-tool-calling-may-update.md](./2026-05-25-103131-gemma4-tool-calling-may-update.md)
  - Previous title: Gemma 4 Tool-Calling May-2026 Update + Bot Audit
- **Supersedes**: None

## Current State Summary

Researched and smoke-tested the new `gemma4:12b` model (released 2026-06-03). The model required Ollama ≥ 0.30.5, which neither homelab daemon met (steel141 0.20.4, pve197 0.20.2) — both were hard-gated with HTTP 412. Seth chose to upgrade steel141's Ollama. Upgraded 0.20.4 → 0.30.8 via the official install script, verified service/override/models all intact, pulled `gemma4:12b`, and ran a 4-capability smoke test (text/vision/tools pass; audio listed but untested). Findings committed to `CORPUS_ollama_variants.md` and pushed. Work is **complete** — only optional follow-ups remain.

## Codebase Understanding

### Architecture Overview

This is a documentation/research corpus repo (no application code). Findings about Gemma 4 models live in `CORPUS_*.md` files indexed by `README.md`. Smoke tests hit the local Ollama HTTP API at `http://localhost:11434` (steel141, GPU-backed: GTX 1660 + RTX 3090 Ti).

### Critical Files

| File | Purpose | Relevance |
|------|---------|-----------|
| `CORPUS_ollama_variants.md` | Model variant table + capabilities | **Edited this session** — added 12b row + 2 callout notes |
| `docs/reference/mort-bakeoff-2026-04-18.md` | Pinned 0.20.4 thinking-in-context finding | **Now stale** — pin invalidated by the upgrade |
| `README.md` | Corpus index | Unchanged; points to all CORPUS docs |
| `.backup/` | Safety backups | Contains old ollama binary + override + ps state from this session |

### Key Patterns Discovered

- Ollama registry existence check without downloading: `curl -sI https://registry.ollama.ai/v2/library/<model>/manifests/<tag>` → HTTP 200 means pullable. `tags/list` 404s for library repos (use per-tag manifest HEAD instead).
- Throughput intuition: dense 12B (70 tok/s) is slower per-token than the 26B **MoE** (~134 tok/s) because MoE activates only ~3.8B params/token. Smaller ≠ faster when comparing dense vs MoE.

## Work Completed

### Tasks Finished

- [x] Verified `gemma4:12b` is a real release (2026-06-03, Apache-2.0) via registry manifest + web search
- [x] Gathered specs: 11.9B dense, 256K ctx, Q4_K_M, encoder-free multimodal, 52.4M CLIP projector, requires Ollama 0.30.5
- [x] Asked Seth how to unblock (he chose: upgrade steel141 Ollama)
- [x] Backed up old binary/override/loaded-model state to `.backup/`
- [x] Upgraded steel141 Ollama 0.20.4 → 0.30.8 (official install script); verified override + 28 models intact
- [x] Pulled `gemma4:12b` (7.6 GB, 80s)
- [x] Smoke test: text reasoning ✅, vision ✅, tools ✅ — all 70 tok/s, 100% GPU on 3090 Ti
- [x] Updated `CORPUS_ollama_variants.md`, committed (2336fb4), pushed to Gitea

### Files Modified

| File | Changes | Rationale |
|------|---------|-----------|
| `CORPUS_ollama_variants.md` | Added 12b row, throughput note, 0.20.4-pin-invalidation warning, audio-unverified note | Persist research + smoke-test results |

### Decisions Made

| Decision | Options Considered | Rationale |
|----------|-------------------|-----------|
| Upgrade steel141 Ollama to pull 12b | (a) upgrade steel141, (b) isolated llama.cpp+GGUF, (c) upgrade pve197 | Seth chose (a) — it's the real production path; isolated build was more work and wouldn't exercise the Ollama path he'd deploy on |
| Upgrade via official install script | manual binary swap | Script preserves systemd drop-in override + `~/.ollama`; cleanest |

## Pending Work

## Immediate Next Steps

1. **(Optional) Re-verify the mort-bakeoff finding on 0.30.8.** The "thinking does NOT accumulate in context on Ollama **0.20.4**" claim (`docs/reference/mort-bakeoff-2026-04-18.md`) was version-pinned and is now unverified. Harness at `scripts/mort-bakeoff/`.
2. **(Optional) Test 12b audio through the Ollama API.** `ollama show` lists audio capability, but it was NOT tested. Historically Ollama rejected audio for E-series despite advertising it. Send an audio payload and confirm.
3. **(Optional) Pull `gemma4:12b-mtp`** and benchmark the Multi-Token-Prediction speculative-decode speedup (Google claims 1.4–2.2×).

### Blockers/Open Questions

- [ ] Does the 26b/31b "think:false silent-stop" behavior (`docs/reference/bakeoff-2026-04-18.md`) change on 0.30.8? Untested.
- [ ] pve197 still on 0.20.2 — kept as a 0.20.x reference point. Decide whether to upgrade it too (mort-bot/pve services hit it).

### Deferred Items

- Did not re-run any bakeoffs (scope) — just flagged the pin invalidation in the corpus.

## Context for Resuming Agent

## Important Context

The headline event this session is NOT just the 12b model — it's that **steel141's Ollama jumped 0.20.4 → 0.30.8**, a 10-minor-version leap. Several pinned findings in `docs/reference/*.md` were measured on 0.20.4 and may no longer hold. If anyone relies on those docs, they need re-verification on 0.30.8. The corpus now carries an explicit ⚠️ warning about this in `CORPUS_ollama_variants.md`.

### Assumptions Made

- The systemd drop-in override (`OLLAMA_HOST`, `OLLAMA_KEEP_ALIVE=30m`) survives the install script — **verified true** via `systemctl show ollama -p Environment`.
- The model symlink `/usr/share/ollama/.ollama/models → /mnt/ai_data/live-models` survives — verified (install script only touches `/usr/local/bin` + base unit).

### Potential Gotchas

- `python` is not on PATH on steel141 — use `python3`.
- Registry `tags/list` endpoint returns 404 for `library/*` repos; use per-tag manifest HEAD to check existence.
- gemma4:26b was loaded in VRAM at upgrade time; the restart dropped it (expected). KEEP_ALIVE re-applies on next use.

## Environment State

### Tools/Services Used

- steel141 Ollama daemon, now **v0.30.8** (was 0.20.4), `http://localhost:11434` (also 0.0.0.0 via override)
- 3090 Ti for inference (12b runs 100% GPU, 8.4 GB VRAM)
- `hf` CLI available at `~/.local/bin/hf` (unused this session — was a fallback option)

### Active Processes

- `ollama serve` running under systemd (`systemctl is-active ollama` = active)
- No background jobs left from this session (the failed early pull was on the OLD binary; superseded)

### Environment Variables

- `OLLAMA_HOST`, `OLLAMA_KEEP_ALIVE` (set in `/etc/systemd/system/ollama.service.d/override.conf`; backed up to `.backup/ollama-override-*.conf`)

## Related Resources

- `CORPUS_ollama_variants.md` (updated) — the 12b row + warnings
- `docs/reference/mort-bakeoff-2026-04-18.md` — the now-stale 0.20.4 pin
- `.backup/ollama-bin-0.20.4-1781646458` — old binary if rollback ever needed
- Smoke-test image: `.backup/vision-smoketest.png`
- Web refs: blog.google/.../introducing-gemma-4-12b, developers.googleblog.com/gemma-4-12b-the-developer-guide, apxml.com/models/gemma-4-12b

---

**Security Reminder**: Before finalizing, run `validate_handoff.py` to check for accidental secret exposure.