Seth/gemma4-research
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: session handoff — gemma4:12b research + smoke test + steel141 Ollama 0.30.8 upgrade

Browse Source 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mortdecai
2026-06-16 23:05:02 -04:00
parent 2336fb437e
commit 043a9aec99
1 changed files with 133 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/handoffs/2026-06-16-230331-gemma4-12b-research-smoketest.md
+133
View File
@@ -0,0 +1,133 @@
# Handoff: gemma4:12b Research + Smoke Test (and steel141 Ollama 0.20.4→0.30.8 upgrade)
## Session Metadata
- Created: 2026-06-16 23:03:31
- Project: /home/claude/bin/gemma4-research
- Branch: master
- Session duration: ~1 hour
### Recent Commits (for context)
- 2336fb4 docs: add gemma4:12b variant + smoke-test results
- f11aa31 docs: session handoff — Gemma 4 tool-calling May-2026 update + bot audit
- 438a96f docs: May 2026 tool-calling update + agentic retrieval landscape
- 0f82cd7 docs: session handoff — GPU bakeoff (3090 Ti vs Strix Halo)
- 91842f3 docs: scrub PII/IPs from gpu-bakeoff
## Handoff Chain
- **Continues from**: [2026-05-25-103131-gemma4-tool-calling-may-update.md](./2026-05-25-103131-gemma4-tool-calling-may-update.md)
- Previous title: Gemma 4 Tool-Calling May-2026 Update + Bot Audit
- **Supersedes**: None
## Current State Summary
Researched and smoke-tested the new `gemma4:12b` model (released 2026-06-03). The model required Ollama ≥ 0.30.5, which neither homelab daemon met (steel141 0.20.4, pve197 0.20.2) — both were hard-gated with HTTP 412. Seth chose to upgrade steel141's Ollama. Upgraded 0.20.4 → 0.30.8 via the official install script, verified service/override/models all intact, pulled `gemma4:12b`, and ran a 4-capability smoke test (text/vision/tools pass; audio listed but untested). Findings committed to `CORPUS_ollama_variants.md` and pushed. Work is **complete** — only optional follow-ups remain.
## Codebase Understanding
### Architecture Overview
This is a documentation/research corpus repo (no application code). Findings about Gemma 4 models live in `CORPUS_*.md` files indexed by `README.md`. Smoke tests hit the local Ollama HTTP API at `http://localhost:11434` (steel141, GPU-backed: GTX 1660 + RTX 3090 Ti).
### Critical Files
| File | Purpose | Relevance |
|------|---------|-----------|
| `CORPUS_ollama_variants.md` | Model variant table + capabilities | **Edited this session** — added 12b row + 2 callout notes |
| `docs/reference/mort-bakeoff-2026-04-18.md` | Pinned 0.20.4 thinking-in-context finding | **Now stale** — pin invalidated by the upgrade |
| `README.md` | Corpus index | Unchanged; points to all CORPUS docs |
| `.backup/` | Safety backups | Contains old ollama binary + override + ps state from this session |
### Key Patterns Discovered
- Ollama registry existence check without downloading: `curl -sI https://registry.ollama.ai/v2/library/<model>/manifests/<tag>` → HTTP 200 means pullable. `tags/list` 404s for library repos (use per-tag manifest HEAD instead).
- Throughput intuition: dense 12B (70 tok/s) is slower per-token than the 26B **MoE** (~134 tok/s) because MoE activates only ~3.8B params/token. Smaller ≠ faster when comparing dense vs MoE.
## Work Completed
### Tasks Finished
- [x] Verified `gemma4:12b` is a real release (2026-06-03, Apache-2.0) via registry manifest + web search
- [x] Gathered specs: 11.9B dense, 256K ctx, Q4_K_M, encoder-free multimodal, 52.4M CLIP projector, requires Ollama 0.30.5
- [x] Asked Seth how to unblock (he chose: upgrade steel141 Ollama)
- [x] Backed up old binary/override/loaded-model state to `.backup/`
- [x] Upgraded steel141 Ollama 0.20.4 → 0.30.8 (official install script); verified override + 28 models intact
- [x] Pulled `gemma4:12b` (7.6 GB, 80s)
- [x] Smoke test: text reasoning ✅, vision ✅, tools ✅ — all 70 tok/s, 100% GPU on 3090 Ti
- [x] Updated `CORPUS_ollama_variants.md`, committed (2336fb4), pushed to Gitea
### Files Modified
| File | Changes | Rationale |
|------|---------|-----------|
| `CORPUS_ollama_variants.md` | Added 12b row, throughput note, 0.20.4-pin-invalidation warning, audio-unverified note | Persist research + smoke-test results |
### Decisions Made
| Decision | Options Considered | Rationale |
|----------|-------------------|-----------|
| Upgrade steel141 Ollama to pull 12b | (a) upgrade steel141, (b) isolated llama.cpp+GGUF, (c) upgrade pve197 | Seth chose (a) — it's the real production path; isolated build was more work and wouldn't exercise the Ollama path he'd deploy on |
| Upgrade via official install script | manual binary swap | Script preserves systemd drop-in override + `~/.ollama`; cleanest |
## Pending Work
## Immediate Next Steps
1. **(Optional) Re-verify the mort-bakeoff finding on 0.30.8.** The "thinking does NOT accumulate in context on Ollama **0.20.4**" claim (`docs/reference/mort-bakeoff-2026-04-18.md`) was version-pinned and is now unverified. Harness at `scripts/mort-bakeoff/`.
2. **(Optional) Test 12b audio through the Ollama API.** `ollama show` lists audio capability, but it was NOT tested. Historically Ollama rejected audio for E-series despite advertising it. Send an audio payload and confirm.
3. **(Optional) Pull `gemma4:12b-mtp`** and benchmark the Multi-Token-Prediction speculative-decode speedup (Google claims 1.4–2.2×).
### Blockers/Open Questions
- [ ] Does the 26b/31b "think:false silent-stop" behavior (`docs/reference/bakeoff-2026-04-18.md`) change on 0.30.8? Untested.
- [ ] pve197 still on 0.20.2 — kept as a 0.20.x reference point. Decide whether to upgrade it too (mort-bot/pve services hit it).
### Deferred Items
- Did not re-run any bakeoffs (scope) — just flagged the pin invalidation in the corpus.
## Context for Resuming Agent
## Important Context
The headline event this session is NOT just the 12b model — it's that **steel141's Ollama jumped 0.20.4 → 0.30.8**, a 10-minor-version leap. Several pinned findings in `docs/reference/*.md` were measured on 0.20.4 and may no longer hold. If anyone relies on those docs, they need re-verification on 0.30.8. The corpus now carries an explicit ⚠️ warning about this in `CORPUS_ollama_variants.md`.
### Assumptions Made
- The systemd drop-in override (`OLLAMA_HOST`, `OLLAMA_KEEP_ALIVE=30m`) survives the install script — **verified true** via `systemctl show ollama -p Environment`.
- The model symlink `/usr/share/ollama/.ollama/models → /mnt/ai_data/live-models` survives — verified (install script only touches `/usr/local/bin` + base unit).
### Potential Gotchas
- `python` is not on PATH on steel141 — use `python3`.
- Registry `tags/list` endpoint returns 404 for `library/*` repos; use per-tag manifest HEAD to check existence.
- gemma4:26b was loaded in VRAM at upgrade time; the restart dropped it (expected). KEEP_ALIVE re-applies on next use.
## Environment State
### Tools/Services Used
- steel141 Ollama daemon, now **v0.30.8** (was 0.20.4), `http://localhost:11434` (also 0.0.0.0 via override)
- 3090 Ti for inference (12b runs 100% GPU, 8.4 GB VRAM)
- `hf` CLI available at `~/.local/bin/hf` (unused this session — was a fallback option)
### Active Processes
- `ollama serve` running under systemd (`systemctl is-active ollama` = active)
- No background jobs left from this session (the failed early pull was on the OLD binary; superseded)
### Environment Variables
- `OLLAMA_HOST`, `OLLAMA_KEEP_ALIVE` (set in `/etc/systemd/system/ollama.service.d/override.conf`; backed up to `.backup/ollama-override-*.conf`)
## Related Resources
- `CORPUS_ollama_variants.md` (updated) — the 12b row + warnings
- `docs/reference/mort-bakeoff-2026-04-18.md` — the now-stale 0.20.4 pin
- `.backup/ollama-bin-0.20.4-1781646458` — old binary if rollback ever needed
- Smoke-test image: `.backup/vision-smoketest.png`
- Web refs: blog.google/.../introducing-gemma-4-12b, developers.googleblog.com/gemma-4-12b-the-developer-guide, apxml.com/models/gemma-4-12b
---
**Security Reminder**: Before finalizing, run `validate_handoff.py` to check for accidental secret exposure.