Files

T

Mortdecai 043a9aec99 docs: session handoff — gemma4:12b research + smoke test + steel141 Ollama 0.30.8 upgrade

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-16 23:05:02 -04:00

7.5 KiB

Raw Blame History

Handoff: gemma4:12b Research + Smoke Test (and steel141 Ollama 0.20.4→0.30.8 upgrade)

Session Metadata

Created: 2026-06-16 23:03:31
Project: /home/claude/bin/gemma4-research
Branch: master
Session duration: ~1 hour

Recent Commits (for context)

2336fb4 docs: add gemma4:12b variant + smoke-test results
f11aa31 docs: session handoff — Gemma 4 tool-calling May-2026 update + bot audit
438a96f docs: May 2026 tool-calling update + agentic retrieval landscape
0f82cd7 docs: session handoff — GPU bakeoff (3090 Ti vs Strix Halo)
91842f3 docs: scrub PII/IPs from gpu-bakeoff

Handoff Chain

Continues from: 2026-05-25-103131-gemma4-tool-calling-may-update.md
- Previous title: Gemma 4 Tool-Calling May-2026 Update + Bot Audit
Supersedes: None

Current State Summary

Researched and smoke-tested the new gemma4:12b model (released 2026-06-03). The model required Ollama ≥ 0.30.5, which neither homelab daemon met (steel141 0.20.4, pve197 0.20.2) — both were hard-gated with HTTP 412. Seth chose to upgrade steel141's Ollama. Upgraded 0.20.4 → 0.30.8 via the official install script, verified service/override/models all intact, pulled gemma4:12b, and ran a 4-capability smoke test (text/vision/tools pass; audio listed but untested). Findings committed to CORPUS_ollama_variants.md and pushed. Work is complete — only optional follow-ups remain.

Codebase Understanding

Architecture Overview

This is a documentation/research corpus repo (no application code). Findings about Gemma 4 models live in CORPUS_*.md files indexed by README.md. Smoke tests hit the local Ollama HTTP API at http://localhost:11434 (steel141, GPU-backed: GTX 1660 + RTX 3090 Ti).

Critical Files

File	Purpose	Relevance
`CORPUS_ollama_variants.md`	Model variant table + capabilities	Edited this session — added 12b row + 2 callout notes
`docs/reference/mort-bakeoff-2026-04-18.md`	Pinned 0.20.4 thinking-in-context finding	Now stale — pin invalidated by the upgrade
`README.md`	Corpus index	Unchanged; points to all CORPUS docs
`.backup/`	Safety backups	Contains old ollama binary + override + ps state from this session

Key Patterns Discovered

Ollama registry existence check without downloading: curl -sI https://registry.ollama.ai/v2/library/<model>/manifests/<tag> → HTTP 200 means pullable. tags/list 404s for library repos (use per-tag manifest HEAD instead).
Throughput intuition: dense 12B (70 tok/s) is slower per-token than the 26B MoE (~134 tok/s) because MoE activates only ~3.8B params/token. Smaller ≠ faster when comparing dense vs MoE.

Work Completed

Tasks Finished

Verified gemma4:12b is a real release (2026-06-03, Apache-2.0) via registry manifest + web search
Gathered specs: 11.9B dense, 256K ctx, Q4_K_M, encoder-free multimodal, 52.4M CLIP projector, requires Ollama 0.30.5
Asked Seth how to unblock (he chose: upgrade steel141 Ollama)
Backed up old binary/override/loaded-model state to .backup/
Upgraded steel141 Ollama 0.20.4 → 0.30.8 (official install script); verified override + 28 models intact
Pulled gemma4:12b (7.6 GB, 80s)
Smoke test: text reasoning ✅, vision ✅, tools ✅ — all 70 tok/s, 100% GPU on 3090 Ti
Updated CORPUS_ollama_variants.md, committed (2336fb4), pushed to Gitea

Files Modified

File	Changes	Rationale
`CORPUS_ollama_variants.md`	Added 12b row, throughput note, 0.20.4-pin-invalidation warning, audio-unverified note	Persist research + smoke-test results

Decisions Made

Decision	Options Considered	Rationale
Upgrade steel141 Ollama to pull 12b	(a) upgrade steel141, (b) isolated llama.cpp+GGUF, (c) upgrade pve197	Seth chose (a) — it's the real production path; isolated build was more work and wouldn't exercise the Ollama path he'd deploy on
Upgrade via official install script	manual binary swap	Script preserves systemd drop-in override + `~/.ollama`; cleanest

Pending Work

Immediate Next Steps

(Optional) Re-verify the mort-bakeoff finding on 0.30.8. The "thinking does NOT accumulate in context on Ollama 0.20.4" claim (docs/reference/mort-bakeoff-2026-04-18.md) was version-pinned and is now unverified. Harness at scripts/mort-bakeoff/.
(Optional) Test 12b audio through the Ollama API. ollama show lists audio capability, but it was NOT tested. Historically Ollama rejected audio for E-series despite advertising it. Send an audio payload and confirm.
(Optional) Pull gemma4:12b-mtp and benchmark the Multi-Token-Prediction speculative-decode speedup (Google claims 1.4–2.2×).

Blockers/Open Questions

Does the 26b/31b "think:false silent-stop" behavior (docs/reference/bakeoff-2026-04-18.md) change on 0.30.8? Untested.
pve197 still on 0.20.2 — kept as a 0.20.x reference point. Decide whether to upgrade it too (mort-bot/pve services hit it).

Deferred Items

Did not re-run any bakeoffs (scope) — just flagged the pin invalidation in the corpus.

Context for Resuming Agent

Important Context

The headline event this session is NOT just the 12b model — it's that steel141's Ollama jumped 0.20.4 → 0.30.8, a 10-minor-version leap. Several pinned findings in docs/reference/*.md were measured on 0.20.4 and may no longer hold. If anyone relies on those docs, they need re-verification on 0.30.8. The corpus now carries an explicit ⚠️ warning about this in CORPUS_ollama_variants.md.

Assumptions Made

The systemd drop-in override (OLLAMA_HOST, OLLAMA_KEEP_ALIVE=30m) survives the install script — verified true via systemctl show ollama -p Environment.
The model symlink /usr/share/ollama/.ollama/models → /mnt/ai_data/live-models survives — verified (install script only touches /usr/local/bin + base unit).

Potential Gotchas

python is not on PATH on steel141 — use python3.
Registry tags/list endpoint returns 404 for library/* repos; use per-tag manifest HEAD to check existence.
gemma4:26b was loaded in VRAM at upgrade time; the restart dropped it (expected). KEEP_ALIVE re-applies on next use.

Environment State

Tools/Services Used

steel141 Ollama daemon, now v0.30.8 (was 0.20.4), http://localhost:11434 (also 0.0.0.0 via override)
3090 Ti for inference (12b runs 100% GPU, 8.4 GB VRAM)
hf CLI available at ~/.local/bin/hf (unused this session — was a fallback option)

Active Processes

ollama serve running under systemd (systemctl is-active ollama = active)
No background jobs left from this session (the failed early pull was on the OLD binary; superseded)

Environment Variables

OLLAMA_HOST, OLLAMA_KEEP_ALIVE (set in /etc/systemd/system/ollama.service.d/override.conf; backed up to .backup/ollama-override-*.conf)

CORPUS_ollama_variants.md (updated) — the 12b row + warnings
docs/reference/mort-bakeoff-2026-04-18.md — the now-stale 0.20.4 pin
.backup/ollama-bin-0.20.4-1781646458 — old binary if rollback ever needed
Smoke-test image: .backup/vision-smoketest.png
Web refs: blog.google/.../introducing-gemma-4-12b, developers.googleblog.com/gemma-4-12b-the-developer-guide, apxml.com/models/gemma-4-12b

Security Reminder: Before finalizing, run validate_handoff.py to check for accidental secret exposure.

7.5 KiB Raw Blame History Unescape Escape