Seth/gemma4-research
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
master
gemma4-research/.claude/handoffs/2026-06-16-230331-gemma4-12b-research-smoketest.md
T
Mortdecai 043a9aec99 docs: session handoff — gemma4:12b research + smoke test + steel141 Ollama 0.30.8 upgrade 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:05:02 -04:00

7.5 KiB
Raw Permalink Blame History

Handoff: gemma4:12b Research + Smoke Test (and steel141 Ollama 0.20.4→0.30.8 upgrade)

Session Metadata

  • Created: 2026-06-16 23:03:31
  • Project: /home/claude/bin/gemma4-research
  • Branch: master
  • Session duration: ~1 hour

Recent Commits (for context)

  • 2336fb4 docs: add gemma4:12b variant + smoke-test results
  • f11aa31 docs: session handoff — Gemma 4 tool-calling May-2026 update + bot audit
  • 438a96f docs: May 2026 tool-calling update + agentic retrieval landscape
  • 0f82cd7 docs: session handoff — GPU bakeoff (3090 Ti vs Strix Halo)
  • 91842f3 docs: scrub PII/IPs from gpu-bakeoff

Handoff Chain

Current State Summary

Researched and smoke-tested the new gemma4:12b model (released 2026-06-03). The model required Ollama ≥ 0.30.5, which neither homelab daemon met (steel141 0.20.4, pve197 0.20.2) — both were hard-gated with HTTP 412. Seth chose to upgrade steel141's Ollama. Upgraded 0.20.4 → 0.30.8 via the official install script, verified service/override/models all intact, pulled gemma4:12b, and ran a 4-capability smoke test (text/vision/tools pass; audio listed but untested). Findings committed to CORPUS_ollama_variants.md and pushed. Work is complete — only optional follow-ups remain.

Codebase Understanding

Architecture Overview

This is a documentation/research corpus repo (no application code). Findings about Gemma 4 models live in CORPUS_*.md files indexed by README.md. Smoke tests hit the local Ollama HTTP API at http://localhost:11434 (steel141, GPU-backed: GTX 1660 + RTX 3090 Ti).

Critical Files

File Purpose Relevance
CORPUS_ollama_variants.md Model variant table + capabilities Edited this session — added 12b row + 2 callout notes
docs/reference/mort-bakeoff-2026-04-18.md Pinned 0.20.4 thinking-in-context finding Now stale — pin invalidated by the upgrade
README.md Corpus index Unchanged; points to all CORPUS docs
.backup/ Safety backups Contains old ollama binary + override + ps state from this session

Key Patterns Discovered

  • Ollama registry existence check without downloading: curl -sI https://registry.ollama.ai/v2/library/<model>/manifests/<tag> → HTTP 200 means pullable. tags/list 404s for library repos (use per-tag manifest HEAD instead).
  • Throughput intuition: dense 12B (70 tok/s) is slower per-token than the 26B MoE (~134 tok/s) because MoE activates only ~3.8B params/token. Smaller ≠ faster when comparing dense vs MoE.

Work Completed

Tasks Finished

  • Verified gemma4:12b is a real release (2026-06-03, Apache-2.0) via registry manifest + web search
  • Gathered specs: 11.9B dense, 256K ctx, Q4_K_M, encoder-free multimodal, 52.4M CLIP projector, requires Ollama 0.30.5
  • Asked Seth how to unblock (he chose: upgrade steel141 Ollama)
  • Backed up old binary/override/loaded-model state to .backup/
  • Upgraded steel141 Ollama 0.20.4 → 0.30.8 (official install script); verified override + 28 models intact
  • Pulled gemma4:12b (7.6 GB, 80s)
  • Smoke test: text reasoning ✅, vision ✅, tools ✅ — all 70 tok/s, 100% GPU on 3090 Ti
  • Updated CORPUS_ollama_variants.md, committed (2336fb4), pushed to Gitea

Files Modified

File Changes Rationale
CORPUS_ollama_variants.md Added 12b row, throughput note, 0.20.4-pin-invalidation warning, audio-unverified note Persist research + smoke-test results

Decisions Made

Decision Options Considered Rationale
Upgrade steel141 Ollama to pull 12b (a) upgrade steel141, (b) isolated llama.cpp+GGUF, (c) upgrade pve197 Seth chose (a) — it's the real production path; isolated build was more work and wouldn't exercise the Ollama path he'd deploy on
Upgrade via official install script manual binary swap Script preserves systemd drop-in override + ~/.ollama; cleanest

Pending Work

Immediate Next Steps

  1. (Optional) Re-verify the mort-bakeoff finding on 0.30.8. The "thinking does NOT accumulate in context on Ollama 0.20.4" claim (docs/reference/mort-bakeoff-2026-04-18.md) was version-pinned and is now unverified. Harness at scripts/mort-bakeoff/.
  2. (Optional) Test 12b audio through the Ollama API. ollama show lists audio capability, but it was NOT tested. Historically Ollama rejected audio for E-series despite advertising it. Send an audio payload and confirm.
  3. (Optional) Pull gemma4:12b-mtp and benchmark the Multi-Token-Prediction speculative-decode speedup (Google claims 1.4–2.2×).

Blockers/Open Questions

  • Does the 26b/31b "think:false silent-stop" behavior (docs/reference/bakeoff-2026-04-18.md) change on 0.30.8? Untested.
  • pve197 still on 0.20.2 — kept as a 0.20.x reference point. Decide whether to upgrade it too (mort-bot/pve services hit it).

Deferred Items

  • Did not re-run any bakeoffs (scope) — just flagged the pin invalidation in the corpus.

Context for Resuming Agent

Important Context

The headline event this session is NOT just the 12b model — it's that steel141's Ollama jumped 0.20.4 → 0.30.8, a 10-minor-version leap. Several pinned findings in docs/reference/*.md were measured on 0.20.4 and may no longer hold. If anyone relies on those docs, they need re-verification on 0.30.8. The corpus now carries an explicit ⚠️ warning about this in CORPUS_ollama_variants.md.

Assumptions Made

  • The systemd drop-in override (OLLAMA_HOST, OLLAMA_KEEP_ALIVE=30m) survives the install script — verified true via systemctl show ollama -p Environment.
  • The model symlink /usr/share/ollama/.ollama/models → /mnt/ai_data/live-models survives — verified (install script only touches /usr/local/bin + base unit).

Potential Gotchas

  • python is not on PATH on steel141 — use python3.
  • Registry tags/list endpoint returns 404 for library/* repos; use per-tag manifest HEAD to check existence.
  • gemma4:26b was loaded in VRAM at upgrade time; the restart dropped it (expected). KEEP_ALIVE re-applies on next use.

Environment State

Tools/Services Used

  • steel141 Ollama daemon, now v0.30.8 (was 0.20.4), http://localhost:11434 (also 0.0.0.0 via override)
  • 3090 Ti for inference (12b runs 100% GPU, 8.4 GB VRAM)
  • hf CLI available at ~/.local/bin/hf (unused this session — was a fallback option)

Active Processes

  • ollama serve running under systemd (systemctl is-active ollama = active)
  • No background jobs left from this session (the failed early pull was on the OLD binary; superseded)

Environment Variables

  • OLLAMA_HOST, OLLAMA_KEEP_ALIVE (set in /etc/systemd/system/ollama.service.d/override.conf; backed up to .backup/ollama-override-*.conf)

Related Resources

  • CORPUS_ollama_variants.md (updated) — the 12b row + warnings
  • docs/reference/mort-bakeoff-2026-04-18.md — the now-stale 0.20.4 pin
  • .backup/ollama-bin-0.20.4-1781646458 — old binary if rollback ever needed
  • Smoke-test image: .backup/vision-smoketest.png
  • Web refs: blog.google/.../introducing-gemma-4-12b, developers.googleblog.com/gemma-4-12b-the-developer-guide, apxml.com/models/gemma-4-12b

Security Reminder: Before finalizing, run validate_handoff.py to check for accidental secret exposure.

Reference in New Issue View Git Blame Copy Permalink