gemma4-research/CORPUS_ollama_variants.md

Gemma 4 on Ollama — Available Variants

Last verified against Seth's homelab: 2026-06-16 (12b added; steel141 Ollama upgraded 0.20.4 → 0.30.8)

Ollama Model Tags

Tag	Params	Quant	Size on Disk	VRAM	Notes
gemma4:e4b-it-q8_0	~8B total / 4B effective	Q8_0	11.6GB	~12GB	Vision + audio capable. ~25 tok/s on V100
gemma4:12b	11.9B dense	Q4_K_M (default)	7.6GB	~8.4GB	Released 2026-06-03, Apache-2.0. Encoder-free unified multimodal — ollama show lists vision + audio + tools + thinking. 256K context (262144). 52.4M CLIP projector. Smoke-tested on steel141 2026-06-16: text reasoning correct, vision accurate (shapes/colors/spatial/OCR), tool call fired on explicit ask — all at 70 tok/s, 100% GPU on 3090 Ti. Fits a 16GB laptop. Requires Ollama ≥ 0.30.5 (HTTP 412 on older — this gated the whole 0.20.x homelab). Separate gemma4:12b-mtp tag adds Multi-Token-Prediction drafters (1.4–2.2× speculative decode). Google: near-26B-MoE quality at <½ the memory; beats Gemma 3 27B on GPQA Diamond / MMLU-Pro / DocVQA.
gemma4:26b	25.2B total / 3.8B active (MoE)	Q4_K_M (default)	18.0GB	~18GB	Sweet spot for quality/speed. ~134 tok/s on 3090 Ti. 8 experts active of 128 + 1 shared — runs at ~4B-speed, hence throughput. Q4_K_M inference is standard (Mixtral/DeepSeek ship same); the "MoE quality degrades at 4-bit" caveat is a training-time concern, not inference. See tooling/huggingface/model-cards/gemma-4-26B-A4B-it-README.md for the full card.
gemma4:31b-it-q4_K_M	31.3B	Q4_K_M	19.9GB	~24.5GB	Sharpest but 5x slower (~28 tok/s on 3090 Ti, memory pressure)

Throughput note: gemma4:12b (70 tok/s) is slower per-token than gemma4:26b (~134 tok/s) despite being smaller — because 26b is MoE (only ~3.8B params active per token) while 12b is fully dense (all 11.9B active). Pick 12b for quality-per-VRAM and audio on a small box; pick 26b for raw speed when you have the VRAM.

⚠️ Ollama 0.20.4 pin invalidated (2026-06-16): steel141 was upgraded 0.20.4 → 0.30.8 to pull gemma4:12b. The mort-bakeoff-2026-04-18.md finding ("thinking does NOT accumulate in context on Ollama 0.20.4") was pinned to that version and must be re-verified on 0.30.8 before relying on it. Binary backup + override saved in .backup/ollama-bin-0.20.4-*.

Capabilities by Variant (from ollama show)

All variants support:

• Text generation (completion, chat)
• Vision (image input via base64 in images field)
• Tool/function calling (native Ollama tool format)
• Thinking (configurable — ollama show lists it; Seth's finding is to leave it false for tool-use workloads)

E-series (E2B, E4B) additionally support:

• Audio input (conformer encoder) — but not via Ollama; requires llama.cpp with the mmproj-*-E*B-it-*.gguf projector, or vLLM's input_features_padded. See tooling/inference-frameworks/README.md.

gemma4:12b ollama show lists audio as a capability (encoder-free design projects raw waveforms directly). UNVERIFIED through the Ollama API — the 2026-06-16 smoke test covered text, vision, and tools only. Before relying on 12b audio, confirm Ollama 0.30.8 actually accepts an audio payload (historically it rejected audio for the E-series even when ollama show advertised it).

GPU Coexistence (pve197 V100 32GB)

• gemma4:26b + SDXL Turbo: ~28.5GB peak VRAM — fits on V100-32GB
• gemma4:31b: 24.5GB alone — memory pressure with any coexisting model
• gemma4:e4b-it-q8_0: ~12GB — comfortable headroom

Ollama API Endpoint

• /api/generate (single-turn, used by AI_Visualizer)
• /api/chat (multi-turn with message history, used by Simon)
• Both accept tools, images, stream, options, keep_alive

Important Ollama Defaults to Override

Parameter	Ollama Default	Recommended	Why
num_ctx	2048	4096-32768	Default is absurdly small, causes truncation
num_predict	128	512-4096+	Default truncates almost all useful output
think	true (Ollama 0.20+)	false	See GOTCHAS doc
keep_alive	5m	30m-4h	Prevents expensive model reload between calls