gemma4-research/CORPUS_ollama_variants.md

Gemma 4 on Ollama — Available Variants

Last verified against Seth's homelab: 2026-06-16 (12b added; steel141 Ollama upgraded 0.20.4 → 0.30.8)

Ollama Model Tags

Tag	Params	Quant	Size on Disk	VRAM	Notes
gemma4:e4b-it-q8_0	~8B total / 4B effective	Q8_0	11.6GB	~12GB	Vision + audio capable. ~25 tok/s on V100
gemma4:12b	11.9B dense	Q4_K_M (default)	7.6GB	~8.4GB	Released 2026-06-03, Apache-2.0. Encoder-free unified multimodal — ollama show lists vision + audio + tools + thinking. 256K context (262144). 52.4M CLIP projector. Smoke-tested on steel141 2026-06-16: text reasoning correct, vision accurate (shapes/colors/spatial/OCR), tool call fired on explicit ask — all at 70 tok/s, 100% GPU on 3090 Ti. Fits a 16GB laptop. Requires Ollama ≥ 0.30.5 (HTTP 412 on older — this gated the whole 0.20.x homelab). Separate gemma4:12b-mtp tag adds Multi-Token-Prediction drafters (1.4–2.2× speculative decode). Google: near-26B-MoE quality at <½ the memory; beats Gemma 3 27B on GPQA Diamond / MMLU-Pro / DocVQA.
gemma4:26b	25.2B total / 3.8B active (MoE)	Q4_K_M (default)	18.0GB	~18GB	Sweet spot for quality/speed. ~134 tok/s on 3090 Ti. 8 experts active of 128 + 1 shared — runs at ~4B-speed, hence throughput. Q4_K_M inference is standard (Mixtral/DeepSeek ship same); the "MoE quality degrades at 4-bit" caveat is a training-time concern, not inference. See tooling/huggingface/model-cards/gemma-4-26B-A4B-it-README.md for the full card.
gemma4:31b-it-q4_K_M	31.3B	Q4_K_M	19.9GB	~24.5GB	Sharpest but 5x slower (~28 tok/s on 3090 Ti, memory pressure)

Throughput note: gemma4:12b (70 tok/s) is slower per-token than gemma4:26b (~134 tok/s) despite being smaller — because 26b is MoE (only ~3.8B params active per token) while 12b is fully dense (all 11.9B active). Pick 12b for quality-per-VRAM and audio on a small box; pick 26b for raw speed when you have the VRAM.

⚠️ Ollama 0.20.4 pin invalidated (2026-06-16): steel141 was upgraded 0.20.4 → 0.30.8 to pull gemma4:12b. The mort-bakeoff-2026-04-18.md finding ("thinking does NOT accumulate in context on Ollama 0.20.4") was pinned to that version and must be re-verified on 0.30.8 before relying on it. Binary backup + override saved in .backup/ollama-bin-0.20.4-*.

DiffusionGemma is NOT an Ollama variant (do not try to pull it). google/diffusiongemma-26B-A4B-it (released 2026-06-10, the 26B-A4B MoE backbone + a text-diffusion head) is a separate generation paradigm — it denoises a 256-token canvas in parallel rather than decoding token-by-token. Ollama/llama-cli/llama-server reject it (unknown model architecture: 'diffusion-gemma'); it needs the dedicated llama-diffusion-cli from ggml-org/llama.cpp PR #24423. Smoke-tested locally on steel141's 3090 Ti 2026-06-17 (Q4_K_M, ~16 GB, ~106 tok/s effective / ~2030 tok/s in-step-parallel; correct code, coherent reasoning; the <|channel>thought CoT eats the canvas so strict short formats need block budgeting). Experimental research artifact, not deployable through the homelab Ollama stack until the diffusion arch merges to llama.cpp mainline. Full writeup + build recipe: docs/reference/diffusiongemma-smoketest-2026-06-17.md.

Capabilities by Variant (from ollama show)

All variants support:

• Text generation (completion, chat)
• Vision (image input via base64 in images field)
• Tool/function calling (native Ollama tool format)
• Thinking (configurable — ollama show lists it; Seth's finding is to leave it false for tool-use workloads)

E-series (E2B, E4B) additionally support:

• Audio input (conformer encoder) — but not via Ollama; requires llama.cpp with the mmproj-*-E*B-it-*.gguf projector, or vLLM's input_features_padded. See tooling/inference-frameworks/README.md.

gemma4:12b ollama show lists audio as a capability (encoder-free design projects raw waveforms directly). UNVERIFIED through the Ollama API — the 2026-06-16 smoke test covered text, vision, and tools only. Before relying on 12b audio, confirm Ollama 0.30.8 actually accepts an audio payload (historically it rejected audio for the E-series even when ollama show advertised it).

GPU Coexistence (pve197 V100 32GB)

• gemma4:26b + SDXL Turbo: ~28.5GB peak VRAM — fits on V100-32GB
• gemma4:31b: 24.5GB alone — memory pressure with any coexisting model
• gemma4:e4b-it-q8_0: ~12GB — comfortable headroom

Ollama API Endpoint

• /api/generate (single-turn, used by AI_Visualizer)
• /api/chat (multi-turn with message history, used by Simon)
• Both accept tools, images, stream, options, keep_alive

Important Ollama Defaults to Override

Parameter	Ollama Default	Recommended	Why
num_ctx	2048	4096-32768	Default is absurdly small, causes truncation
num_predict	128	512-4096+	Default truncates almost all useful output
think	true (Ollama 0.20+)	false	See GOTCHAS doc
keep_alive	5m	30m-4h	Prevents expensive model reload between calls