Seth/gemma4-research
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: DiffusionGemma research + first-hand smoke test on 3090 Ti

Browse Source 
google/diffusiongemma-26B-A4B-it (released 2026-06-10) — Google's first
open-weight text-diffusion LLM. Does NOT run in Ollama (unknown arch
'diffusion-gemma'); built llama-diffusion-cli from ggml-org/llama.cpp PR
#24423 and smoke-tested Q4_K_M on steel141's 3090 Ti.

- New reference doc with specs, build recipe, throughput, and gotchas
- CORPUS_ollama_variants.md: "not an Ollama variant" callout
- README index line for the reference doc
- scripts/diffusiongemma-smoketest/ harness + raw result logs

Findings: ~106 tok/s effective / ~2030 tok/s in-step-parallel; correct code
+ coherent reasoning; <|channel>thought CoT eats the 256-tok canvas so strict
short formats need block budgeting. nvidia-smi index != CUDA index on steel141
(select 3090 Ti by UUID). Experimental research artifact, not homelab-deployable
until diffusion arch merges to llama.cpp mainline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mortdecai
2026-06-17 22:09:48 -04:00
parent 043a9aec99
commit 6bce2d4c3b
8 changed files with 322 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CORPUS_ollama_variants.md
+2
View File
@@ -15,6 +15,8 @@
> **⚠️ Ollama 0.20.4 pin invalidated (2026-06-16):** steel141 was upgraded 0.20.4 → 0.30.8 to pull `gemma4:12b`. The `mort-bakeoff-2026-04-18.md` finding ("thinking does NOT accumulate in context **on Ollama 0.20.4**") was pinned to that version and must be **re-verified on 0.30.8** before relying on it. Binary backup + override saved in `.backup/ollama-bin-0.20.4-*`.
> **DiffusionGemma is NOT an Ollama variant (do not try to pull it).** `google/diffusiongemma-26B-A4B-it` (released 2026-06-10, the 26B-A4B MoE backbone + a text-**diffusion** head) is a separate generation paradigm — it denoises a 256-token canvas in parallel rather than decoding token-by-token. Ollama/`llama-cli`/`llama-server` reject it (`unknown model architecture: 'diffusion-gemma'`); it needs the dedicated `llama-diffusion-cli` from **ggml-org/llama.cpp PR #24423**. Smoke-tested locally on steel141's 3090 Ti 2026-06-17 (Q4_K_M, ~16 GB, ~106 tok/s effective / ~2030 tok/s in-step-parallel; correct code, coherent reasoning; the `<|channel>thought` CoT eats the canvas so strict short formats need block budgeting). Experimental research artifact, **not deployable through the homelab Ollama stack** until the diffusion arch merges to llama.cpp mainline. Full writeup + build recipe: `docs/reference/diffusiongemma-smoketest-2026-06-17.md`.
## Capabilities by Variant (from `ollama show`)
All variants support: