Seth/gemma4-research
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: DiffusionGemma research + first-hand smoke test on 3090 Ti

Browse Source 
google/diffusiongemma-26B-A4B-it (released 2026-06-10) — Google's first
open-weight text-diffusion LLM. Does NOT run in Ollama (unknown arch
'diffusion-gemma'); built llama-diffusion-cli from ggml-org/llama.cpp PR
#24423 and smoke-tested Q4_K_M on steel141's 3090 Ti.

- New reference doc with specs, build recipe, throughput, and gotchas
- CORPUS_ollama_variants.md: "not an Ollama variant" callout
- README index line for the reference doc
- scripts/diffusiongemma-smoketest/ harness + raw result logs

Findings: ~106 tok/s effective / ~2030 tok/s in-step-parallel; correct code
+ coherent reasoning; <|channel>thought CoT eats the 256-tok canvas so strict
short formats need block budgeting. nvidia-smi index != CUDA index on steel141
(select 3090 Ti by UUID). Experimental research artifact, not homelab-deployable
until diffusion arch merges to llama.cpp mainline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mortdecai
2026-06-17 22:09:48 -04:00
parent 043a9aec99
commit 6bce2d4c3b
8 changed files with 322 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1
View File
@@ -20,6 +20,7 @@ Research corpus and implementation guidance for Google Gemma 4, based on product
| `docs/openwebui-setup.md` | How to configure Gemma 4 inside OpenWebUI — per-setting reference, two ready-to-bake Workspace Model profiles (chat + extract), and a symptom→cause troubleshooting table mapped back to GOTCHAS.md. Assumes Ollama + OpenWebUI are already running. | When setting up or debugging a Gemma 4 model in OpenWebUI, or handing the front-end config to someone else |
| `docs/reference/bakeoff-2026-04-18.md` | CLI-coding-agent bakeoff on 3090 Ti. **Rounds 1/2 misidentified the cause; Round 3 (the correct one): `think: false` silent-stops gemma4:26b at certain multi-turn states on 32K context.** 31B and Qwen3-Coder robust to the flag. Harness at `scripts/bakeoff/` | When deciding which model to back a CLI agent with, writing a custom agent payload, or debugging a silent tool-call halt |
| `docs/reference/mort-bakeoff-2026-04-18.md` | mort-bot-specific `think=true` vs `think=false` bakeoff on mort's actual loop shape (gemma4:26b, num_ctx=8192). **Thinking does NOT accumulate in context on Ollama 0.20.4** — strips it from serialized history. Both settings behave identically on step counts, tool counts, wall clock. Harness at `scripts/mort-bakeoff/` | When deciding mort-bot's THINK env var, or when someone claims "think=true eats context" without pinning an Ollama version |
| `docs/reference/diffusiongemma-smoketest-2026-06-17.md` | **DiffusionGemma** (`google/diffusiongemma-26B-A4B-it`, released 2026-06-10) — Google's first open-weight text-**diffusion** LLM (26B-A4B MoE + canvas diffusion head). Research + first-hand smoke test on steel141's 3090 Ti: does NOT run in Ollama (`unknown model architecture: 'diffusion-gemma'`), needs `llama-diffusion-cli` from ggml-org/llama.cpp PR #24423. Build recipe, throughput (~106 tok/s effective / ~2030 tok/s in-step-parallel at Q4_K_M), the nvidia-smi-vs-CUDA device-ordering gotcha, and the "thought channel eats the canvas" behavior. | When evaluating a text-diffusion model, deciding if DiffusionGemma is homelab-deployable, or reproducing the llama-diffusion-cli build |
| `docs/reference/gpu-bakeoff-2026-04-20.md` | Cross-GPU throughput bakeoff: steel141 RTX 3090 Ti vs strix-halo (AMD Strix Halo). **3090 Ti wins decode decisively (128 tok/s on 26B MoE). Strix gets ~42% of that on ~25% of the bandwidth.** Also quantifies the MoE vs dense gap: 26B decodes ~4.7× faster than 31B on both cards. Harness at `scripts/gpu-bakeoff/` | When choosing which host to run a Gemma 4 workload on |
| `tooling/` | **Canonical upstream tooling** — real scripts, notebooks, model cards, and configs pulled from Google / HF / framework maintainers (147 files). Subdirs: `google-official/`, `huggingface/`, `inference-frameworks/`, `gemma-family/`, `fine-tuning/`. See `tooling/README.md` for index and findings that update the older `CORPUS_*` docs | When you need authoritative source material — model cards, chat templates, fine-tuning recipes, serving commands for vLLM / llama.cpp / MLX, or to scope a specialized sibling (ShieldGemma, EmbeddingGemma, etc.) |