docs: DiffusionGemma research + first-hand smoke test on 3090 Ti
google/diffusiongemma-26B-A4B-it (released 2026-06-10) — Google's first open-weight text-diffusion LLM. Does NOT run in Ollama (unknown arch 'diffusion-gemma'); built llama-diffusion-cli from ggml-org/llama.cpp PR #24423 and smoke-tested Q4_K_M on steel141's 3090 Ti. - New reference doc with specs, build recipe, throughput, and gotchas - CORPUS_ollama_variants.md: "not an Ollama variant" callout - README index line for the reference doc - scripts/diffusiongemma-smoketest/ harness + raw result logs Findings: ~106 tok/s effective / ~2030 tok/s in-step-parallel; correct code + coherent reasoning; <|channel>thought CoT eats the 256-tok canvas so strict short formats need block budgeting. nvidia-smi index != CUDA index on steel141 (select 3090 Ti by UUID). Experimental research artifact, not homelab-deployable until diffusion arch merges to llama.cpp mainline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Mortdecai
@@ -0,0 +1,44 @@
|
||||
0.00.836.466 W load: control-looking token: 212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
|
||||
0.00.837.141 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
|
||||
0.00.873.644 W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list
|
||||
0.04.996.883 I diffusion: -n 160 -> 1 blocks, n_ubatch=2304 n_batch=2304 n_ctx=2304 (canvas_length=256)
|
||||
0.04.996.886 I diffusion: --fit has no effect here; context is sized from -n and the canvas. Set -ngl / --n-cpu-moe to control device memory.
|
||||
0.04.997.441 W llama_context: n_ctx_seq (2304) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
|
||||
0.05.002.413 W sched_reserve: layer 5 is assigned to device CUDA0 but the Flash Attention tensor is assigned to device CPU (usually due to missing support)
|
||||
0.05.002.416 W sched_reserve: Flash Attention was auto, set to disabled
|
||||
0.07.343.745 I diffusion_params: steps=128 schedule=0 algorithm=4 temperature=0.800 eps=0.001000 mask_token=4
|
||||
0.07.344.014 I diffusion_eb: max_steps=48 t=[0.400,0.800] entropy_bound=0.1000 stability=1 confidence=0.0050 kv_cache=on gpu_sampling=on sample_reduce=on
|
||||
0.07.345.228 W init: embeddings required but some input tokens were not marked as outputs -> overriding
|
||||
0.07.822.959 I
|
||||
diffusion step: 0/48 [ ] 0%0.07.925.523 I
|
||||
diffusion step: 1/48 [= ] 2%0.08.028.933 I
|
||||
diffusion step: 2/48 [== ] 4%0.08.132.168 I
|
||||
diffusion step: 3/48 [=== ] 6%0.08.238.755 I
|
||||
diffusion step: 4/48 [==== ] 8%0.08.343.911 I
|
||||
diffusion step: 5/48 [===== ] 10%0.08.450.845 I
|
||||
diffusion step: 6/48 [====== ] 12%0.08.558.796 I
|
||||
diffusion step: 7/48 [======= ] 14%0.08.665.123 I
|
||||
diffusion step: 8/48 [======== ] 16%0.08.771.930 I
|
||||
diffusion step: 9/48 [========= ] 18%0.08.881.776 I
|
||||
diffusion step: 10/48 [========== ] 20%0.08.988.729 I
|
||||
diffusion step: 11/48 [=========== ] 22%0.09.096.269 I
|
||||
diffusion step: 12/48 [============ ] 25%
|
||||
<|channel>thought
|
||||
* Task: Write a Python function `is_prime(n)`.
|
||||
* Requirement: Include a docstring.
|
||||
* Constraint: Output *only* the code, no explanation.
|
||||
|
||||
* A prime number is a natural number greater than 1 that has no positive divisors other than 1 and itself.
|
||||
* If $n \le 1$, not prime.
|
||||
* If $n = 2$, prime.
|
||||
* If $n$ is even and $>2$, not prime.
|
||||
* Check divisors from 3 to $\sqrt{n}$.
|
||||
|
||||
```python
|
||||
def is_prime(n):
|
||||
"""
|
||||
Determines if a number n is prime.
|
||||
|
||||
Args:
|
||||
n (int): The number to check.
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
0.00.841.557 W load: control-looking token: 212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
|
||||
0.00.842.233 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
|
||||
0.00.879.927 W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list
|
||||
0.04.814.391 I diffusion: -n 48 -> 1 blocks, n_ubatch=2304 n_batch=2304 n_ctx=2304 (canvas_length=256)
|
||||
0.04.814.394 I diffusion: --fit has no effect here; context is sized from -n and the canvas. Set -ngl / --n-cpu-moe to control device memory.
|
||||
0.04.814.907 W llama_context: n_ctx_seq (2304) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
|
||||
0.04.819.412 W sched_reserve: layer 5 is assigned to device CUDA0 but the Flash Attention tensor is assigned to device CPU (usually due to missing support)
|
||||
0.04.819.415 W sched_reserve: Flash Attention was auto, set to disabled
|
||||
0.07.106.365 I diffusion_params: steps=128 schedule=0 algorithm=4 temperature=0.800 eps=0.001000 mask_token=4
|
||||
0.07.106.635 I diffusion_eb: max_steps=48 t=[0.400,0.800] entropy_bound=0.1000 stability=1 confidence=0.0050 kv_cache=on gpu_sampling=on sample_reduce=on
|
||||
0.07.107.839 W init: embeddings required but some input tokens were not marked as outputs -> overriding
|
||||
0.07.565.506 I
|
||||
diffusion step: 0/48 [ ] 0%0.07.667.965 I
|
||||
diffusion step: 1/48 [= ] 2%0.07.772.514 I
|
||||
diffusion step: 2/48 [== ] 4%0.07.876.431 I
|
||||
diffusion step: 3/48 [=== ] 6%0.07.981.552 I
|
||||
diffusion step: 4/48 [==== ] 8%0.08.087.446 I
|
||||
diffusion step: 5/48 [===== ] 10%0.08.196.277 I
|
||||
diffusion step: 6/48 [====== ] 12%0.08.302.551 I
|
||||
diffusion step: 7/48 [======= ] 14%0.08.407.803 I
|
||||
diffusion step: 8/48 [======== ] 16%0.08.514.697 I
|
||||
diffusion step: 9/48 [========= ] 18%0.08.621.268 I
|
||||
diffusion step: 10/48 [========== ] 20%0.08.728.440 I
|
||||
diffusion step: 11/48 [=========== ] 22%0.08.835.336 I
|
||||
diffusion step: 12/48 [============ ] 25%0.08.942.227 I
|
||||
diffusion step: 13/48 [============= ] 27%0.09.050.835 I
|
||||
diffusion step: 14/48 [============== ] 29%0.09.160.522 I
|
||||
diffusion step: 15/48 [=============== ] 31%0.09.268.593 I
|
||||
diffusion step: 16/48 [================ ] 33%0.09.377.200 I
|
||||
diffusion step: 17/48 [================= ] 35%0.09.485.784 I
|
||||
diffusion step: 18/48 [================== ] 37%0.09.595.474 I
|
||||
@@ -0,0 +1,43 @@
|
||||
0.00.778.413 W load: control-looking token: 212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
|
||||
0.00.778.868 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
|
||||
0.00.803.531 W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list
|
||||
0.04.672.736 I diffusion: -n 512 -> 2 blocks, n_ubatch=2560 n_batch=2560 n_ctx=2560 (canvas_length=256)
|
||||
0.04.672.739 I diffusion: --fit has no effect here; context is sized from -n and the canvas. Set -ngl / --n-cpu-moe to control device memory.
|
||||
0.04.673.245 W llama_context: n_ctx_seq (2560) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
|
||||
0.04.677.664 W sched_reserve: layer 5 is assigned to device CUDA0 but the Flash Attention tensor is assigned to device CPU (usually due to missing support)
|
||||
0.04.677.667 W sched_reserve: Flash Attention was auto, set to disabled
|
||||
0.06.828.575 I diffusion_params: steps=128 schedule=0 algorithm=4 temperature=0.800 eps=0.001000 mask_token=4
|
||||
0.06.828.825 I diffusion_eb: max_steps=48 t=[0.400,0.800] entropy_bound=0.1000 stability=1 confidence=0.0050 kv_cache=on gpu_sampling=on sample_reduce=on
|
||||
0.06.829.987 W init: embeddings required but some input tokens were not marked as outputs -> overriding
|
||||
0.07.297.536 I
|
||||
diffusion step: 0/48 [ ] 0%0.07.401.754 I
|
||||
diffusion step: 1/48 [= ] 2%0.07.506.767 I
|
||||
diffusion step: 2/48 [== ] 4%0.07.612.593 I
|
||||
diffusion step: 3/48 [=== ] 6%0.07.720.060 I
|
||||
diffusion step: 4/48 [==== ] 8%0.07.827.278 I
|
||||
diffusion step: 5/48 [===== ] 10%0.07.933.928 I
|
||||
diffusion step: 6/48 [====== ] 12%0.08.042.088 I
|
||||
diffusion step: 7/48 [======= ] 14%0.08.150.433 I
|
||||
diffusion step: 8/48 [======== ] 16%0.08.259.575 I
|
||||
diffusion step: 9/48 [========= ] 18%0.08.367.825 I
|
||||
diffusion step: 10/48 [========== ] 20%0.08.475.147 I
|
||||
diffusion step: 11/48 [=========== ] 22%0.08.584.038 I
|
||||
diffusion step: 12/48 [============ ] 25%0.08.693.905 I
|
||||
diffusion step: 13/48 [============= ] 27%0.08.803.010 I
|
||||
diffusion step: 14/48 [============== ] 29%0.08.912.275 I
|
||||
diffusion step: 15/48 [=============== ] 31%0.09.021.574 I
|
||||
diffusion step: 16/48 [================ ] 33%0.09.130.837 I
|
||||
diffusion step: 17/48 [================= ] 35%0.09.240.344 I
|
||||
diffusion step: 18/48 [================== ] 37%0.09.350.482 I
|
||||
diffusion step: 19/48 [=================== ] 39%0.09.458.691 I
|
||||
diffusion step: 20/48 [==================== ] 41%0.09.458.811 W init: embeddings required but some input tokens were not marked as outputs -> overriding
|
||||
0.09.908.398 I
|
||||
diffusion step: 0/48 [ ] 0%0.10.012.788 I
|
||||
diffusion step: 1/48 [= ] 2%0.10.116.823 I
|
||||
diffusion step: 2/48 [== ] 4%0.10.221.634 I
|
||||
diffusion step: 3/48 [=== ] 6%0.10.327.638 I
|
||||
diffusion step: 4/48 [==== ] 8%0.10.436.834 I
|
||||
diffusion step: 5/48 [===== ] 10%0.10.544.714 I
|
||||
diffusion step: 6/48 [====== ] 12%0.10.652.512 I
|
||||
diffusion step: 7/48 [======= ] 14%0.10.762.174 I
|
||||
diffusion step: 8/48 [======== ] 16%0.10.871.902 I
|
||||
@@ -0,0 +1,27 @@
|
||||
0.00.896.419 W load: control-looking token: 212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
|
||||
0.00.897.085 W load: control-looking token: 50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
|
||||
0.00.933.523 W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list
|
||||
0.04.856.194 I diffusion: -n 128 -> 1 blocks, n_ubatch=2304 n_batch=2304 n_ctx=2304 (canvas_length=256)
|
||||
0.04.856.197 I diffusion: --fit has no effect here; context is sized from -n and the canvas. Set -ngl / --n-cpu-moe to control device memory.
|
||||
0.04.864.961 W llama_context: n_ctx_seq (2304) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
|
||||
0.04.869.425 W sched_reserve: layer 5 is assigned to device CUDA0 but the Flash Attention tensor is assigned to device CPU (usually due to missing support)
|
||||
0.04.869.427 W sched_reserve: Flash Attention was auto, set to disabled
|
||||
0.07.226.382 I diffusion_params: steps=128 schedule=0 algorithm=4 temperature=0.800 eps=0.001000 mask_token=4
|
||||
0.07.226.626 I diffusion_eb: max_steps=48 t=[0.400,0.800] entropy_bound=0.1000 stability=1 confidence=0.0050 kv_cache=on gpu_sampling=on sample_reduce=on
|
||||
0.07.228.370 W init: embeddings required but some input tokens were not marked as outputs -> overriding
|
||||
0.07.721.777 I
|
||||
diffusion step: 0/48 [ ] 0%0.07.822.673 I
|
||||
diffusion step: 1/48 [= ] 2%0.07.924.966 I
|
||||
diffusion step: 2/48 [== ] 4%0.08.030.579 I
|
||||
diffusion step: 3/48 [=== ] 6%0.08.135.224 I
|
||||
diffusion step: 4/48 [==== ] 8%0.08.240.214 I
|
||||
diffusion step: 5/48 [===== ] 10%0.08.346.392 I
|
||||
diffusion step: 6/48 [====== ] 12%0.08.452.589 I
|
||||
diffusion step: 7/48 [======= ] 14%0.08.556.808 I
|
||||
diffusion step: 8/48 [======== ] 16%0.08.663.759 I
|
||||
diffusion step: 9/48 [========= ] 18%0.08.770.846 I
|
||||
diffusion step: 10/48 [========== ] 20%0.08.876.181 I
|
||||
diffusion step: 11/48 [=========== ] 22%0.08.984.389 I
|
||||
diffusion step: 12/48 [============ ] 25%0.09.090.492 I
|
||||
diffusion step: 13/48 [============= ] 27%0.09.198.211 I
|
||||
diffusion step: 14/48 [============== ] 29%0.09.304.460 I
|
||||
Reference in New Issue
Block a user