0.00.896.419 W load: control-looking token:    212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.897.085 W load: control-looking token:     50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.933.523 W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list
0.04.856.194 I diffusion: -n 128 -> 1 blocks, n_ubatch=2304 n_batch=2304 n_ctx=2304 (canvas_length=256)
0.04.856.197 I diffusion: --fit has no effect here; context is sized from -n and the canvas. Set -ngl / --n-cpu-moe to control device memory.
0.04.864.961 W llama_context: n_ctx_seq (2304) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.04.869.425 W sched_reserve: layer 5 is assigned to device CUDA0 but the Flash Attention tensor is assigned to device CPU (usually due to missing support)
0.04.869.427 W sched_reserve: Flash Attention was auto, set to disabled
0.07.226.382 I diffusion_params: steps=128 schedule=0 algorithm=4 temperature=0.800 eps=0.001000 mask_token=4
0.07.226.626 I diffusion_eb: max_steps=48 t=[0.400,0.800] entropy_bound=0.1000 stability=1 confidence=0.0050 kv_cache=on gpu_sampling=on sample_reduce=on
0.07.228.370 W init: embeddings required but some input tokens were not marked as outputs -> overriding
0.07.721.777 I 
diffusion step: 0/48 [                                                  ] 0%0.07.822.673 I 
diffusion step: 1/48 [=                                                 ] 2%0.07.924.966 I 
diffusion step: 2/48 [==                                                ] 4%0.08.030.579 I 
diffusion step: 3/48 [===                                               ] 6%0.08.135.224 I 
diffusion step: 4/48 [====                                              ] 8%0.08.240.214 I 
diffusion step: 5/48 [=====                                             ] 10%0.08.346.392 I 
diffusion step: 6/48 [======                                            ] 12%0.08.452.589 I 
diffusion step: 7/48 [=======                                           ] 14%0.08.556.808 I 
diffusion step: 8/48 [========                                          ] 16%0.08.663.759 I 
diffusion step: 9/48 [=========                                         ] 18%0.08.770.846 I 
diffusion step: 10/48 [==========                                        ] 20%0.08.876.181 I 
diffusion step: 11/48 [===========                                       ] 22%0.08.984.389 I 
diffusion step: 12/48 [============                                      ] 25%0.09.090.492 I 
diffusion step: 13/48 [=============                                     ] 27%0.09.198.211 I 
diffusion step: 14/48 [==============                                    ] 29%0.09.304.460 I 
diffusion step: 15/48 [===============                                   ] 31%0.09.410.010 I 
diffusion step: 16/48 [================                                  ] 33%0.09.516.289 I 
diffusion step: 17/48 [=================                                 ] 35%0.09.623.885 I 
diffusion step: 18/48 [==================                                ] 37%
<|channel>thought
*   Topic: Why diffusion language models are faster than autoregressive ones.
    *   Constraint: Exactly three sentences.

    *   *Autoregressive (AR) models:* Generate one token at a time (sequentially). To generate a sequence of length $N$, they need $N$ forward passes through the neural network. This is $O(N)$ complexity.
    *   *Diffusion models:* Generate the entire sequence in parallel (or through a few iterative denoising steps). While they require multiple steps, the number of steps is often independent of the sequence length or significantly fewer than the length for long sequences.

    *   *Sentence 1:* Autoregressive models must generate tokens one by one, meaning the number of required forward passes scales linearly with the length of the output sequence.
    *   *Sentence 2:* In contrast, diffusion models can refine the entire sequence simultaneously through a fixed or limited number of denoising steps.
    *   *Sentence 3:* Because the number of iterations in diffusion models is often constant or independent of the total sequence length, they can achieve faster inference speeds for long-form text.

    *   Sentence 1: Autoregressive models generate tokens sequentially,
total time: 2396.01ms, time per step: 126.11ms (19 steps over 1 blocks, entropy-bound)
throughput: 106.8 tok/s (256 tok in 2396.01ms), in-step parallel 2030 tok/s (256-tok canvas x 19.0 steps/block)
WALL_SECONDS=10.156527687 RC=0