Google DiffusionGemma: 1,000+ Tokens/Sec via Uniform State Diffusion

The Breakthrough

Google DeepMind's DiffusionGemma shifts text generation from a sequential, auto-regressive paradigm to a parallelized diffusion process, enabling the model to fill a 256-token canvas simultaneously rather than generating one token at a time.

How DiffusionGemma Works

• Uniform State Diffusion: Instead of masking tokens, the model treats random placeholder tokens as noise. It iteratively refines these tokens across multiple bidirectional passes, allowing the model to self-correct previous guesses as it gains context.
• Compute-Bound Architecture: By generating 256 tokens in a single pass, the model keeps the GPU busy with computation rather than waiting for memory-bound weight loading, which is the primary bottleneck for single-user local LLM inference.
• Bidirectional Attention: Unlike standard causal LLMs that only look backward, DiffusionGemma uses bidirectional attention, allowing every token position to attend to all other positions simultaneously to refine the output.
• Encoder-Denoising Hybrid: The model utilizes a 26-billion parameter Gemma 4 base, splitting operations into an encoder mode to process prompts into a KV cache and a denoising mode to iteratively clean the canvas.

Performance and Tradeoffs

• Speed: In practical testing on an H100 GPU, the model achieved generation speeds of approximately 700 tokens per second, significantly faster than standard auto-regressive models, though falling short of the theoretical 1,000+ tokens per second ceiling.
• Quality: The model prioritizes speed and non-linear tasks, such as code filling or inline editing, over the high-fidelity reasoning of standard auto-regressive models like Gemma 4.
• Deployment: The model is available on Hugging Face under an Apache 2.0 license and can be deployed via vLLM containers on platforms like RunPod for local experimentation.