DeepSeek V4.1 Flash: Reducing KV Cache to 890 Bytes per Token

Prompt Engineering2026-09-13go watch the original →

the gist

DeepSeek V4.1 Flash achieves a 437x reduction in KV cache memory footprint by splitting the transformer stack, sharing memory across layers, and using 4-bit quantization to enable efficient 1-million-token context windows.

Architectural Breakthroughs in Memory Efficiency

DeepSeek V4.1 Flash reduces the KV cache footprint to 890 bytes per token, down from 3.5 MB in V4 Flash and 389 MB in their 2023 models. The architecture achieves this through four primary modifications:

Causal Encoder-Decoder (CED) Split: The 40-layer transformer is split into two halves. The bottom 20 layers act as an encoder, and the top 20 act as a decoder. The decoder does not compute its own KV cache, instead projecting a shared set of keys and values from the final encoder layer.
Compressed Sparse Attention (CSA2): Layers are categorized into full-compute, re-index, and reuse layers. Re-index layers borrow memory from preceding layers but rerun the attention picker, while reuse layers copy both the nodes and the selection, significantly reducing redundant writes.
4-bit Quantization: Global memory nodes are quantized to 4-bit precision, using a high-precision scale factor for every 16 values to maintain numerical stability.
Local Cache Replay: Instead of saving local sliding window caches to disk, the model discards them and reconstructs the state on the fly by replaying the recent token window through the network during the next interaction.

Performance and Trade-offs

The model utilizes a Mixture-of-Experts (MoE) configuration with 552 billion total parameters, activating only 8 billion parameters for reading (prefill) and 16 billion for writing (decode). While it achieves high scores on developer-centric benchmarks like Terminal Bench 2.1 (90.6) and Codeforces (3,471 ELO), it shows a performance collapse on complex scientific agent tasks, such as Terminal Bench 4.0 (31.2 vs. 51.8 for Claude 3 Opus). The model trades encyclopedic factual recall for extreme efficiency in logical reasoning and code-heavy workflows.

Inference Optimization

Beyond the core architecture, the model integrates two specific speed modules:

ENGRAM: A 196 billion parameter lookup table stored in system memory that retrieves common phrases without consuming GPU compute.
DSpark: A speculative decoding engine that drafts five tokens ahead for parallel verification by the main model.