Back to DeepSeek V4-Flash in the gallery

DeepSeek

DeepSeek's efficient V4 preview keeps the million-token architecture while reducing the MoE scale to 284B parameters and 13B active parameters.

DeepSeek V4-Flash architecture diagram
DeepSeek V4-Flash architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
284B total, 13B active (4.6% active)
Context (tokens)
1,048,576
License
MIT License
Date
2026-04-24
Decoder type
Sparse MoE
Attention
MLA-style CSA/HCA with mHC
Layer mix
43 CSA/HCA
KV cache / token (bf16) info
6.72 KiB (growing global cache only) · Very low
Key detail
Uses 256 experts, 6 routed plus 1 shared expert per token, hash-based routing, compressed attention, and the V4 MTP path.

Training and benchmarks

Pretraining data
32T tokens Includes staged context extension to 1M tokens. DeepSeek V4 report
Post-training methods
SFT, RLVR, RL, Distillation Domain experts receive SFT and GRPO, using verifiers and generative rewards, then are combined through on-policy distillation. DeepSeek V4 report
AA Intelligence Index v4.3.2 info
Total score 34.3 General 31.8 Scientific 27.6 Coding 31.2 Agents 43.5

Further reading and tools