DeepSeek V4-Flash Architecture
Back to DeepSeek V4-Flash in the gallery
DeepSeek
DeepSeek's efficient V4 preview keeps the million-token architecture while reducing the MoE scale to 284B parameters and 13B active parameters.
Architecture and specifications
- Scale
- 284B total, 13B active (4.6% active)
- Context (tokens)
- 1,048,576
- License
- MIT License
- Date
- 2026-04-24
- Decoder type
- Sparse MoE
- Attention
- MLA-style CSA/HCA with mHC
- Layer mix
- 43 CSA/HCA
- KV cache / token (bf16) info
- 6.72 KiB (growing global cache only) · Very low
- Key detail
- Uses 256 experts, 6 routed plus 1 shared expert per token, hash-based routing, compressed attention, and the V4 MTP path.
Training and benchmarks
- Pretraining data
- 32T tokens Includes staged context extension to 1M tokens. DeepSeek V4 report
- Post-training methods
- SFT, RLVR, RL, Distillation Domain experts receive SFT and GRPO, using verifiers and generative rewards, then are combined through on-policy distillation. DeepSeek V4 report