DeepSeek V3.2 Architecture
Back to DeepSeek V3.2 in the gallery
DeepSeek
DeepSeek's successor keeps the V3 template but adds sparse attention to cut long-context costs.
Architecture and specifications
- Scale
- 671B total, 37B active (5.5% active)
- Context (tokens)
- 128,000
- License
- MIT License
- Date
- 2025-12-01
- Decoder type
- Sparse MoE
- Attention
- MLA with DeepSeek Sparse Attention
- Layer mix
- 61 MLA/DSA
- KV cache / token (bf16) info
- 83.875 KiB · Moderate
- Key detail
- Adds sparse attention while retaining the DeepSeek V3-style MTP path.
Training and benchmarks
- Pretraining data
- 943.7B continued-training tokens + 2.1B indexer warm-up Additional DSA training from V3.1-Terminus. The report does not give a cumulative pretraining total. DeepSeek V3.2 report
- Post-training methods
- SFT, RLVR, RL, Distillation Specialist distillation and mixed GRPO; rule-based reasoning/agent rewards and rubric-based generative rewards. Tool-use cold start uses SFT. DeepSeek V3.2 report