Kimi K2 Architecture
Back to Kimi K2 in the gallery
Moonshot AI
Trillion-parameter Moonshot model that essentially scales the DeepSeek V3 recipe upward.
Architecture and specifications
- Scale
- 1T total, 32B active (3.2% active)
- Context (tokens)
- 131,072
- License
- Modified MIT License
- Date
- 2025-07-10
- Decoder type
- Sparse MoE
- Attention
- MLA
- Layer mix
- 61 MLA
- KV cache / token (bf16) info
- 68.6 KiB · Low
- Key detail
- More experts and fewer MLA heads than DeepSeek V3.
Training and benchmarks
- Pretraining data
- 15.5T tokens Total tokens used to pretrain the released Kimi K2 base. Technical report
- Post-training methods
- SFT, RLVR Kimi-K2-Instruct uses SFT and RL with verifiable rewards, plus self-critique rubric rewards for open-ended tasks. The gallery architecture is shared with the base model. Technical report