Back to Kimi K2 in the gallery

Moonshot AI

Trillion-parameter Moonshot model that essentially scales the DeepSeek V3 recipe upward.

Kimi K2 architecture diagram
Kimi K2 architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
1T total, 32B active (3.2% active)
Context (tokens)
131,072
License
Modified MIT License
Date
2025-07-10
Decoder type
Sparse MoE
Attention
MLA
Layer mix
61 MLA
KV cache / token (bf16) info
68.6 KiB · Low
Key detail
More experts and fewer MLA heads than DeepSeek V3.

Training and benchmarks

Pretraining data
15.5T tokens Total tokens used to pretrain the released Kimi K2 base. Technical report
Post-training methods
SFT, RLVR Kimi-K2-Instruct uses SFT and RL with verifiable rewards, plus self-critique rubric rewards for open-ended tasks. The gallery architecture is shared with the base model. Technical report
AA Intelligence Index v4.3.2 info
Total score 12.7 General N/A Scientific 3.7 Coding N/A Agents N/A

Further reading and tools