Kimi K2.5 Architecture
Back to Kimi K2.5 in the gallery
Moonshot AI
Native-multimodal Moonshot flagship that keeps the K2/DeepSeek-style MoE layout and pushes native context to 256k.
Architecture and specifications
- Scale
- 1T total, 32B active (3.2% active)
- Context (tokens)
- 262,144
- License
- Modified MIT License
- Date
- 2026-01-27
- Decoder type
- Sparse MoE
- Attention
- MLA
- Layer mix
- 61 MLA
- KV cache / token (bf16) info
- 68.6 KiB · Low
- Key detail
- Keeps the 384-expert K2 backbone, but adds multimodal capabilities (not shown) and doubles the native context length.
Training and benchmarks
- Pretraining data
- ~15T additional tokens Continued pretraining from Kimi K2 on a mixed visual and text corpus; this excludes K2's earlier pretraining. Technical report
- Post-training methods
- SFT, RLVR SFT and RL combine verifiable outcome rewards with generative reward models. Parallel-agent RL trains the orchestrator to delegate work. Technical report