Xiaomi MiMo-V2.5-Pro 1.02T Architecture
Back to Xiaomi MiMo-V2.5-Pro 1.02T in the gallery
Xiaomi
Trillion-parameter sparse MoE model for long-horizon agentic and software-engineering tasks.
Architecture and specifications
- Scale
- 1.02T total, 42B active (4.1% active)
- Context (tokens)
- 1,048,576
- Vocabulary
- 152,576 (~152k)
- License
- MIT License
- Date
- 2026-04-22
- Decoder type
- Sparse MoE
- Attention
- GQA with 6:1 sliding-window/global attention
- Layer mix
- 60 sliding-window + 10 global
- KV cache / token (bf16) info
- 350 KiB · Very high
- Key detail
- Scales the MiMo-V2.5 Pro stack to 70 layers, 384 routed experts, top-8 routing, and a 1M-token context window.
Training and benchmarks
- Pretraining data
- 27T tokens Reported pretraining budget for the Pro model. Model card
- Post-training methods
- SFT, RL, Distillation SFT, domain-specialized RL teachers, and multi-teacher on-policy distillation (MOPD). Model card