Xiaomi MiMo-V2-Flash 309B Architecture
Back to Xiaomi MiMo-V2-Flash 309B in the gallery
Xiaomi
Large MoE model that pushes sliding-window attention harder than most contemporaries.
Architecture and specifications
- Scale
- 309B total, 15B active (4.9% active)
- Context (tokens)
- 262,144
- License
- MIT License
- Date
- 2025-12-16
- Decoder type
- Sparse MoE
- Attention
- GQA with 8 sliding-window and 4 global KV heads. Keys have 192 dimensions and values have 128.
- Layer mix
- 39 sliding-window + 9 global
- KV cache / token (bf16) info
- 217.5 KiB · High
- Key detail
- Uses an unusually small 128-token local window plus multi-token prediction.
Training and benchmarks
- Pretraining data
- 27T tokens Reported pretraining budget. Model card
- Post-training methods
- SFT, RL, Distillation, RLVR SFT, domain-specialized RL, and multi-teacher on-policy distillation. Verifiable tasks use programmatic and LLM verification; subjective helpfulness and safety use rubric-based LLM rewards. Technical report