Back to Xiaomi MiMo-V2-Flash 309B in the gallery

Xiaomi

Large MoE model that pushes sliding-window attention harder than most contemporaries.

Xiaomi MiMo-V2-Flash 309B architecture diagram
Xiaomi MiMo-V2-Flash 309B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
309B total, 15B active (4.9% active)
Context (tokens)
262,144
License
MIT License
Date
2025-12-16
Decoder type
Sparse MoE
Attention
GQA with 8 sliding-window and 4 global KV heads. Keys have 192 dimensions and values have 128.
Layer mix
39 sliding-window + 9 global
KV cache / token (bf16) info
217.5 KiB · High
Key detail
Uses an unusually small 128-token local window plus multi-token prediction.

Training and benchmarks

Pretraining data
27T tokens Reported pretraining budget. Model card
Post-training methods
SFT, RL, Distillation, RLVR SFT, domain-specialized RL, and multi-teacher on-policy distillation. Verifiable tasks use programmatic and LLM verification; subjective helpfulness and safety use rubric-based LLM rewards. Technical report
AA Intelligence Index v4.3.2 info
Total score 16.0 General N/A Scientific 4.3 Coding N/A Agents N/A

Further reading and tools