Back to Kimi K3.1 Pro in the gallery

Moonshot AI

Post-trained from Kimi K3, with its 24 Gated MLA layers converted to Gated DeepSeek Sparse Attention. The figure shows the text backbone of this native multimodal model.

Kimi K3.1 Pro architecture diagram
Kimi K3.1 Pro architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
2.8T total, 104B active (3.7% active)
Context (tokens)
1,048,576
Vocabulary
163,840 (~160k)
License
Kimi K3.1 Pro License
Date
2026-11-04
Decoder type
Sparse hybrid
Attention
Kimi Delta Attention and Gated MLA with DeepSeek Sparse Attention and NoPE
Layer mix
69 KDA + 24 Gated DSA
KV cache / token (bf16) info
33 KiB · Low
Key detail
Keeps eight Block Attention Residual groups and Stable LatentMoE with 16 of 896 routed experts plus two shared experts per token. The DSA indexer selects up to 2,048 tokens using 32 heads of 128 dimensions.

Training and benchmarks

Pretraining data
Not disclosed Derived from Kimi K3; the release card does not disclose a pretraining-token total for K3.1 Pro. Model card
Post-training methods
RL The K3.1 Pro update is post-trained from K3 using large-scale joint RL across hundreds of domains. The card does not specify the reward types. Model card

Further reading and tools