INTELLECT-3 Architecture
Back to INTELLECT-3 in the gallery
Prime Intellect
Large-scale RL post-training of GLM-4.5-Air that keeps the compact 106B sparse MoE backbone intact.
Architecture and specifications
- Scale
- 106B total, 12B active (11.3% active)
- Context (tokens)
- 131,072 in config
- License
- MIT License
- Date
- 2025-11-26
- Decoder type
- Sparse MoE
- Attention
- GQA
- Layer mix
- 46 GQA
- KV cache / token (bf16) info
- 184 KiB · High
- Key detail
- Keeps the GLM-4.5-Air architecture unchanged and shifts the capability profile through SFT plus large-scale RL.
Training and benchmarks
- Pretraining data
- Not disclosed Post-trained from GLM-4.5-Air-Base. A pretraining-token total for that Air base is not disclosed in the checked sources. Technical report · Model card
- Post-training methods
- SFT, RLVR Two SFT phases are followed by large-scale RL with verifiable rewards across reasoning and agentic tasks. Technical report