GLM-5 744B Architecture
Back to GLM-5 744B in the gallery
Z.ai
Huge GLM refresh that adopts both MLA and DeepSeek Sparse Attention for flagship-scale inference.
Architecture and specifications
- Scale
- 744B total, 40B active (5.4% active)
- Context (tokens)
- 202,752
- License
- MIT License
- Date
- 2026-02-11
- Decoder type
- Sparse MoE
- Attention
- MLA with DeepSeek Sparse Attention
- Layer mix
- 78 MLA/DSA
- KV cache / token (bf16) info
- 107.25 KiB · Moderate
- Key detail
- Bigger than GLM-4.7, with more experts, fewer layers, and DeepSeek-style MTP.
Training and benchmarks
- Pretraining data
- 28.5T tokens Combined pretraining and mid-training budget for the base model. GLM-5 report
- Post-training methods
- SFT, RLVR, RL, Distillation SFT, reasoning/agentic/general RL, and on-policy cross-stage distillation. General RL combines rule-based and model-based rewards. GLM-5 report