GLM-4.7 355B Architecture
Back to GLM-4.7 355B in the gallery
Z.ai
Immediate GLM predecessor that stays closer to the older GLM-4.5 style before the MLA shift.
Architecture and specifications
- Scale
- 355B total, 32B active (9% active)
- Context (tokens)
- 202,752
- License
- MIT License
- Date
- 2025-12-22
- Decoder type
- Sparse MoE
- Attention
- GQA with QK-Norm
- Layer mix
- 92 GQA
- KV cache / token (bf16) info
- 368 KiB · Very high
- Key detail
- Serves as the pre-MLA, pre-sparse-attention baseline and uses an MTP-3-style setup.
Training and benchmarks
- Pretraining data
- 23T tokens (inherited GLM-4.5 base) The team confirms GLM-4.5, 4.6, and 4.7 share a base model; 23T is its pretraining and mid-training budget. GLM-4.5 report · GLM team AMA
- Post-training methods
- SFT, RL Reprocessed SFT data followed by RL using the slime framework; the full reward recipe is not specified. GLM team AMA