Back to GLM-4.7 355B in the gallery

Z.ai

Immediate GLM predecessor that stays closer to the older GLM-4.5 style before the MLA shift.

GLM-4.7 355B architecture diagram
GLM-4.7 355B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
355B total, 32B active (9% active)
Context (tokens)
202,752
License
MIT License
Date
2025-12-22
Decoder type
Sparse MoE
Attention
GQA with QK-Norm
Layer mix
92 GQA
KV cache / token (bf16) info
368 KiB · Very high
Key detail
Serves as the pre-MLA, pre-sparse-attention baseline and uses an MTP-3-style setup.

Training and benchmarks

Pretraining data
23T tokens (inherited GLM-4.5 base) The team confirms GLM-4.5, 4.6, and 4.7 share a base model; 23T is its pretraining and mid-training budget. GLM-4.5 report · GLM team AMA
Post-training methods
SFT, RL Reprocessed SFT data followed by RL using the slime framework; the full reward recipe is not specified. GLM team AMA
AA Intelligence Index v4.3.2 info
Total score 17.4 General N/A Scientific 3.2 Coding N/A Agents N/A

Further reading and tools