Back to GLM-4.5 355B in the gallery

Z.ai

Agent-oriented instruction/reasoning hybrid that borrows DeepSeek's dense-prefix MoE layout.

GLM-4.5 355B architecture diagram
GLM-4.5 355B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
355B total, 32B active (9% active)
Context (tokens)
131,072
License
MIT License
Date
2025-07-28
Decoder type
Sparse MoE
Attention
GQA with QK-Norm
Layer mix
92 GQA
KV cache / token (bf16) info
368 KiB · Very high
Key detail
Starts with three dense layers before MoE routing, keeps a shared expert, and uses MTP during training.

Training and benchmarks

Pretraining data
23T tokens Combined pretraining and mid-training budget. GLM-4.5 report
Post-training methods
SFT, RLVR, RLHF, RLAIF, Distillation SFT and expert RL cover reasoning, agents, and chat; expert capabilities are distilled into the final model. GLM-4.5 report
AA Intelligence Index v4.3.2 info
Total score 12.8 General N/A Scientific 6.5 Coding N/A Agents N/A

Further reading and tools