Back to Grok 2.5 270B in the gallery

xAI

Rare production-model release that shows an older MoE style with fewer, larger experts.

Grok 2.5 270B architecture diagram
Grok 2.5 270B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
270B parameters
Context (tokens)
131,072
License
Grok 2 Community License Agreement
Date
2025-08-22
Decoder type
Sparse MoE
Attention
GQA
Layer mix
64 GQA
KV cache / token (bf16) info
256 KiB · High
Key detail
Adds an always-on SwiGLU path that effectively behaves like a shared expert.

Training and benchmarks

Pretraining data
Not disclosed The released Grok 2 checkpoint does not publish its training-data size. Grok 2 model card
Post-training methods
Not disclosed The release is post-trained, but its model card does not specify SFT, DPO, or reward methods. Grok 2 model card

Further reading and tools