Back to Reflection Beam 501B in the gallery

Reflection

Text-only MoE decoder with three dense FFN layers followed by 49 expert layers. Each MoE layer activates 8 of 256 experts without a shared expert.

Reflection Beam 501B architecture diagram
Reflection Beam 501B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
501.3B total, 23.4B active (4.7% active)
Context (tokens)
1,048,576
Vocabulary
151,936 (~152k)
License
Apache License 2.0
Date
2026-10-16
Decoder type
Sparse MoE
Attention
Gated GQA with QK-Norm and sliding-window/global attention
Layer mix
43 sliding-window + 9 global GQA
KV cache / token (bf16) info
208 KiB · High
Key detail
Combines 512-token local RoPE attention with global NoPE and query scaling. Uses sandwich RMSNorm, depth-scaled residuals, and a final logit softcap of 30.

Training and benchmarks

Pretraining data
Not disclosed The card describes the text/code corpus and mid-training sources without a token count. Model card
Post-training methods
RL RL environments cover coding, agentic tasks, and STEM; the full recipe is not disclosed. Model card
AA Intelligence Index v5.0 info
Total score 17.7 Knowledge Work 26.3 Coding 4.5 Science 10.9 General 29.2

Further reading and tools