Back to Llama 4 Maverick in the gallery

Meta

Meta's large MoE follows the DeepSeek V3 playbook but with a more conventional attention stack.

Llama 4 Maverick architecture diagram
Llama 4 Maverick architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
400B total, 17B active (4.3% active)
Context (tokens)
1,000,000
License
Llama 4 Community License Agreement
Date
2025-04-05
Decoder type
Sparse MoE
Attention
GQA with RoPE in chunked layers and NoPE in full-attention layers
Layer mix
36 chunked + 12 full GQA
KV cache / token (bf16) info
192 KiB · High
Key detail
Alternates dense and MoE blocks and uses fewer, larger experts than DeepSeek V3.

Training and benchmarks

Pretraining data
~22T tokens Pretraining-token count reported for Maverick, including its multimodal training data. Model card
Post-training methods
SFT, RL, DPO The Maverick Instruct recipe uses SFT, online RL, and a final lightweight DPO stage. Release post
AA Intelligence Index v4.3.2 info
Total score 9.3 General 18.8 Scientific 2.5 Coding 15.9 Agents 0.0

Further reading and tools