Back to DeepSeek R1 in the gallery

DeepSeek

Reasoning-tuned DeepSeek model built on the V3 architecture rather than a new base design.

DeepSeek R1 architecture diagram
DeepSeek R1 architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
671B total, 37B active (5.5% active)
Context (tokens)
128,000
License
MIT License
Date
2025-01-20
Decoder type
Sparse MoE
Attention
MLA
Layer mix
61 MLA
KV cache / token (bf16) info
68.6 KiB · Low
Key detail
Architecture matches DeepSeek V3, including the MTP-1 path; the main change is the reasoning-oriented training recipe.

Training and benchmarks

Pretraining data
14.8T tokens (inherited base) R1 starts from DeepSeek V3 Base; this is the base pretraining count, excluding R1 post-training. DeepSeek V3 report · DeepSeek R1 report
Post-training methods
SFT, RLVR, RLHF Cold-start SFT, reasoning GRPO, rejection-sampling SFT, then RL with verifiable rewards and human-preference reward models. DeepSeek R1 report
AA Intelligence Index v4.3.2 info
Total score 11.4 General 22.2 Scientific 4.5 Coding 19.2 Agents 0.0

Further reading and tools