Back to Qwen3 32B in the gallery

Alibaba Cloud

Large dense Qwen3 model that serves as the clearest like-for-like comparison for OLMo 3 32B.

Qwen3 32B architecture diagram
Qwen3 32B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
32B parameters
Context (tokens)
32,768 native (131,072 with YaRN)
License
Apache License 2.0
Date
2025-04-28
Decoder type
Dense
Attention
GQA with QK-Norm
Layer mix
64 GQA
KV cache / token (bf16) info
256 KiB · High
Key detail
Reference dense Qwen stack with QK-Norm and 8 KV heads.

Training and benchmarks

Pretraining data
≈36T tokens Family pretraining budget; excludes post-training. Qwen3 technical report
Post-training methods
SFT, RLVR, RLHF Cold-start/fusion SFT, reasoning RL, and general RL with human-preference rewards. Qwen3 technical report
AA Intelligence Index v4.3.2 info
Total score 7.3

Further reading and tools