Qwen3 32B Architecture
Back to Qwen3 32B in the gallery
Alibaba Cloud
Large dense Qwen3 model that serves as the clearest like-for-like comparison for OLMo 3 32B.
Architecture and specifications
- Scale
- 32B parameters
- Context (tokens)
- 32,768 native (131,072 with YaRN)
- License
- Apache License 2.0
- Date
- 2025-04-28
- Decoder type
- Dense
- Attention
- GQA with QK-Norm
- Layer mix
- 64 GQA
- KV cache / token (bf16) info
- 256 KiB · High
- Key detail
- Reference dense Qwen stack with QK-Norm and 8 KV heads.
Training and benchmarks
- Pretraining data
- ≈36T tokens Family pretraining budget; excludes post-training. Qwen3 technical report
- Post-training methods
- SFT, RLVR, RLHF Cold-start/fusion SFT, reasoning RL, and general RL with human-preference rewards. Qwen3 technical report