Back to Qwen3 235B-A22B in the gallery

Alibaba Cloud

Large sparse Qwen variant that stays very close to DeepSeek V3 while removing the shared expert.

Qwen3 235B-A22B architecture diagram
Qwen3 235B-A22B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
235B total, 22B active (9.4% active)
Context (tokens)
32,768 native (131,072 with YaRN)
License
Apache License 2.0
Date
2025-04-28
Decoder type
Sparse MoE
Attention
GQA with QK-Norm
Layer mix
94 GQA
KV cache / token (bf16) info
188 KiB · High
Key detail
High-capacity MoE design optimized for serving efficiency without a shared expert.

Training and benchmarks

Pretraining data
≈36T tokens Family pretraining budget; excludes post-training. Qwen3 technical report
Post-training methods
SFT, RLVR, RLHF Cold-start/fusion SFT, reasoning RL, and general RL with human-preference rewards. Qwen3 technical report
AA Intelligence Index v4.3.2 info
Total score 8.3 General N/A Scientific 2.1 Coding N/A Agents N/A

Further reading and tools