Back to Qwen3 30B-A3B in the gallery

Alibaba Cloud

Small sparse Qwen3 model that sits close to GPT-OSS in active size but uses a deeper stack.

Qwen3 30B-A3B architecture diagram
Qwen3 30B-A3B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
30B total, 3B active (10% active)
Context (tokens)
32,768 native (131,072 with YaRN)
License
Apache License 2.0
Date
2025-04-28
Decoder type
Sparse MoE
Attention
GQA
Layer mix
48 GQA
KV cache / token (bf16) info
96 KiB · Moderate
Key detail
Deeper, narrower MoE alternative to GPT-OSS without a shared expert.

Training and benchmarks

Pretraining data
≈36T tokens Family pretraining budget; excludes post-training. Qwen3 technical report
Post-training methods
SFT, Distillation Instruction/reasoning counterpart uses response-supervised and on-policy teacher distillation. Base checkpoints have no post-training. Qwen3 technical report
AA Intelligence Index v4.3.2 info
Total score 6.6 General N/A Scientific 2.3 Coding N/A Agents N/A

Further reading and tools