Qwen3 Next 80B-A3B Architecture
Back to Qwen3 Next 80B-A3B in the gallery
Alibaba Cloud
Efficiency-focused Qwen refresh that swaps standard attention for a DeltaNet-attention hybrid.
Architecture and specifications
- Scale
- 80B total, 3B active (3.8% active)
- Context (tokens)
- 262,144 native (1,010,000 with YaRN)
- License
- Apache License 2.0
- Date
- 2025-09-09
- Decoder type
- Sparse hybrid
- Attention
- 3:1 Gated DeltaNet and gated attention with GQA
- Layer mix
- 12 gated attention + 36 DeltaNet
- KV cache / token (bf16) info
- 24 KiB · Very low
- Key detail
- Adds many more experts, a shared expert, native 262k context, and MTP for speculative decoding.
Training and benchmarks
- Pretraining data
- 15T tokens Base pretraining count reported for Qwen3-Next-80B-A3B. Model card
- Post-training methods
- RL The release describes RL for the Instruct and Thinking variants without detailing the full recipe. Release blog