Back to Qwen3 Next 80B-A3B in the gallery

Alibaba Cloud

Efficiency-focused Qwen refresh that swaps standard attention for a DeltaNet-attention hybrid.

Qwen3 Next 80B-A3B architecture diagram
Qwen3 Next 80B-A3B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
80B total, 3B active (3.8% active)
Context (tokens)
262,144 native (1,010,000 with YaRN)
License
Apache License 2.0
Date
2025-09-09
Decoder type
Sparse hybrid
Attention
3:1 Gated DeltaNet and gated attention with GQA
Layer mix
12 gated attention + 36 DeltaNet
KV cache / token (bf16) info
24 KiB · Very low
Key detail
Adds many more experts, a shared expert, native 262k context, and MTP for speculative decoding.

Training and benchmarks

Pretraining data
15T tokens Base pretraining count reported for Qwen3-Next-80B-A3B. Model card
Post-training methods
RL The release describes RL for the Instruct and Thinking variants without detailing the full recipe. Release blog
AA Intelligence Index v4.3.2 info
Total score 9.6 General N/A Scientific 3.8 Coding N/A Agents N/A

Further reading and tools