Back to Qwen3 0.6B in the gallery

Alibaba Cloud

Tiny current-generation Qwen model that trades width for more depth and a low memory footprint.

Qwen3 0.6B architecture diagram
Qwen3 0.6B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
0.6B parameters
Context (tokens)
32,768
License
Apache License 2.0
Date
2025-04-28
Decoder type
Dense
Attention
GQA
Layer mix
28 GQA
KV cache / token (bf16) info
112 KiB · Moderate
Key detail
Deeper than Llama 3.2 1B and unusually practical for local experiments and teaching.

Training and benchmarks

Pretraining data
≈36T tokens Family pretraining budget; excludes post-training. Qwen3 technical report
Post-training methods
SFT, Distillation Instruction/reasoning counterpart uses response-supervised and on-policy teacher distillation. Base checkpoints have no post-training. Qwen3 technical report
AA Intelligence Index v4.3.2 info
Total score 4.8 General N/A Scientific 2.5 Coding N/A Agents N/A

Further reading and tools