Back to VibeThinker-3B in the gallery

Weibo

Qwen2.5-Coder-based dense reasoner whose gains mainly come from verifier-driven post-training rather than a new block design.

VibeThinker-3B architecture diagram
VibeThinker-3B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
3B parameters
Context (tokens)
131,072
Vocabulary
151,936 (~152k)
License
MIT License
Date
2026-06-15
Decoder type
Dense
Attention
8:1 GQA with RoPE
Layer mix
36 GQA
KV cache / token (bf16) info
36 KiB · Low
Key detail
Standard Qwen2.5-Coder-style dense stack strengthened by curriculum SFT, MGPO RL, Long2Short RL, offline self-distillation, and Instruct RL.

Training and benchmarks

Pretraining data
5.5T tokens (base-model coding training) Inherited Qwen2.5-Coder-3B budget; VibeThinker adds post-training rather than a new reported pretraining run. Technical report · Model card
Post-training methods
SFT, RLVR, Distillation, RL Two-stage SFT, verifiable reasoning RL, offline self-distillation, then instruction RL with rubric rewards. Technical report

Further reading and tools