Back to Nanbeige 4.1 3B in the gallery

BOSS Zhipin

Small on-device oriented model that stays close to Llama 3.2 while nudging the scaling choices.

Nanbeige 4.1 3B architecture diagram
Nanbeige 4.1 3B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
3B parameters
Context (tokens)
262,144
License
Apache License 2.0
Date
2026-02-10
Decoder type
Dense
Attention
GQA
Layer mix
32 GQA
KV cache / token (bf16) info
64 KiB · Low
Key detail
Llama-like stack without tying input embeddings to the output layer.

Training and benchmarks

Pretraining data
23T tokens (inherited base) Uses Nanbeige4-3B-Base, whose four pretraining phases total 23T tokens. The 4.1 release adds post-training. Nanbeige 4 base model card · Nanbeige 4.1 technical report
Post-training methods
SFT, RLHF, RLVR SFT, human-preference reward-model RL, pairwise RL, and reasoning/code RL with executable correctness checks. Nanbeige 4.1 technical report
AA Intelligence Index v4.3.2 info
Total score 8.4 General N/A Scientific 5.4 Coding N/A Agents N/A

Further reading and tools