Back to Soofi S 30B-A3B in the gallery

SOOFI Consortium

German-English base model that retrains the Nemotron 3 Nano architecture on 26.68T tokens with a fully documented data mixture.

Soofi S 30B-A3B architecture diagram
Soofi S 30B-A3B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
31.6B total, 3.2B active (10.1% active)
Context (tokens)
1,048,576
License
Custom license (preview)
Date
2026-07-10
Decoder type
Hybrid MoE
Attention
Mostly Mamba-2 with 6 GQA layers and no positional embeddings (NoPE)
Layer mix
6 GQA + 23 Mamba-2 + 23 MoE
KV cache / token (bf16) info
6 KiB · Very low
Key detail
Reuses the Nemotron 3 Nano architecture without modification, isolating the effect of a German-English data and training recipe.

Training and benchmarks

Pretraining data
~26.68T tokens Consumed tokens across pretraining, annealing, and context extension; the report rounds this to ~27T. Technical report
Post-training methods
None (base model) This entry is Soofi-S-Base. Instruction and reasoning post-training are described as future work. Technical report

Further reading and tools