Back to Sarvam 30B in the gallery

Sarvam AI

Reasoning-oriented Indian-language sparse MoE that keeps GQA at the smaller size.

Sarvam 30B architecture diagram
Sarvam 30B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
30B total, 2.4B active (8% active)
Context (tokens)
131,072
License
Apache License 2.0
Date
2026-03-03
Decoder type
Sparse MoE
Attention
GQA with QK-Norm
Layer mix
19 GQA
KV cache / token (bf16) info
19 KiB · Very low
Key detail
Large vocabulary and strong Indic language support paired with a reasoning-focused sparse MoE design.

Training and benchmarks

Pretraining data
16T tokens Reported pretraining budget. Training overview
Post-training methods
SFT, RLVR RL combines verifiable outcomes with rubric-based rewards, using asynchronous group-relative optimization. Training overview
AA Intelligence Index v4.3.2 info
Total score 6.6 General N/A Scientific 3.9 Coding N/A Agents N/A

Further reading and tools