Sarvam 30B Architecture
Back to Sarvam 30B in the gallery
Sarvam AI
Reasoning-oriented Indian-language sparse MoE that keeps GQA at the smaller size.
Architecture and specifications
- Scale
- 30B total, 2.4B active (8% active)
- Context (tokens)
- 131,072
- License
- Apache License 2.0
- Date
- 2026-03-03
- Decoder type
- Sparse MoE
- Attention
- GQA with QK-Norm
- Layer mix
- 19 GQA
- KV cache / token (bf16) info
- 19 KiB · Very low
- Key detail
- Large vocabulary and strong Indic language support paired with a reasoning-focused sparse MoE design.
Training and benchmarks
- Pretraining data
- 16T tokens Reported pretraining budget. Training overview
- Post-training methods
- SFT, RLVR RL combines verifiable outcomes with rubric-based rewards, using asynchronous group-relative optimization. Training overview