Back to Nemotron 3.5 Lightning 30B-A3B in the gallery

NVIDIA

NVIDIA's 30B hybrid model keeps 3B parameters active and extends the Nemotron 3 Nano backbone with a 1M-token context window and multi-token prediction.

Nemotron 3.5 Lightning 30B-A3B architecture diagram
Nemotron 3.5 Lightning 30B-A3B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
30B total, 3B active (10% active)
Context (tokens)
1,048,576
Vocabulary
131,072
License
OpenMDW 1.1
Date
2026-08-11
Decoder type
Hybrid MoE
Attention
Mostly Mamba-2 with six GQA layers using NoPE
Layer mix
6 GQA + 23 Mamba-2 + 23 MoE; GQA + MoE MTP predictor
KV cache / token (bf16) info
6 KiB · Very low
Key detail
Adds a shared GQA-plus-MoE MTP predictor to the 52-layer Nemotron 3 Nano backbone.

Training and benchmarks

Pretraining data
1.021T additional tokens Continued from Nemotron 3 Nano Base with 1T distillation tokens and 21B long-context tokens. A separate 100B-token MTP-head warmup freezes the backbone. Nemotron 3.5 technical report
Post-training methods
SFT, RLVR Single-stage text-only supervised fine-tuning followed by reinforcement learning with verifiable rewards. Nemotron 3.5 technical report
AA Intelligence Index v5.0 info
Total score 5.5 Knowledge Work 1.0 Coding 0.2 Science 0.6 General 20.6
AA Intelligence Index v4.3.2 info
Total score 12.9 General 26.3 Scientific 5.3 Coding 16.3 Agents 2.6

Further reading and tools