Back to Nemotron 3 Super 120B-A12B in the gallery

NVIDIA

The Super variant scales up Nano and adds both latent experts and native speculative decoding support.

Nemotron 3 Super 120B-A12B architecture diagram
Nemotron 3 Super 120B-A12B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
120B total, 12B active (10% active)
Context (tokens)
1,000,000
License
NVIDIA Nemotron Open Model License
Date
2026-03-11
Decoder type
Hybrid MoE
Attention
Mostly Mamba-2 with a few GQA layers using NoPE
Layer mix
8 GQA + 40 Mamba-2 + 40 MoE
KV cache / token (bf16) info
8 KiB · Very low
Key detail
Adds latent-space MoE and shared-weight MTP for fast inference.

Training and benchmarks

Pretraining data
25T + 51B long-context tokens The main pretraining run uses 25T tokens. Two context-extension phases add 34B and 17B tokens. Nemotron 3 Super technical report
Post-training methods
SFT, RLVR, RLHF Supervised fine-tuning, multi-environment and software-engineering RL with verifiable rewards, then human-feedback alignment. Nemotron 3 Super technical report
AA Intelligence Index v4.3.2 info
Total score 12.8 General 22.1 Scientific 12.0 Coding 18.1 Agents 0.6

Further reading and tools