GPT-OSS 120B Architecture
Back to GPT-OSS 120B in the gallery
OpenAI
Larger gpt-oss variant keeps the same alternating-attention recipe as the 20B model.
Architecture and specifications
- Scale
- 117B total, 5.1B active (4.4% active)
- Context (tokens)
- 131,072
- License
- Apache License 2.0
- Date
- 2025-08-04
- Decoder type
- Sparse MoE
- Attention
- GQA with alternating sliding-window and global layers
- Layer mix
- 18 sliding-window + 18 global
- KV cache / token (bf16) info
- 72 KiB · Low
- Key detail
- Shared architectural template scaled up for OpenAI's flagship open-weight release.
Training and benchmarks
- Pretraining data
- Trillions of tokens; exact count undisclosed Text-only pretraining; no per-model token total is published. gpt-oss model card
- Post-training methods
- SFT, RL SFT followed by high-compute reasoning and tool-use RL, with deliberative alignment and instruction-hierarchy training. gpt-oss release