GPT-OSS 20B Architecture
Back to GPT-OSS 20B in the gallery
OpenAI
OpenAI's smaller open-weight MoE model favors width and alternating local/global attention.
Architecture and specifications
- Scale
- 21B total, 3.6B active (17.1% active)
- Context (tokens)
- 131,072
- License
- Apache License 2.0
- Date
- 2025-08-04
- Decoder type
- Sparse MoE
- Attention
- GQA with alternating sliding-window and global layers
- Layer mix
- 12 sliding-window + 12 global
- KV cache / token (bf16) info
- 48 KiB · Low
- Key detail
- Wider and shallower than Qwen3, with attention bias and sink mechanisms.
Training and benchmarks
- Pretraining data
- Trillions of tokens; exact count undisclosed Text-only pretraining; no per-model token total is published. gpt-oss model card
- Post-training methods
- SFT, RL SFT followed by high-compute reasoning and tool-use RL, with deliberative alignment and instruction-hierarchy training. gpt-oss release