Back to GPT-OSS 20B in the gallery

OpenAI

OpenAI's smaller open-weight MoE model favors width and alternating local/global attention.

GPT-OSS 20B architecture diagram
GPT-OSS 20B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
21B total, 3.6B active (17.1% active)
Context (tokens)
131,072
License
Apache License 2.0
Date
2025-08-04
Decoder type
Sparse MoE
Attention
GQA with alternating sliding-window and global layers
Layer mix
12 sliding-window + 12 global
KV cache / token (bf16) info
48 KiB · Low
Key detail
Wider and shallower than Qwen3, with attention bias and sink mechanisms.

Training and benchmarks

Pretraining data
Trillions of tokens; exact count undisclosed Text-only pretraining; no per-model token total is published. gpt-oss model card
Post-training methods
SFT, RL SFT followed by high-compute reasoning and tool-use RL, with deliberative alignment and instruction-hierarchy training. gpt-oss release
AA Intelligence Index v4.3.2 info
Total score 9.0 General 12.8 Scientific 6.2 Coding 19.4 Agents 0.0

Further reading and tools