Ling 2.5 1T Architecture
Back to Ling 2.5 1T in the gallery
InclusionAI
Trillion-parameter long-context model that swaps DeltaNet for Lightning Attention.
Architecture and specifications
- Scale
- 1T total, 63B active (6.3% active)
- Context (tokens)
- 256K native (1M with YaRN)
- License
- MIT License
- Date
- 2026-02-15
- Decoder type
- Sparse hybrid
- Attention
- Lightning Attention plus MLA
- Layer mix
- 10 MLA + 70 Lightning Attention
- KV cache / token (bf16) info
- 11.2 KiB · Very low
- Key detail
- Uses a 7:1 linear-attention/MLA ratio and a much larger 63B active path.
Training and benchmarks
- Pretraining data
- 9T additional tokens Continued pretraining after retrofitting the Ling 2.0 base. The card separately describes a pretraining corpus expanded from 20T to 29T tokens. Model card
- Post-training methods
- RL Reported methods include correctness and redundancy rewards, bidirectional RL feedback, and agentic RL in interactive environments. Model card