Command A+ 218B-A25B Architecture
Back to Command A+ 218B-A25B in the gallery
Cohere
Cohere's open sparse MoE model scales the Command A parallel transformer block to 218B total parameters while keeping 25B active per token.
Architecture and specifications
- Scale
- 218B total, 25B active/token (11.5% active)
- Context (tokens)
- 128K input, 64K output
- Vocabulary
- 262,144
- License
- Apache License 2.0
- Date
- 2026-05-20
- Decoder type
- Sparse MoE
- Attention
- 16:1 GQA with 3:1 sliding-window/global attention
- Layer mix
- 24 sliding-window + 8 global GQA/MoE
- KV cache / token (bf16) info
- 128 KiB · Moderate
- Key detail
- Runs attention and MoE in parallel, which Cohere reports gives similar quality to a sequential block with better throughput.
Training and benchmarks
- Pretraining data
- Not disclosed The model card does not report a pretraining token count. Model card
- Post-training methods
- Distillation Quantization-aware distillation (QAD) is confirmed for the W4A4 release. The broader alignment recipe is not disclosed. Model card