Back to Command A+ 218B-A25B in the gallery

Cohere

Cohere's open sparse MoE model scales the Command A parallel transformer block to 218B total parameters while keeping 25B active per token.

Command A+ 218B-A25B architecture diagram
Command A+ 218B-A25B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
218B total, 25B active/token (11.5% active)
Context (tokens)
128K input, 64K output
Vocabulary
262,144
License
Apache License 2.0
Date
2026-05-20
Decoder type
Sparse MoE
Attention
16:1 GQA with 3:1 sliding-window/global attention
Layer mix
24 sliding-window + 8 global GQA/MoE
KV cache / token (bf16) info
128 KiB · Moderate
Key detail
Runs attention and MoE in parallel, which Cohere reports gives similar quality to a sequential block with better throughput.

Training and benchmarks

Pretraining data
Not disclosed The model card does not report a pretraining token count. Model card
Post-training methods
Distillation Quantization-aware distillation (QAD) is confirmed for the W4A4 release. The broader alignment recipe is not disclosed. Model card
AA Intelligence Index v5.0 info
Total score 6.2 Knowledge Work 0.6 Coding 0.2 Science 1.0 General 23.2
AA Intelligence Index v4.3.2 info
Total score 13.1 General 26.4 Scientific 6.1 Coding 19.5 Agents 0.5

Further reading and tools