Back to Mistral Small 3.1 24B in the gallery

Mistral AI

Fast dense 24B model that drops the sliding-window setup used in older Mistral releases.

Mistral Small 3.1 24B architecture diagram
Mistral Small 3.1 24B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
24B parameters
Context (tokens)
131,072
Vocabulary
131,072
License
Apache License 2.0
Date
2025-03-18
Decoder type
Dense
Attention
GQA with 32 query heads and 8 KV heads
Layer mix
40 GQA
KV cache / token (bf16) info
160 KiB · Moderate
Key detail
Latency-focused design with a smaller KV cache and fewer layers than Gemma 3 27B.

Training and benchmarks

Pretraining data
Not disclosed No pretraining-token total is given for this version. Mistral Small 3.1 Instruct model card
Post-training methods
Instruction tuning The released Instruct counterpart is instruction-tuned; the detailed supervised, preference, or RL recipe is not specified. Mistral Small 3.1 Instruct model card
AA Intelligence Index v4.3.2 info
Total score 7.1 General 12.8 Scientific 2.2 Coding 13.9 Agents 0.2

Further reading and tools