Mistral Small 3.1 24B Architecture
Back to Mistral Small 3.1 24B in the gallery
Mistral AI
Fast dense 24B model that drops the sliding-window setup used in older Mistral releases.
Architecture and specifications
- Scale
- 24B parameters
- Context (tokens)
- 131,072
- Vocabulary
- 131,072
- License
- Apache License 2.0
- Date
- 2025-03-18
- Decoder type
- Dense
- Attention
- GQA with 32 query heads and 8 KV heads
- Layer mix
- 40 GQA
- KV cache / token (bf16) info
- 160 KiB · Moderate
- Key detail
- Latency-focused design with a smaller KV cache and fewer layers than Gemma 3 27B.
Training and benchmarks
- Pretraining data
- Not disclosed No pretraining-token total is given for this version. Mistral Small 3.1 Instruct model card
- Post-training methods
- Instruction tuning The released Instruct counterpart is instruction-tuned; the detailed supervised, preference, or RL recipe is not specified. Mistral Small 3.1 Instruct model card