xLSTM 7B Architecture
Back to xLSTM 7B in the gallery
NXAI
Recurrent 7B language model that replaces self-attention with xLSTM blocks built around matrix memory.
Architecture and specifications
- Scale
- 7B parameters
- Context (tokens)
- No explicit limit
- License
- NXAI Community License Agreement
- Date
- 2025-03-17
- Decoder type
- Recurrent
- Attention
- No self-attention; mLSTM recurrent layers with matrix memory
- Layer mix
- 32 mLSTM
- KV cache / token (bf16) info
- 0 B · No cache
- Key detail
- Stateful recurrent architecture aimed at fast long-context inference without an explicit context window.
Training and benchmarks
- Pretraining data
- ~2.3T tokens DCLM and selected high-quality data. Model card
- Post-training methods
- None (base model) The linked xLSTM-7b release is a pretrained completion model. Model card