LFM2.5 350M Architecture
Back to LFM2.5 350M in the gallery
Liquid AI
Liquid AI's compact LFM2.5 model combines double-gated LIV convolution blocks with periodic GQA layers.
Architecture and specifications
- Scale
- 350M parameters
- Context (tokens)
- 32,768
- Vocabulary
- 65,536 (~65k)
- License
- LFM Open License v1.0
- Date
- 2026-05-28
- Decoder type
- Dense hybrid
- Attention
- LIV convolution blocks plus GQA with QK-Norm
- Layer mix
- 10 LIV conv + 6 GQA
- KV cache / token (bf16) info
- 12 KiB · Very low
- Key detail
- GQA appears in layers 3, 6, 9, 11, 13, and 15 of the 16-layer stack. The 4,608-wide feed-forward network follows each sequence mixer.
Training and benchmarks
- Pretraining data
- 28T tokens The pretraining budget was extended from 10T to 28T tokens. Model card · Release post
- Post-training methods
- RL The release confirms large-scale multi-stage reinforcement learning without a complete breakdown of the alignment methods. Release post