Back to LFM3 1.7B in the gallery

Liquid AI

Liquid AI's 1.73B dense hybrid combines short convolutions with sliding-window and global GQA in a 24-layer stack.

LFM3 1.7B architecture diagram
LFM3 1.7B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
1.73B parameters
Context (tokens)
262,144
Vocabulary
128,000
License
LFM Open License v1.0
Date
2026-10-27
Decoder type
Dense hybrid
Attention
ShortConv plus gated GQA with QK-Norm and sliding-window/global attention
Layer mix
13 ShortConv + 5 sliding-window + 6 global GQA
KV cache / token (bf16) info
22 KiB · Very low
Key detail
Uses kernel-7 ShortConv, a 256-token attention window, per-head sigmoid gates, and sandwich RMSNorm. Global attention uses 50% partial RoPE.

Training and benchmarks

Pretraining data
22T tokens Includes the last 3.5T tokens of midtraining and context extension; the separate SFT budget is excluded. Model card
Post-training methods
SFT, RL, Distillation SFT is followed by mixed-domain RL, on-policy distillation from six specialist RL teachers, and a final agentic RL stage. Model card

Further reading and tools