Back to Laguna S 2.1 in the gallery

Poolside

Poolside's 118B sparse MoE scales the Laguna XS recipe to 48 layers and 1M context while activating about 8B parameters per token.

Laguna S 2.1 architecture diagram
Laguna S 2.1 architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
118B total, 8B active (6.8% active)
Context (tokens)
1,048,576
Vocabulary
100,352 (~100k)
License
OpenMDW License 1.1
Date
2026-07-21
Decoder type
Sparse MoE
Attention
Gated GQA with QK-Norm and 3:1 sliding-window/global attention
Layer mix
36 sliding-window + 12 global GQA
KV cache / token (bf16) info
192 KiB · High
Key detail
Uses 256 routed experts with top-10 routing plus one shared expert, a dense first layer, and per-layer query-head counts.

Training and benchmarks

Pretraining data
Not disclosed Uses the same pretraining data as XS 2.1, but the release post does not give a token count for either run. Release post
Post-training methods
SFT, RLVR SFT with partly synthetic data is followed by RL on harder tasks, including software changes checked against executable test suites. Release post

Further reading and tools