Back to Laguna XS.2 in the gallery

Poolside

Poolside's open-weight agentic coding MoE uses mixed sliding-window/global attention and activates only 3B of 33B parameters per token.

Laguna XS.2 architecture diagram
Laguna XS.2 architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
33B total, 3B active (9.1% active)
Context (tokens)
262,144
License
Apache License 2.0
Date
2026-04-28
Decoder type
Sparse MoE
Attention
Gated GQA with QK-Norm and 3:1 sliding-window/global attention
Layer mix
30 sliding-window + 10 global
KV cache / token (bf16) info
160 KiB · Moderate
Key detail
Uses per-layer query-head counts, a 512-token SWA window, sigmoid MoE routing, and one shared expert alongside the top-8 routed experts.

Training and benchmarks

Pretraining data
>30T tokens Pretraining tokens seen, sampled from a pool of about 27T unique tokens. The corpus size and training budget are different quantities. Technical report
Post-training methods
SFT, RLVR Supervised instruction tuning is followed by reinforcement learning with verifiable rewards using CISPO. Technical report

Further reading and tools