Laguna XS.2 Architecture
Back to Laguna XS.2 in the gallery
Poolside
Poolside's open-weight agentic coding MoE uses mixed sliding-window/global attention and activates only 3B of 33B parameters per token.
Architecture and specifications
- Scale
- 33B total, 3B active (9.1% active)
- Context (tokens)
- 262,144
- License
- Apache License 2.0
- Date
- 2026-04-28
- Decoder type
- Sparse MoE
- Attention
- Gated GQA with QK-Norm and 3:1 sliding-window/global attention
- Layer mix
- 30 sliding-window + 10 global
- KV cache / token (bf16) info
- 160 KiB · Moderate
- Key detail
- Uses per-layer query-head counts, a 512-token SWA window, sigmoid MoE routing, and one shared expert alongside the top-8 routed experts.
Training and benchmarks
- Pretraining data
- >30T tokens Pretraining tokens seen, sampled from a pool of about 27T unique tokens. The corpus size and training budget are different quantities. Technical report
- Post-training methods
- SFT, RLVR Supervised instruction tuning is followed by reinforcement learning with verifiable rewards using CISPO. Technical report