Back to Inkling in the gallery

Thinking Machines Lab

Thinking Machines Lab's native multimodal model combines a 66-layer sparse MoE decoder with a 5:1 mix of local and global GQA.

Inkling architecture diagram
Inkling architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
975B total, 41B active (4.2% active)
Context (tokens)
1,048,576
Vocabulary
200,058 (~200k)
License
Apache License 2.0
Date
2026-07-15
Decoder type
Sparse MoE
Attention
GQA with QK-Norm and learned relative-position bias in a 5:1 sliding-window/global pattern
Layer mix
55 sliding-window + 11 global GQA
KV cache / token (bf16) info
484 KiB · Very high
Key detail
Uses learned relative-position bias instead of RoPE and adds four short convolutions to each decoder layer.

Training and benchmarks

Pretraining data
45T tokens Multimodal pretraining on text, images, audio, and video. Training overview
Post-training methods
SFT, RL Synthetic-data SFT bootstrap followed by large-scale asynchronous RL across multiple domains. Training overview
AA Intelligence Index v5.0 info
Total score 12.5 Knowledge Work 13.5 Coding 2.8 Science 7.2 General 26.5
AA Intelligence Index v4.3.2 info
Total score 25.0 General 36.4 Scientific 18.7 Coding 24.0 Agents 18.7

Further reading and tools