Back to Datology XL 30B-A3B in the gallery

DatologyAI

Qwen3-style research model, included for its excellent technical report on data curation, optimization, and controlled training experiments.

Datology XL 30B-A3B architecture diagram
Datology XL 30B-A3B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
30.4B total, 3.26B active (10.7% active)
Context (tokens)
65,536
Vocabulary
129,280 (~129k)
License
N/A
Date
2026-11-16
Decoder type
Sparse MoE
Attention
GQA with QK-Norm and RoPE
Layer mix
48 GQA
KV cache / token (bf16) info
96 KiB · Moderate
Key detail
Uses Qwen3-30B-A3B's 48-layer backbone with 8 of 128 experts active per token, the DeepSeek V4 tokenizer, and AdamH optimization with explicit weight-norm constraints.

Training and benchmarks

Pretraining data
1T tokens XL curated model, pretrained from scratch on Datology-curated data. A further 10B tokens extend context from 8k to 64k. All token counts use the DeepSeek V4 tokenizer. Datology technical report
Post-training methods
SFT, RLVR, Model merging About 20B SFT tokens, followed by four GRPO specialists for mathematics, code, instruction following, and tool use. The specialists are merged into one model. Datology technical report

Further reading and tools