Nous Research Arete 27B Architecture
Back to Nous Research Arete 27B in the gallery
Nous Research
Nous Research's supervised fine-tune of Qwen3.8-27B. The vision encoder was retained and frozen during text-only fine-tuning; the figure shows the language backbone.
Architecture and specifications
- Scale
- 27B parameters
- Context (tokens)
- 262,144
- Vocabulary
- 248,320 (~248k)
- License
- Apache License 2.0
- Date
- 2026-11-06
- Decoder type
- Dense hybrid
- Attention
- 3:1 Gated DeltaNet and gated attention with GQA
- Layer mix
- 16 gated attention + 48 DeltaNet
- KV cache / token (bf16) info
- 64 KiB · Low
- Key detail
- Retains Qwen3.8-27B's 64-layer hybrid decoder with partial RoPE. The released checkpoint has no MTP head.
Training and benchmarks
- Pretraining data
- Not disclosed Derived from Qwen3.8-27B. The Arete card does not disclose the inherited pretraining-token total. Arete 27B model card
- Post-training methods
- SFT Supervised fine-tuning with Inkling-generated supervision. The card explicitly excludes RL and preference optimization. Arete 27B model card