Back to Tencent Hy3-preview 295B-A21B in the gallery

Tencent

Tencent Hunyuan's 295B sparse MoE preview combines a dense first FFN layer, GQA with QK-Norm, and one extra MTP layer for faster generation.

Tencent Hy3-preview 295B-A21B architecture diagram
Tencent Hy3-preview 295B-A21B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
295B total, 21B active (7.1% active)
Context (tokens)
262,144
License
Tencent Hy Community License Agreement
Date
2026-04-23
Decoder type
Sparse MoE
Attention
GQA with QK-Norm
Layer mix
80 GQA + 1 MTP layer
KV cache / token (bf16) info
320 KiB · Very high
Key detail
Uses 192 routed experts, 8 routed plus 1 shared expert active per token, sigmoid routing with expert bias, and a 3.8B MTP layer.

Training and benchmarks

Pretraining data
Not disclosed The model card and release post describe pretraining without a token count. Model card · Release post
Post-training methods
RL Tencent reports reinforcement learning for coding and agentic tasks; the full alignment recipe and reward types are not specified. Model card
AA Intelligence Index v4.3.2 info
Total score 17.0 General N/A Scientific 3.6 Coding N/A Agents N/A

Further reading and tools