Back to ERNIE 5.5 Flash Base in the gallery

Baidu

Text-only base model with 37 heavily compressed attention layers and 6 MLA/DSA layers. Pretrained with up to 128K tokens of context; fine-tuning and inference support 1M tokens.

ERNIE 5.5 Flash Base architecture diagram
ERNIE 5.5 Flash Base architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
280B total, 14B active (5.0% active)
Context (tokens)
1,048,576
Vocabulary
201,216 (~201k)
License
ERNIE 5.5 Flash Base Model License Agreement v1.0
Date
2026-11-05
Decoder type
Sparse MoE
Attention
Gated HCA and MLA with DeepSeek Sparse Attention, VHA, and QK-Norm
Layer mix
37 HCA + 6 MLA/DSA; 1 dense FFN + 42 latent MoE
KV cache / token (bf16) info
8.54 KiB (growing global cache only) · Very low
Key detail
Four-stream mHC surrounds attention and latent MoE. Two sigmoid routers select 10 of 512 routed experts, alongside one always-active shared expert. ECHO reuses the final backbone layer for multi-token prediction.

Training and benchmarks

Pretraining data
33.5T tokens Text-only pretraining from scratch. ERNIE 5.5 Flash Base model card
Post-training methods
None (base model) Released at the end of pretraining, before any mid-training or post-training. ERNIE 5.5 Flash Base model card

Further reading and tools