Back to Gemma 4 E2B in the gallery

Google

Smallest Gemma 4 edge model keeps the family's hybrid attention stack and adds native audio on a phone-scale multimodal footprint. Uses per-layer embeddings, which add small layer-specific token vectors without scaling the full compute path, so its compute footprint is closer to 2.3B than a full 5.1B dense model.

Gemma 4 E2B architecture diagram
Gemma 4 E2B architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
5.1B parameters (2.3B effective)
Context (tokens)
131,072
Vocabulary
262,144 (~262k)
License
Apache License 2.0
Date
2026-04-02
Decoder type
Dense
Attention
Multi-query attention with QK-Norm, cross-layer KV sharing, p-RoPE on global layers, and 4:1 sliding-window/global attention
Layer mix
28 sliding-window + 7 global
KV cache / token (bf16) info
18 KiB · Very low
Key detail
The first 15 layers have a 6,144-wide GELU MLP. The final 20 reuse earlier KV tensors and double the MLP width to 12,288.

Training and benchmarks

Pretraining data
Not disclosed No per-model pretraining-token count is published. Gemma 4 model card
Post-training methods
Instruction tuning IT variant; the report refers to Gemma 3-like post-training without itemizing alignment stages. Gemma 4 report
AA Intelligence Index v4.3.2 info
Total score 6.5 General N/A Scientific 2.4 Coding N/A Agents N/A

Further reading and tools