Back to Ember-1 in the gallery

Fireworks AI

Fireworks' Kimi K3 derivative, post-trained for shorter reasoning traces. Architecture details follow the published Kimi K3 base model; the diagram focuses on its text backbone.

Ember-1 architecture diagram
Ember-1 architecture diagram. Select the diagram to view it at full size.

Architecture and specifications

Scale
2.8T total, 104B active (3.7% active)
Context (tokens)
1,048,576
License
N/A
Date
2026-09-23
Decoder type
Sparse hybrid
Attention
3:1 Kimi Delta Attention and Gated MLA with NoPE
Layer mix
69 KDA + 24 Gated MLA
KV cache / token (bf16) info
27 KiB · Low
Key detail
Inherits Kimi K3's eight Block Attention Residual groups and Stable LatentMoE, with one dense layer followed by 92 MoE layers.

Training and benchmarks

Pretraining data
Not disclosed Derived from Kimi K3; the release does not disclose a pretraining-token total. Ember-1 release
Post-training methods
Not disclosed The release describes additional training to shorten reasoning but does not name its training stages. Ember-1 release

Further reading and tools