LLM Architecture Gallery Changelog
November 17, 2026
Added Datology XL 30B-A3B
Added Datology XL 30B-A3B.
November 14, 2026
Separate DeepSeek V3 and R1 figures
Added separate architecture figures for DeepSeek V3 and DeepSeek R1.
November 13, 2026
Added Inkling 2 Small
Added Inkling 2 Small.
November 11, 2026
Added Nanbeige 4.3 33B and training details
Added Nanbeige 4.3 33B, plus sourced pretraining data sizes and post-training methods in expanded card details.
November 10, 2026
Added LongCat-2.5 and Nous Research Arete 27B
Added LongCat-2.5 and Arete-27B.
November 9, 2026
Clarified NoPE and query-key normalization in 14 diagrams
Updated architecture figures, thumbnails, and card details to show missing NoPE and query-key normalization annotations.
-
Added NoPE labels to Llama 4 Maverick, Kimi Linear, six Nemotron variants, and Soofi S.
-
Updated the corresponding card descriptions and NoPE or QK-Norm tutorial links.
November 5, 2026
Added ERNIE 5.5 Flash Base and RoPE reference links
Added ERNIE 5.5 Flash Base and RoPE reference links to architectures that use rotary positional embeddings.
November 4, 2026
Added Kimi K3.1 Pro
Added Kimi K3.1 Pro with its hybrid Kimi Delta Attention and Gated DeepSeek Sparse Attention architecture.
November 3, 2026
Updated the Recursive Flint diagram
The figure now shows Flint’s 5:2 ratio of sliding-window to global attention layers and labels its learned attention sinks.
November 2, 2026
Seven new architectures and gallery updates
Added seven architectures and updated model details, country browsing, and AAI views.
-
New architectures: Reflection Beam 501B, Dory1.0 4B, Recursive Flint 40B-A2.5B, Ling 3.1 Flash, LFM3 1.7B, Mistral Large 4, and MiniMax M3.1-Flash 428B.
-
Fixes: Corrected Ling 3.0 Flash’s KV normalization and clarified its SiLU activations and dense FFN labels. Added Mistral Large 4’s technical report link and updated both DeepSeek V4 report links.
-
New features: Country flags beside model companies and sorting by country.
-
Changes: Updated the Artificial Analysis Intelligence Index (AAI) to v5.0 and preserved the archived v4.3.2 scores and profiles. Added six AA model-page matches, separate version cards, explainers, and sorting.
October 17, 2026
Added Nemotron 3.5 Super VL and refreshed Artificial Analysis scores
Added Nemotron 3.5 Super VL, updated the Lightning figure, and refreshed Artificial Analysis totals and capability profiles across the gallery.
October 16, 2026
Added Muse Spark Open
Added Meta’s Muse Spark Open diagram and fact sheet with its 816B latent MoE architecture and gated local/global attention.
October 3, 2026
Added Kolibri-1
Added Aleph Alpha’s Kolibri-1 diagram and fact sheet with its 78B MoE architecture and 4:1 sliding-window/global attention.
September 30, 2026
Added Step-5 Preview and Naive-N0.5-Flash
Added StepFun’s Step-5 Preview and NaiveAI’s Naive-N0.5-Flash diagrams and fact sheets with their hybrid sparse-attention MoE architectures.
September 27, 2026
Added Ember-1
Added Fireworks’ Ember-1 with its inherited Kimi K3 architecture, including Kimi Delta Attention, gated MLA, Block Attention Residuals, and Stable LatentMoE.
September 21, 2026
Added Xiaomi MiMo-V2.6 Pro and Flash
Added the MiMo-V2.6-Pro-RL and MiMo-V2.6-Flash-RL text-backbone figures and fact sheets, with links to their configurations and technical report.
-
Added MiMo-V2.6-Pro-RL with 70 layers, 384 routed experts, and 42B active parameters.
-
Added MiMo-V2.6-Flash-RL with 48 layers, 256 routed experts, and 15B active parameters.
September 10, 2026
Added DeepSeek V4.1-Flash
Added the DeepSeek V4.1-Flash architecture with separate causal encoder and decoder blocks, CSA2 cache sharing, Engram memory, and Single-Pass mHC.
September 5, 2026
Added an interactive model memory calculator
Estimate weight and KV-cache memory separately using the gallery’s model data, with controls for context length, batch size, and independent weight and cache precision.
Fixed comparison links and keyboard navigation
Model B selections keep their slot after reloading, enlarged figures keep keyboard focus inside the dialog, and long equations scroll within the page on mobile.
-
Preserved empty Model A and Model B slots in comparison links.
-
Improved keyboard navigation and focus restoration in enlarged figure previews.
-
Corrected heading levels and contained long equations on narrow screens.
September 3, 2026
Added Ouro-Thinking 2.6B
Added the ByteDance Ouro-Thinking 2.6B architecture card, expanded the Looped Transformer explainer, and refreshed the gallery’s KV-cache and attention summaries.
September 2, 2026
Improved gallery theme controls and dark-mode figures
Added a gallery theme selector and improved dark-mode rendering across architecture cards, model comparisons, and enlarged views.
August 29, 2026
Added Qwen3.8-Flash-Next and Tencent Hy4-preview architectures
Added Qwen3.8-Flash-Next and Tencent Hy4-preview architecture cards, Qwen Sparse Attention and Gated Residuals explainers, and refreshed the gallery summary tables.
August 27, 2026
Added company metadata, sorting, and gallery filters
Added company information to every architecture card, introduced filters for release date, total parameter count, architecture family, and company, and made the gallery table of contents alphabetical.
-
Added company sorting and company-name search to the gallery controls.
-
Kept the gallery table of contents alphabetical independently of the selected card sort.
August 26, 2026
Added GLM-5.3-Flash and updated the poster download
Added the Z.ai GLM-5.3-Flash architecture card, refreshed the gallery summary tables, and updated the poster download information.
-
Updated the support section to offer the print-ready digital download without a physical poster listing.
August 24, 2026
Added Ornith 1.5
Added the Ornith 1.5 35B-A3B architecture card and refreshed the gallery summary tables.
August 23, 2026
Added Ling 3.0 Flash
Added the InclusionAI Ling 3.0 Flash architecture card and refreshed the gallery summary tables.
August 15, 2026
Added Qwen3.8 27B
Added the Qwen3.8 27B architecture card and refreshed the attention mechanism distribution table.
August 11, 2026
Added Nemotron 3.5 Lightning
Added the NVIDIA Nemotron 3.5 Lightning 30B-A3B architecture card and refreshed the gallery summary tables.
August 10, 2026
Added Muse Glimmer 30B
Added the Meta Muse Glimmer 30B architecture card and refreshed the attention mechanism distribution table.
July 27, 2026
Added Kimi K3
Added the Moonshot AI Kimi K3 architecture card and refreshed the gallery summary tables.
July 25, 2026
Mega update: Added Antares 1B, BTL-3, Laguna 2.1 models, Motif 3 Beta, Nanbeige 4.2, Solar Open 2, and new explainers
Added architecture cards for Antares 1B, BTL-3, Laguna S 2.1, Laguna XS 2.1, Motif 3 Beta, Nanbeige 4.2, and Solar Open 2, plus explainers for ShortConv, Looped Transformers, IndexShare, and PolyNorm.
-
Added the Antares-1B, BTL-3 27B, Laguna S 2.1, Laguna XS 2.1, Motif 3 Beta, Nanbeige 4.2 3B, and Solar Open 2 architecture cards.
July 16, 2026
Added Soofi S
Added the Soofi S 30B-A3B architecture card.
July 15, 2026
Added Inkling
Added the Thinking Machines Lab Inkling 975B architecture card.
June 18, 2026
Added GLM-5.2 and refreshed AA data
Added the Z.ai GLM-5.2 architecture card and refreshed Artificial Analysis Intelligence Index data.
June 16, 2026
Added VibeThinker-3B
Added the WeiboAI VibeThinker-3B architecture card to the gallery.
June 13, 2026
Added Kimi K2.7 Code and MiniMax M3
Added the Moonshot AI Kimi K2.7 Code and MiniMax M3 428B architecture cards.
June 9, 2026
Added North Mini Code
Added the Cohere North Mini Code 30B-A3B architecture card and refreshed the gallery summary tables.
June 4, 2026
Added Nemotron 3 Ultra
Added the NVIDIA Nemotron 3 Ultra 550B-A55B architecture card and refreshed the gallery summary tables.
June 3, 2026
Added Gemma 4, Mellum2, and LFM2.5 models
Added the Gemma 4 12B, JetBrains Mellum2 Thinking 12B-A2.5B, and Liquid AI LFM2.5 architecture cards and refreshed the gallery summary tables.
May 27, 2026
Linked the MiniMax M2 series and Laguna reports
Added technical report links to the MiniMax M2, M2.5, M2.7, and Laguna XS.2 gallery cards.
-
Added the MiniMax-M2 Series technical report link to the MiniMax M2, M2.5, and M2.7 cards.
-
Added the Laguna XS.2 technical report link to the Laguna XS.2 card.
May 21, 2026
Corrected the OLMo 2 architecture diagram
Updated the OLMo 2 7B gallery figure to show the post-attention and post-feed-forward RMSNorm placement more accurately.
May 20, 2026
Added Command A+
Added the Cohere Command A+ 218B-A25B architecture card and refreshed the active-parameter ratio and attention mechanism distribution tables.
May 17, 2026
Added Gemma 4, Laguna, ZAYA1, and DeepSeek V4 explainers
Added cross-layer KV-sharing and per-layer embedding explainers for Gemma 4 E2B and E4B, added Laguna attention-budgeting, ZAYA1 compressed convolutional attention, and DeepSeek V4 mHC and CSA/HCA explainers, and linked recent architecture cards to the May 16 article sections.
-
Added explainers for cross-layer KV sharing, per-layer embeddings, Laguna XS.2 attention budgeting, ZAYA1-8B compressed convolutional attention, DeepSeek V4 manifold-constrained hyper-connections, and DeepSeek V4 CSA/HCA compressed attention.
-
Added View in article links for Gemma 4 E2B, Gemma 4 E4B, Laguna XS.2, ZAYA1-8B, and DeepSeek V4.
May 14, 2026
Added Xiaomi MiMo-V2.5-Pro
Added the Xiaomi MiMo-V2.5-Pro 1.02T architecture card (and updated the active-parameter ratio and attention mechanism distribution tables).
May 14, 2026
Added active-parameter ratio meta analysis
Added a meta-analysis block below the architecture cards and linked standalone tables for active-parameter ratios and attention mechanism distribution.
-
Added the standalone active-parameter ratio table for sparse MoE and hybrid models.
-
Added the standalone attention mechanism distribution table for visible gallery cards.
May 12, 2026
Updated gallery metadata and performance
Restored missing architecture cards, corrected gallery metadata, and reduced card rendering cost with generated thumbnails, offscreen rendering hints, and cheaper sticky controls.
-
Removed the incorrect Gemma 4 Artificial Analysis score and source link from the GPT-2 XL card.
-
Added the missing Llama 3.2 3B, Qwen3 0.6B, and Qwen3 30B-A3B figure cards.
-
Added smaller generated thumbnails for architecture cards while keeping full-size images for zoom.
-
Removed backdrop blur from the sticky search and sort controls to reduce scroll work.
May 10, 2026
Added May 10 architecture gallery updates
Added ZAYA1-8B and LongCat-Flash-Lite 68.5B-A3B to the public architecture gallery.
-
Clarified that the gallery focuses on text-only LLMs and language-model backbones.
May 3, 2026
Added May 3 architecture gallery updates
Added Laguna XS.2, Tencent Hy3-preview 295B-A21B, Granite 4.1 30B, and a Multi-Token Prediction explainer to the public architecture gallery.
May 1, 2026
Added May architecture gallery updates
Added Xiaomi MiMo-V2.5 310B, MiniMax M2.7 230B, and Ling 2.6 1T to the public architecture gallery; added AA Index score sorting and refreshed AA profile data for newly covered models.
-
Added an AA Index score sort option that ranks models with numeric Artificial Analysis scores first.
April 26, 2026
Added new April architecture cards
Added Kimi K2.6, Qwen3.6 35B-A3B, Qwen3.6 27B, DeepSeek V4-Pro, and DeepSeek V4-Flash to the public architecture gallery. Details on mHC and compressed attention will be added at a later time.
April 10, 2026
Implemented Gemma 4 E2B and E4B from scratch
Implemented Gemma 4 E2B and E4B from scratch (link in the architecture cards).
April 9, 2026
Expanded Gemma 4 gallery coverage
Added Gemma 4 E2B and E4B architecture cards and filled missing AA Intelligence Index data for the larger Gemma 4 models and GLM-5.1.
April 7, 2026
Added the GLM-5.1 architecture card
Added GLM-5.1 to the public architecture gallery.
April 4, 2026
Added GLM-4.5-Air and INTELLECT-3
Added standalone technical-report cards for GLM-4.5-Air and INTELLECT-3.
April 2, 2026
Added Gemma 4 architecture cards
Added two new Gemma 4 architecture cards to the gallery.
March 29, 2026
Added a digital poster purchase link
Added a Gumroad option alongside the existing Redbubble poster listing.
- Added a Gumroad link for the print-ready digital poster alongside the existing Redbubble physical poster option.
March 27, 2026
Expanded gallery controls and benchmark metadata
Added Artificial Analysis Intelligence Index data and more flexible ways to browse dense cards.
-
Added Artificial Analysis Intelligence Index scores for each model where applicable.
-
Added a Detailed / Compact view switch for the main card grid.
-
Added per-card Show details / Show less toggles in compact view.
March 26, 2026
Added architecture comparison and sorting tools
Turned the gallery into a more interactive comparison tool and expanded several card fact fields.
-
Added a side-by-side architecture diff with Model A / Model B selectors and per-card compare actions.
-
Added a Sort by control for Release date (newest first), Release date (oldest first), A-Z, and Size.
-
Added active-parameter percentages to MoE Scale fields where available.
-
Added KV cache / token (bf16) estimates to the fact sheets.
-
Added Layer mix to the fact sheets.
March 25, 2026
Added the Phi-4 architecture card
Added Phi-4 to the public gallery.
March 20, 2026
Added four new architecture cards
Added Nemotron 3 Nano 4B, Kimi K2.5, Mistral Small 4, and xLSTM 7B.
March 17, 2026
Added license metadata
Added license info and links to license files where applicable.
- Added license info and links to license files where applicable.