LLM Memory Calculator
Estimate the memory occupied by model weights and the KV cache during inference. Change the context length or batch size to see how the cache grows while the weights stay fixed.
These are logical payload estimates. A running model also needs memory for activations, temporary buffers, and other runtime state.
Enable JavaScript to use the calculator. The formulas and assumptions below are available without JavaScript.
Weights + KV cache
Waiting for settings
Estimated payload before runtime overhead.
Weight memory
Unavailable
One resident copy of the model.KV-cache memory
Unavailable
Across all sequences.Settings are saved in the page URL when you finish changing a control. Copy the address to share the estimate.
How the estimate works
The calculator uses the gallery's rounded parameter counts and BF16 cache figures. M, B, and T mean million, billion, and trillion parameters. Memory is displayed in GiB (230 bytes); GB uses 109 bytes.
Weight bytes = parameter count × weight bits ÷ 8
KV-cache bytes = BF16 cache bytes per token × context tokens × batch size × cache bits ÷ 16
- Resident weights. All counted parameters remain in memory, including every MoE expert. Active parameters per token describe computation and do not replace the total weight count. Weights are shared across the batch.
- Retained tokens. Each sequence keeps its own cache, with no shared prefixes or beam search. Sliding-window and chunked models use a full-retention projection here because the gallery does not provide each layer's retention limit. A runtime that evicts old entries can use less cache.
- Architecture scope. Cache figures include persistent keys used by sparse-attention indexers. Shared keys count once. Fixed recurrent state, temporary index buffers, and optional multi-token prediction layers fall outside the estimate.
- Precision. The selected bit width applies uniformly to the counted weights or cache. Quantization scales, zero points, padding, and tensors retained at higher precision add memory. Selecting a format here does not imply that a particular model or runtime supports it.
- Runtime memory. Activations, prefill workspace, allocator reserves, and other serving buffers are excluded. CPU offloading and distribution across GPUs are not modeled. The sum alone cannot determine whether a model fits on a GPU.
See the gallery's per-layer formulas for GQA, MLA, shared caches, and recurrent hybrids. The Transformers documentation explains how caching works and how cache strategies affect memory.
How much KV cache does Qwen3 8B need at 32K tokens?
For one sequence of 32,768 tokens (batch size 1), Qwen3 8B uses 4.5 GiB of BF16 KV cache. Each of its 36 layers stores a key and a value for each of eight KV heads. Each head has 128 elements, and each BF16 element takes two bytes. Qwen3 8B model configuration.
That gives 36 × 8 × 128 × 2 × 2 = 147,456 bytes (144 KiB) per token. Multiplying by 32,768 tokens gives 4.5 GiB. The weights take another 14.90 GiB using the gallery's rounded count of 8 billion parameters at 16 bits each. Activations and runtime buffers require additional memory.