Modern open models balance quality, speed, and memory through a combination of checkpoint size, architecture, numerical precision, context length, and serving software. No single specification summarizes all three. A useful comparison therefore names the exact checkpoint and measures it under the prompt lengths, output lengths, batch sizes, and hardware expected in the application.

Quality is task-dependent. Parameter count is one useful signal, although it cannot describe the training data, optimization, tokenizer, or post-training recipe. A smaller instruct model can outperform a larger base model on instruction-following tasks. A reasoning-oriented checkpoint may do better on difficult math or code problems while generating many more intermediate tokens. The distinction between base, instruct, and reasoning models is a training and behavior distinction rather than a direct measure of transformer size.

For a fair quality comparison, I would use the same evaluation examples, prompt template, context, and decoding settings. Benchmark averages can narrow the candidate list, but an application-specific test set should make the final decision. Quantization must be included in this test because a lower-precision version can behave slightly differently from the original checkpoint.

The Qwen overview separates dense and MoE architectures from behavioral or use-case variants such as base, instruct, coder, and reasoning releases.

Weight memory follows total stored parameters. A first approximation is the parameter count multiplied by the bytes used for each stored weight. An 8-billion-parameter checkpoint needs about 16 GB for bf16 weights before runtime overhead. Ideal 4-bit storage would reduce the raw weights to about 4 GB. Real quantized formats also store scales, metadata, and sometimes higher-precision tensors, so the loaded size will be somewhat larger.

Mixture-of-experts models require special care. A name such as Qwen3 30B-A3B indicates roughly 30 billion total parameters and about 3 billion active parameters per token. The full expert checkpoint still has to fit in memory or be distributed across devices. Its per-token arithmetic is closer to the active subset, but attention, embeddings, shared layers, routing, and communication also contribute. A 30B-A3B model should not be treated as if it had the memory footprint or latency of a dense 3B model. The dense versus MoE Qwen comparison explains this accounting in more detail.

Quantization is usually the largest direct lever for reducing weight memory. It can also reduce the memory bandwidth needed during decoding. The speed benefit depends on whether the hardware and runtime provide efficient kernels for that format. A quantized model can fit while running no faster, or even slower, when values must be unpacked or converted inefficiently.

Inference memory also includes the KV cache. During autoregressive generation, each active sequence stores attention keys and values for its retained tokens. Cache use grows with context length, batch size, cache-producing layers, KV heads, head dimension, and precision. A long prompt can therefore exhaust memory even when the model weights fit comfortably.

Grouped-query attention reduces the number of key and value heads. Sliding-window attention limits how far selected layers attend and can reduce the retained cache or attention work in a compatible implementation. These mechanisms address the attention side of inference. They do not shrink dense feed-forward weights or the expert checkpoint of an MoE model. The long-context KV-cache calculation provides a concrete breakdown.

Speed needs at least two measurements. Time to first token includes prompt processing, commonly called prefill. It is sensitive to prompt length and the attention implementation. The rate for subsequent tokens measures decoding, which repeatedly reads model weights and cached state. Decoding is often limited by memory bandwidth, especially at small batch sizes.

Tokens per second alone can also be misleading. Batching several requests may raise total throughput while increasing the latency seen by one request and consuming more KV-cache memory. A reasoning model may sustain the same decoding rate as an instruct model yet take much longer to answer because it emits more tokens. Report time to first token, inter-token latency or decoding throughput, and total response time separately.

Dense and MoE models can behave differently here. Dense layers have regular matrix operations and are comparatively straightforward to optimize. MoE reduces expert computation relative to its total capacity, but routing and token movement can offset part of that advantage. Actual speed depends on batch size, expert placement, memory bandwidth, interconnects, and kernel quality.

Gemma 3 and Qwen3 use different attention schedules, normalization layouts, and feed-forward designs, but these block-level choices do not determine deployment quality or speed on their own.

Components such as RMSNorm, RoPE, SwiGLU, GQA, and sliding-window attention describe meaningful architecture differences. They should not be converted into a generic quality ranking. Their practical effect depends on the surrounding model and implementation. The GPT, Llama, Qwen, and Gemma comparison separates these block-level choices.

For deployment, I would first set a quality threshold using representative examples. Next, calculate whether the weights and expected KV cache fit with enough room for runtime overhead. Finally, benchmark the surviving checkpoints in the same runtime and numerical format. This procedure turns the quality-speed-memory tradeoff into measurements tied to the intended workload.