IndexShare
IndexShare reduces the cost of DeepSeek Sparse Attention (DSA) by reusing its selected token positions across nearby transformer layers. In GLM-5.2, one layer computes fresh top-k indices and the next three layers reuse them.
Only the indices are shared. Each layer still has its own attention and feed-forward weights and computes its own attention output over the selected tokens.
What is shared
The top-k token indices selected by the DSA indexer
GLM-5.2 pattern
One full indexer layer followed by three index-sharing layers
Example architectures
Why The Indexer Becomes Expensive
DSA separates sparse attention into two stages. First, a lightweight indexer scores the preceding tokens and selects the top-k positions for the current query. Then, the main attention mechanism operates only on this smaller subset.
This reduces the main attention computation from quadratic O(L²) scaling to O(Lk), where L is the sequence length and k is the number of selected tokens. However, the indexer still compares each query against the full preceding context. Its cost remains O(L²), and standard DSA repeats this work in every layer.
At shorter context lengths, the indexer is relatively cheap. At hundreds of thousands or one million tokens, repeating it throughout the transformer stack becomes a larger part of the inference cost.
How IndexShare Works
IndexShare assigns DSA layers one of two roles:
- A full layer runs the indexer, performs the top-k operation, and temporarily stores the selected token positions.
- A shared layer skips the indexer and uses the most recent full layer’s selected positions.
GLM-5.2 groups four layers around one full indexer result. The first layer in the group computes the positions, and the next three layers reuse them. This skips the indexer dot products and top-k operation in three out of four layers.
The name can be slightly misleading if “share” is read too broadly. IndexShare does not share transformer weights, attention outputs, or KV tensors across the four layers. It shares the integer positions that tell each layer which earlier tokens to revisit.
full or shared.
Why Reusing Indices Can Work
The IndexCache study found that the top-k token sets selected by adjacent DSA layers overlap substantially. Its reported measurements show 70% to 100% overlap between adjacent layers. This means consecutive layers often spend indexer compute rediscovering many of the same token positions.
IndexCache describes a general version of this idea. It separates layers into full layers that compute new indices and shared layers that reuse the nearest preceding result. The paper explores both a training-free search for deciding which indexers to retain and a training-aware method that trains each retained indexer to serve several layers.
GLM-5.2 uses a regular four-layer sharing pattern and introduces it during continued mid-training at a 128K sequence length. This lets the retained indexers adapt to serving the following layers instead of applying the sharing pattern only after training.
Savings And Tradeoffs
Z.ai reports that IndexShare reduces GLM-5.2’s per-token FLOPs by 2.9× at a one-million-token context length. The separate IndexCache paper reports that removing 75% of indexer computations on a 30B DSA model produced up to a 1.82× prefill speedup and a 1.48× decode speedup at a 200K context length.
These numbers measure different systems and context lengths, so they should not be compared directly. The practical gain depends on how much of the runtime is spent in the indexer, which grows as the context becomes longer.
The tradeoff is that a shared layer cannot choose a fresh token set from its own hidden states. It must use the positions selected by the preceding full layer. Training the retained indexers for multi-layer use helps make those shared selections useful across the group.
Sources