No AI summary available for this article.
Why It Matters
Decoder-only Transformer language models cache keys and values (KV) to reuse past computation during generation.
Provenance
Discovered via ArXiv and published by ArXiv.
Key Claims
Original description
Decoder-only Transformer language models cache keys and values (KV) to reuse past computation during generation. Sharing KV across layers saves storage but reduces the diversity of representations available across depth. We study what local memory should retain alongside shared global KV, separating historical content from the input source used to form it. At 126M parameters and 2K context, an eight-seed study finds about 1.4% lower held-out test perplexity with local history than with a current-token local branch. Capacity, entry-count and training-compute controls support the value of histor...
Discovered via ArXiv
Research papers and preprints from arXiv.
Publisher: arxiv.org
ID: http://arxiv.org/abs/2609.28006v1 · Indexed about 1 hour ago