Carryover Effects in KV Caches Reused in Agentic Workflows and RL Training

What does a reused KV cache do to the model that reads it?

AuthorsHossain Pazooki, Vikram Nadathur, Lingyuan Yan, Samuel Alake, Muhammad Zain Asad, Emerson Yu, Ritvik Aggarwal, Md Farhan Ishmam
VenueNeurIPS 2026 Workshop on Long Context Foundation Models
Code and ledgerslinear-ceiling · lag-ladder

Abstract

A carryover effect occurs when a model reads a KV cache that another computation wrote: the text or weights differ from the writer's, and the reader's next-token distribution shifts as a result. Prior work shows that reusing a cache at new positions or across weight updates can change a model's outputs. We ask what predicts the size of this effect, and we measure it in two pre-registered records: agent handoffs and policy updates.

The oracle removal fraction is the smallest share of matched tokens, ranked by deviation from a fresh cache, as in CacheBlend's analysis, whose removal brings the remaining keys' mean deviation within a cross-model map's held-out error. At agent handoffs, a cache written by the reader's own model has a removal fraction of zero on the median handoff of every cohort. The reader's per-token KL divergence compares its predictions under the reused cache and a fresh one; PipelineRL and CacheReforge report related KL divergences for stale caches. On 35 long coding-agent handoffs, each with a same-model removal fraction of zero, the reader's mean per-token KL(fresh ‖ reused) has a median of 0.14 nats, with no threshold registered.

Across GRPO policy updates in three model families, with the cache written before the first update, the same KL, as a median over sequences, stays under a registered threshold of 1e-3 nats on two held-out texts at every lag read, but on GSM8K rollouts sampled by the initial policy it is about 2,000 to 150,000 times as large as on those texts at the same lag and above the threshold after one update. With one Qwen2.5-1.5B run's policies, the KL also stays under the threshold at every lag read on the opening 1,024 tokens of fifty MLPerf Agentic Inference trajectories.

The greedy answer flip rate is the share of test problems whose final answer changes under the reused cache, a wider count than the flips reported for compressed models. In that run, the stale cache changes the greedy final answer on 71% of GSM8K test prompts after one update and on 84% to 90% at 2 to 32 updates. A reused cache should therefore be checked on the reader's predictions on the kind of text it will serve, not only on its keys or on held-out text, since in both records those checks passed while the predictions moved.

Citation

For attribution in academic contexts, please cite this work as

@inproceedings{pazooki2026carryover,
  title     = {Carryover Effects in {KV} Caches Reused in Agentic Workflows and {RL} Training},
  author    = {Pazooki, Hossain and Nadathur, Vikram and Yan, Lingyuan and Alake, Samuel and Asad, Muhammad Zain and Yu, Emerson and Aggarwal, Ritvik and Ishmam, Md Farhan},
  booktitle = {NeurIPS 2026 Workshop on Long Context Foundation Models},
  year      = {2026},
  note      = {Non-archival},
  url       = {https://carryover.pub}
}

Open source record

The code and the ledger of each record are public: linear-ceiling (the text axis; ledger/ledger.md) and lag-ladder (the weights axis; ledger/ledger.md). The source is at carryoverpub/carryoverpub.github.io.

Reuse

Text on this page is licensed under Creative Commons Attribution CC-BY 4.0. The two records carry their own licences.