具有有限工作记忆的语言模型必须反复决定保留哪些存储项。每个部署的方法都会根据过去的(StreamingLLM, H2O) 或对未来(SnapKV) 的猜测来决定项目到达, 的时刻。我们将选择重新定义为隐藏信号的估计问题, 是否会重用某个项目, 将现有方法放在一个轴上, 提交滞后$H$: 在线过滤器和学习预测器在$H=0$, 提交，而Belady的 离线最佳位置位于整个未来已知的位置。 , 固定滞后平滑, 之间的缺失机制等待有限数量的步骤, 观察正确的近期预测关注, 的哪些项目，然后才提交。这个测量,证明了实用性,将Belady'的不可观察的未来请求转变为我们从模型本身读取的东西。我们将其实例化为免训练策略, RMM, H2O 的严格泛化，当测量均匀时精确地简化为 H2O。在受控环境中，重用是内生的并且在时间上是分开的,，证明实用程序比累积注意力更好地识别已使用的内存,，并且小的有界内存的行为与大得多的内存相似。但在独立第三方基准测试中, 在 NVIDIA的 KVPress 线束中运行，针对其自己的 SnapKV, H2O, 和 StreamingLLM 实现,，优势几乎消失: RMM 在单轮问答方面与 H2O 相当，但在流式多轮设置中输给了 H2O 和 SnapKV。原因很简单: 在自然文本上，模型对大多数标记都是正确的,，因此通过正确性来加权注意力几乎不会改变它,，并且证明效用会崩溃到积累的注意力上，除非重用是尖锐的和内生的,，而标准基准不执行这一点。我们的贡献是测量节拍时累积, 的框架和诚实的地图，而不是新的技术水平。

A language model with a bounded working memory must repeatedly decide which stored items to keep. Every deployed method decides the moment an item arrives, from the past (StreamingLLM, H2O) or from a guess about the future (SnapKV). We recast the choice as an estimation problem on a hidden signal, whether an item will be reused, placing existing methods on one axis, the commit lag $H$: online filters and learned predictors commit at $H=0$, while Belady的 offline optimum sits where the whole future is known. The missing regime in between, fixed-lag smoothing, waits a bounded number of steps, observes which items a correct near-future prediction attended to, and only then commits. This measurement, demonstrated utility, turns Belady的 unobservable future request into something we read off the model itself. We instantiate it as a training-free policy, RMM, a strict generalization of H2O that reduces to it exactly when the measurement is uniform. In controlled settings where reuse is endogenous and separated in time, demonstrated utility identifies used memory far better than accumulated attention, and a small bounded memory behaves like a much larger one. But on independent third-party benchmarks, run inside NVIDIA的 KVPress harness against its own SnapKV, H2O, and StreamingLLM implementations, the advantage mostly disappears: RMM is on par with H2O for single-turn question answering and loses to both H2O and SnapKV in a streaming multi-turn setting. The cause is simple: on natural text the model is correct about most tokens, so weighting attention by correctness barely changes it, and demonstrated utility collapses onto accumulated attention unless reuse is sharp and endogenous, which standard benchmarks do not exercise. Our contribution is the framework and an honest map of when measuring beats accumulating, not a new state of the art.

科目:人工智能(cs.AI)

Subjects: Artificial Intelligence (cs.AI)