Q-Delta: Beyond Key–Value Associative State Evolution

Linear attention reformulates sequence modeling as recurrent state evolution, enabling efficient linear-time inference. Under the key–value associative paradigm, existing approaches restrict the role of the query to the readout operation, decoupling it from state evolution. We show that query-conditioned state readout induces a structured value prediction over accumulated memory that complements key-based retrieval. Based on this insight, we propose Q-Delta, a query-aware delta rule that integrates mixed key--query prediction errors into state evolution, enabling jointly corrective dynamics while preserving delta-rule efficiency. We establish stability guarantees for the resulting dynamics and derive a hardware-efficient chunkwise-parallel formulation with a custom Triton implementation. Empirical results demonstrate stable optimization, competitive throughput, and consistent improvements over strong baselines on language modeling and long-context retrieval tasks. Code is available at \url{https://github.com/psmiz/Q-Delta}.

Modern AI language models face a fundamental tradeoff. The most capable ones (Transformers) become impractically slow on long documents because they re-examine every previous word for each new prediction. Efficient alternatives instead keep a compact "memory" of past content, writing new information using keys and values, and later looking it up using queries. A long-standing assumption in this design is that queries only retrieve information; they play no role in shaping what gets stored. We challenge this view. We show that the answer a query retrieves is itself a meaningful prediction about what comes next, a useful signal that current models simply throw away. Our method, Q-Delta, captures this overlooked signal and uses it to correct the model's memory on the fly, while preserving the fast linear-time computation these architectures are valued for. We also prove theoretically that this added feedback loop stays stable rather than amplifying errors. In experiments, Q-Delta improves both language modeling quality and the ability to retrieve specific facts from long contexts, outperforming leading efficient architectures.