The First Drop of Ink: Nonlinear Impact of Misleading Information in Long-Context Reasoning

As large language models (LLMs) are increasingly deployed in retrieval augmented generation (RAG) and agentic systems that accumulate extensive context, understanding how distracting information affects performance in long context becomes critical. Prior work shows that semantically relevant but misleading documents can cause performance degradation, yet the quantitative relationship between the proportion of distractors and performance remains unstudied. In this work, we systematically vary the proportion of hard distractors within fixed-length contexts, revealing a striking nonlinear pattern: as the proportion of hard distractors increases, performance drops sharply within the first small fraction, while the remainder of the range yields only marginal additional decline. We term this ''The First Drop of Ink'' effect, analogous to how a single drop of ink contaminates water. We provide both theoretical and empirical analysis grounded in attention mechanics: hard distractors disproportionately capture attention even at small proportions, with diminishing marginal impact as their proportion increases. Through controlled experiments, we further show that filtering yields performance gains primarily from context length reduction rather than distractor removal, and only achieves substantial recovery when hard distractor proportion is reduced to near zero, which highlights the importance of upstream retrieval precision.

AI assistants can read many documents before answering, but not every relevant-looking document is helpful. Some documents are about the right topic yet actually distracting or misleading. We studied how these distracting documents affect AI systems that aggregate long inputs, using controlled tests where the total text length stayed fixed while the share of misleading documents changed. We found a “first drop of ink” effect: even a small amount of misleading information can cause a large accuracy drop, while adding more later often hurts much less. This happens because distracting documents strongly compete for the model’s focus, much like a drop of ink can quickly cloud clear water. We also found that removing some negative documents after the search step gives limited recovery unless nearly all are removed. The practical lesson is that being able to read more text is not a safety net. AI systems that search the web, read large files, or combine many sources should prioritize keeping misleading evidence out of the input in the first place, rather than relying on the model to ignore it later.