Are Large Reasoning Models Interruptible?
Abstract
Real-world applications of Large Reasoning Models (LRMs) often require reasoning about changing prompts or environments. In this work, we challenge the frozen world assumption and evaluate LRM robustness under two realistic dynamic scenarios: interruptions, which test the accuracy of model responses under budget-constrained outputs, and dynamic context, which tests model adaptation to in-flight changes. Across mathematics and programming benchmarks that require long-form reasoning, static evaluations consistently overestimate robustness: even state-of-the-art LRMs, which achieve high accuracy in static settings, can fail unpredictably when interrupted or exposed to changing context, with performance dropping by up to 60\% when updates are introduced late in the reasoning process. Our analysis further reveals several novel failure modes, including reasoning leakage, where models fold the reasoning into their final answer when interrupted; panic, where under time pressure models abandon reasoning entirely and return incorrect answers; and self-doubt, where performance degrades when trying to incorporate updated information. The project page is at https://dynamic-lm.github.io
Lay Summary
Large reasoning models are increasingly used for tasks that require multi-step thinking, such as solving math problems or writing code. However, real-world tasks are rarely perfectly static: users may interrupt the model, give new instructions, or change the problem while the model is still reasoning. In this work, we study how well these models handle such changing situations. We test two realistic cases: when a model must answer under a shortened reasoning budget, and when new information is introduced partway through the task. We find that standard evaluations, which usually test models only on fixed prompts, can make these systems look more reliable than they actually are. Even strong models can fail unpredictably when interrupted or when the context changes, especially if updates arrive late in the reasoning process. We also identify several recurring failure patterns: models may leak unfinished reasoning into the final answer, panic and stop reasoning too early, or become less accurate after trying to incorporate new information. These results suggest that future reasoning models should be evaluated not only on static problems, but also on their ability to stay reliable under interruptions and changing user needs.