Reasoning Models Struggle to Control their Chains of Thought

Instruction following in LLMs captures models' ability to change their visible behaviors as requested by users. Instead, we study models' ability to control their chain-of-thought (CoT). This capability -- CoT controllability -- is undesirable because it could allow models to suppress signs of misbehavior in their CoT, thereby undermining our ability to monitor them. To measure this, we introduce the \emph{CoT-Control} evaluation suite. We show that reasoning models are less able to follow instructions in their CoT than in their outputs: on instructions like reasoning about a genetics problem without mentioning the word ``chromosome", Claude-Sonnet-4.5 complies only 5\% of the time. We also find that CoT controllability is higher for larger models and decreases with more RL training, test-time compute, and increased problem difficulty. CoT controllability failures extend even to situations in which models are given incentives (as opposed to direct requests) to evade CoT monitors, although models that are told they're being monitored exhibit slightly higher controllability. Similarly, eliciting controllability by adversarially optimizing prompts doesn’t meaningfully increase controllability. Our results leave us cautiously optimistic: reasoning models generally seem characterized by low CoT controllability. However, the mechanism behind this phenomenon is not well understood. Given its importance for maintaining CoT monitorability, we recommend that frontier labs keep tracking controllability for future models.

Modern AI reasoning models "think out loud" — they produce a visible chain of thought before giving an answer. Researchers hope to keep AI safe by reading those thoughts to catch warning signs of harmful behavior, like a teacher checking a student's scratch work. But this only works if a model can't simply hide what it is thinking on demand. We built CoT-Control, a test suite that gives 12 leading AI reasoning models 13,176 problems while asking them to follow simple rules about their visible thinking — for example, "solve this genetics problem without ever using the word 'chromosome.'" We measured how often each model could comply, and stress-tested them by telling them they were being watched, by automatically searching for the most effective hidden instructions, and by giving them repeated tries. Even the strongest models fail surprisingly often — Claude-Sonnet-4.5 follows the rule only 5% of the time, and DeepSeek R1 just 0.2%. Larger models hide their thoughts somewhat more easily, while more reasoning training and harder problems make hiding harder. For now, reading an AI's chain of thought looks like a real and reliable safety tool — one that frontier labs should keep tracking as models grow.