Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM Agents

Large language models (LLMs) are increasingly deployed as autonomous agents for multi-turn decision-making tasks. However, current agents typically rely on fixed cognitive patterns: non-thinking models generate immediate responses, while thinking models engage in deep reasoning uniformly. This rigidity is inefficient for long-horizon tasks, where cognitive demands vary significantly from step to step, with some requiring strategic planning and others only routine execution. In this paper, we introduce CogRouter, a framework that trains agents to dynamically adapt cognitive depth at each step. Grounded in ACT-R theory, we design four hierarchical cognitive levels ranging from instinctive responses to strategic planning. Our two-stage training approach includes Cognition-aware Supervised Fine-tuning (CogSFT) to instill stable level-specific patterns, and Cognition-aware Policy Optimization (CoPO) for step-level credit assignment via confidence-aware advantage reweighting. The key insight is that appropriate cognitive depth should maximize the confidence of the resulting action. Experiments on ALFWorld and ScienceWorld demonstrate that CogRouter achieves state-of-the-art performance with superior efficiency.

Large language model agents can now carry out multi-step tasks, such as searching for objects, navigating virtual environments, or solving interactive science problems. However, most agents use the same amount of “thinking” at every step. This is inefficient: simple steps may only need a quick response, while difficult moments may require reflection or careful planning. We propose CogRouter, a method that teaches an AI agent to choose how deeply it should think at each step. Instead of always responding quickly or always reasoning in detail, the agent can switch among four levels of thinking, from fast instinctive actions to deeper strategic planning. We train the agent so that it learns which level is most useful for the current situation. In interactive task environments, CogRouter achieves higher success rates while using fewer generated words than methods that rely on deep reasoning throughout the whole task. This suggests a practical path toward AI agents that are not only more capable, but also more efficient: they learn when to think fast and when to think slow.