MedCoG: Maximizing LLM Inference Density in Medical Reasoning via Meta-Cognitive Regulation

Large Language Models (LLMs) have shown strong potential in complex medical reasoning yet face diminishing gains under inference scaling laws. While existing studies augment LLMs with various knowledge types, it remains unclear how effectively the additional costs translate into accuracy. In this paper, we explore how meta-cognition of LLMs, i.e., their self-assessment of their own cognitive states, can regulate the reasoning process. Specifically, we propose MedCoG, a Medical Meta-Cognition Agent with Knowledge Graph, where the meta-cognitive assessments of task complexity, familiarity, and knowledge density dynamically regulate utilization of procedural, episodic, and factual knowledge. The LLM-centric on-demand reasoning aims to mitigate the diminishing returns under scaling law by (1) reducing costs via avoiding indiscriminate scaling, (2) improving accuracy via filtering out distractive knowledge. To validate this, we empirically characterize the scaling curve and introduce inference density to quantify inference efficiency. Experiments demonstrate the effectiveness and efficiency of MedCoG on five hard sets of medical benchmarks, yielding 6.2x inference density. Furthermore, the Oracle study highlights the significant potential of meta-cognitive regulation.

When doctors reason through complex medical cases, they instinctively know when to rely on memory, look up a reference, or think step by step, rather than doing all three every time. Current AI medical reasoning systems lack this self-awareness, blindly applying expensive reasoning strategies regardless of whether they are needed, leading to diminishing returns as costs scale up. We propose MedCoG, a medical AI agent that equips large language models with meta-cognitive regulation: the ability to assess their own knowledge state across three dimensions, including task complexity, case familiarity, and factual knowledge demand, and dynamically select only the reasoning strategies they actually need. This on-demand approach simultaneously reduces inference cost by avoiding indiscriminate scaling and improves accuracy by avoiding distractive knowledge. Across five challenging medical benchmarks, MedCoG achieves 6.2× inference density, meaning it delivers the same level of performance at a fraction of the cost of comparable methods. Our findings suggest that a key bottleneck in medical AI reasoning lies not only in the breadth of available knowledge, but in the precision with which that knowledge is allocated, a gap that meta-cognitive regulation for LLMs can effectively bridge.