When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems

LLM-based multi-agent systems can fail even when planned actions are executed correctly because agents may misjudge their knowledge when evaluating plan feasibility, a phenomenon we term epistemic miscalibration in planning. Unlike execution errors, epistemic miscalibration is latent during planning, as generated plans can remain self-consistent and executable without observable errors; the miscalibration is also dynamic, as new information can alter feasibility assessments, potentially obscuring past miscalibration signals and causing them to recur over time. To address this, we propose the Epistemic Planning Calibration Agentic Workflow (EPC-AW), which assesses whether plans remain supported under varying information conditions rather than directly verifying feasibility. EPC-AW employs Information-consistency-based Plan Selection, selecting plans whose evaluations are stable across agents, together with Consistency-guided Epistemic State Refinement to adapt calibration over time by leveraging past discrepancies to guide future planning. Experiments show that EPC-AW improves system-level success by an average of 9.75\%. Code is available in the public repository (https://github.com/wzhSteve/EPC-AW).

LLM-based multi-agent systems can still fail even when agents follow their plans correctly. One major reason is that these systems often misjudge what they truly know or do not know when deciding whether a plan will work. This problem is difficult to detect because the generated plans may appear logical and executable, even when they are based on incomplete or unreliable understanding. The issue also changes over time, since newly discovered information can make earlier mistakes hard to recognize and cause similar failures to happen again. To address this, we developed a new multi-agent workflow called EPC-AW. Instead of only checking whether a plan seems feasible, our system tests whether the plan remains reliable under different information conditions. It also helps agents learn from past disagreements and adjust their future plans over time. Experiments on several challenging tasks show that our method improves overall task success rates by nearly 10%. This work could help make future AI assistants and collaborative agent systems more reliable in complex real-world tasks such as research, information gathering, and long-term decision making.