Fix the Mind, Not the Move: Interpretable AI Assistance via Knowledge-Gap Localization

AI assistants in human-AI collaboration often correct suboptimal human actions through behavioral feedback (e.g., alerts or steering-wheel nudges in assistive driving). Such interventions can mitigate immediate errors, but long-term improvement requires addressing the underlying misconceptions that cause repeated mistakes. We introduce SENSEI, a framework that infers user misconceptions from interaction behavior and provides targeted, minimal yet sufficient suggestions to correct them. Our approach departs from action- or trajectory-level interventions by operating over a structured knowledge representation to localize and correct the sources of erroneous behavior. Across three long-horizon tasks with diverse misconceptions and corresponding behaviors, SENSEI demonstrates zero-shot compositional generalization, disentangling multiple overlapping misconceptions despite training only on single-misconception cases. A user study further shows that our method identifies real human misconceptions and provides effective guidance that improves long-horizon task performance, successfully correcting 90% of student misconceptions.

When people make mistakes, AI assistants usually offer a quick fix, like a steering wheel nudge in a car. However, these corrections do not address the deeper misunderstandings that cause people to repeat the same errors. To tackle this problem, we developed SENSEI, an AI framework that corrects these misunderstandings. SENSEI represents a person’s knowledge in a structured and interpretable way, with different components capturing their understanding of task procedures, constraints, and relevant concepts. When a person makes a mistake, SENSEI infers which flaw in which part of their knowledge caused the error, then provides targeted advice to repair that misunderstanding. We tested SENSEI on several complex tasks and found it could successfully identify multiple, overlapping misunderstandings at once. In studies with real people, our method identified their confusion and provided helpful guidance, successfully correcting 90% of their underlying misconceptions and improving their long-term performance.