Closing the Expression Gap in LLM Instructions via Socratic Questioning

A fundamental bottleneck in human-AI collaboration is the ``intention expression gap", the difficulty for humans to effectively convey complex, high-dimensional thoughts to AI. This challenge often traps users in inefficient trial-and-error loops and is exacerbated by the diverse expertise levels of users. We reframe this problem from passive instruction following to a Socratic collaboration paradigm, proposing an agent that actively probes for information to resolve its uncertainty about user intent. We name the proposed agent Nous, trained to acquire proficiency in this inquiry policy. The core mechanism of Nous is a training framework grounded in the first principles of information theory. Within this framework, we define the information gain from dialogue as an intrinsic reward signal, which is fundamentally equivalent to the reduction of Shannon entropy over a structured task space. This reward design enables us to avoid reliance on costly human preference annotations or external reward models. To validate our framework, we develop an automated simulation pipeline to generate a large-scale, preference-based dataset for the challenging task of scientific diagram generation. Comprehensive experiments, including ablations, subjective and objective evaluations, and tests across user expertise levels, demonstrate the effectiveness of our proposed framework. Nous achieves leading efficiency and output quality, while remaining robust to varying user expertise. Our research provides a systematic methodology and a new perspective for addressing ambiguous intentions in complex human-machine collaboration.

When people use AI to help create something complex like a detailed scientific diagram, they often struggle to fully describe what they have in mind. This leads to a frustrating back-and-forth: giving instructions, getting unsatisfying results, and trying again. We wanted to see if AI could take a more active role by asking the right questions first, rather than waiting for a perfect description. We built an agent called Nous that interviews the user before doing any work, much like a thoughtful collaborator would. To teach it which questions matter most, we measured how much each answer reduces the AI's confusion about what the user actually wants, an idea borrowed from information theory. This meant we could train the agent automatically, without needing people to manually judge every conversation. In our tests, Nous consistently figured out what users wanted in fewer exchanges and produced better results than existing approaches. It worked well regardless of whether the user was an expert or a complete beginner, suggesting that AI can become a more proactive partner in complex creative tasks.