Breaking the Reference Bottleneck via Learning to Rewrite Conversational Queries without Gold Reference Passages

Conversational query reformulation (CQR) has become indispensable for improving retrieval in dialogue-based applications. However, existing approaches typically rely on reference passages for optimization, which are impractical to acquire in real-world scenarios. To address this limitation, we introduce a novel reference-free preference optimization framework DualReform that generates pseudo reference passages from commonly-encountered conversational datasets containing only queries and responses. DualReform attains this goal through two key innovations: (1) response-based inference, where responses serve as proxies to infer pseudo reference passages, and (2) response refinement via the dual-role of CQR, where a CQR model refines responses based on the shared objectives between response refinement and CQR. Despite not relying on reference passages, DualReform achieves 96.9--99.1% of the retrieval accuracy attainable only with reference passages and surpasses the state-of-the-art method by up to 31.6%.

When we ask an AI a question during a conversation, it often searches external databases for the answer. To do this well, the AI must rewrite our casual, incomplete questions into clear search queries. Traditionally, teaching this skill requires humans to manually find the perfect reference document for every training question. This is incredibly expensive and impossible to scale for massive real-world datasets. We introduced a new method to train the AI without any human-provided reference documents. Our approach allows the AI to automatically deduce the ideal reference material simply by analyzing the surrounding conversation. Surprisingly, this self-taught method achieves up to 99% of the search accuracy of models that rely on expensive human labeling. We can now build highly accurate, searchable conversational AI assistants much more affordably and on a significantly larger scale.