KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering

Knowledge Base Question Answering (KBQA) challenges models to bridge the gap between natural language and strict knowledge graph schemas by generating executable logical forms. While Large Language Models (LLMs) have advanced this field, current approaches often struggle with a dichotomy of failure: they either generate hallucinated queries without verifying schema existence or exhibit rigid, template-based reasoning that mimics synthesized traces without true comprehension of the environment. To address these limitations, we present KBQA-R1, a framework that shifts the paradigm from text imitation to interaction optimization via Reinforcement Learning. Treating KBQA as a multi-turn decision process, our model learns to autonomously navigate the knowledge base using a structured action space, refining its reasoning strategies based on concrete execution feedback rather than static supervision. Furthermore, we introduce Referenced Rejection Sampling (RRS), a data synthesis method that resolves cold-start challenges by strictly aligning reasoning traces with ground-truth action sequences. Extensive experiments on WebQSP, GrailQA, and GraphQuestions demonstrate that KBQA-R1 achieves state-of-the-art performance. Code is available at https://anonymous.4open.science/r/KBQA-R1-814F.

This paper studies how to help language models answer questions using structured knowledge bases, such as databases of people, places, films, organizations, and their relationships. Existing language models can often produce fluent answers, but they may use invalid database fields, miss important relationships, or make unsupported claims. We propose KBQA-R1, a method that trains a language model to interact with a knowledge base step by step, check whether its actions are valid, and improve based on feedback from the database. This makes the model better at finding reliable answers and reduces unnecessary trial-and-error at test time. Experiments on several public question-answering benchmarks show that our method improves accuracy compared with previous approaches.