SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models

We introduce SelfCite, a novel self-supervised approach that aligns LLMs to generate high-quality, fine-grained, sentence-level citations for the statements in their generated responses. Instead of only relying on costly and labor-intensive annotations, SelfCite leverages a reward signal provided by the LLM itself through context ablation: If a citation is necessary, removing the cited text from the context should prevent the same response; if sufficient, retaining the cited text alone should preserve the same response. This reward can guide the inference-time best-of-N sampling strategy to improve citation quality significantly, as well as be used in preference optimization to directly fine-tune the models for generating better citations. The effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3 points on the LongBench-Cite benchmark across five long-form question answering tasks. The source code is available at https://github.com/facebookresearch/SelfCite.

LLMs can support their answers with citations, but these citations are usually not accurate or fine-grained enough.Current methods rely on human-labeled data to train LLMs to generate citations, which is costly and time-consuming.We built SelfCite, a self-rewarding system that enables LLMs to self-teach and judge their own citations without additional human labeling.The idea is simple: if removing cited sentences from the source documents changes the LLM’s answer, those sentences were necessary; if keeping only those sentences still yields the same answer, they were sufficient.By rewarding the LLM based on these two checks, SelfCite helps it choose better citations—ones that are both necessary and sufficient—as it generates responses.The LLM drafts several possible citations and selects the best candidate with the highest self-score. We can also train the model to directly produce better candidates.On a challenging benchmark of long, open-ended questions, this approach improved citation quality by up to five points.Clearer citations make it easier for journalists, educators, and everyday readers to trust or critically assess AI-generated content, bringing us closer to transparent and verifiable AI systems.