Learning from Self Critique and Refinement for Faithful LLM Summarization

Large Language Models (LLMs) often suffer from hallucinations, i.e. information that is not grounded by the input context, while performing long-form text generation tasks. Prior works showed that strong LLMs can mitigate hallucinations by judging and refining the previous generated output from themselves or weaker models. However, these approaches typically require additional computation in test time, or rely on stronger teacher models, making them costly and less practical. In this work, we propose a self-distillation framework to enhance the faithfulness of LLMs without external supervision. Our method constructs a preference dataset by leveraging the model’s own self-refinement and self-rewarding capabilities, and then applies preference learning to improve the same LLM. Experiments on three summarization benchmarks—XSUM, CNN/DailyMail, and SAMSum—demonstrate that our approach outperforms state-of-the-art self-supervised learning baselines in faithfulness metrics, while incurring no performance degradation in other metrics. In addition, our approach also improve both faithfulness and test-time efficiency in comparison to the test-time scaling method with the same refinement mechanism.