RePro: Training Language Models to Faithfully Recycle the Web for Pretraining

High-quality data is a cornerstone of large language model (LLM) pretraining, yet its growth has not kept pace with the needs of frontier models. In this paper, we introduce RePro, a novel web recycling method that trains a relatively small LM with reinforcement learning to generate effective and faithful rephrasings of pretraining data. Specifically, we design one *quality* reward and three *faithfulness* rewards, optimizing the LM rephraser to convert organic data into high-quality rephrasings while maintaining its core semantics and structure. In our experiment, we train a rephraser as small as 1B parameters to recycle 72B tokens sampled from DCLM-RefinedWeb. Pretraining results on 400M, 1.4B, and 2.8B models demonstrate that RePro delivers 3.7\%-14.5\% relative accuracy gains over organic-only baseline on 22 downstream tasks, doubling the performance gains achieved by the state-of-the-art web recycling method that prompts a 70B rephraser. Experiments with different amounts of recycled data highlight that RePro improves organic data efficiency by 2-3$\times$. Individual and distributional analyses validate that RePro preserves more critical information and faithfully reflects the characteristics of organic data compared to prompting-based methods. Together, these results show that RePro provides an efficient and controllable path to effectively recycle organic data for pretraining. Our code is available at https://github.com/cxcscmu/RePro.

Today's large language models learn from huge amounts of text scraped from the open web, but the supply of high-quality text is running out. Researchers can rewrite low-quality web pages into cleaner training material, but the standard recipe leans on a giant, expensive AI model to do the rewriting, and the rewrites often drift away from what the original page actually said. We built RePro, a small AI rewriter trained with reinforcement learning, where the rewriter receives rewards on its own outputs and gradually improves to produce better ones. Each candidate rewrite is scored on two things: whether it reads as higher quality, and whether it stays faithful to the original in meaning, structure, and length. A rewriter just 1 billion parameters in size, trained this way, turns low-quality web text into clean material that a language model can actually learn from. Despite being roughly 87 times faster than the leading alternative, RePro doubles its accuracy gains across 22 reading, reasoning, and knowledge tests, and even beats baselines that have access to four times more raw web text. Faithful, efficient rewriting can keep stretching the limited supply of human-written text, pointing to a sustainable path for scaling the next generation of frontier models.