Dissecting Post-Training: Uncovering the Complementary Roles of SFT and RL for Document Parsing

Document parsing, the task of extracting diverse content from PDFs while preserving their structural integrity, has been significantly advanced by Multimodal Large Language Models (MLLMs). These models have achieved remarkable success, largely driven by extensive post-training on massive datasets. This paper therefore undertakes a deep analysis of the two dominant adaptation strategies, Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), prompted by a puzzling observation on the PDF-to-Markdown task: SFT makes a negligible impact, especially on parsing complex tables and formulas, while RL achieves substantial overall gains. To unravel the reasons, our systematic investigation reveals a clear and complementary division of labor: SFT primarily operates as a structure learner, biased towards mastering the low-entropy syntax of document layouts. While it learns the format of a table, it struggles to ensure the fidelity of its high-entropy cell content. Conversely, RL excels as a content refiner by optimizing a holistic reward that reflects final accuracy. We further ground this phenomenon in the distinct theoretical nature of their respective objective functions. Based on these findings, we introduce a unified strategy that explicitly harnesses their individual strengths while mitigating their weaknesses. This work shows that a deep understanding of post-training methods is key to unlocking performance beyond what data scaling alone can achieve.

Many AI systems now read PDFs and turn them into structured text, but this is difficult because a document is not just a sequence of words: it may contain headings, tables, formulas, and a reading order that must all be preserved. This paper asks a practical question: after a large vision-language model is built, what is the best way to further train it for document parsing? We compare two common training approaches: teaching the model by showing correct examples, and teaching it by giving rewards for better outputs. Surprisingly, simply showing examples brings limited improvement when the model must directly generate Markdown, while reward-based training improves the final parsing quality much more. By studying this behavior, we find that example-based training is better at learning the “skeleton” of a document, such as layout and formatting rules, whereas reward-based training is better at polishing the actual content, such as text, tables, and formulas. Based on this insight, we design a combined strategy: first help the model learn structure more clearly, then use carefully designed rewards to refine content. This leads to large improvements across text, formula, table, and reading-order accuracy. More broadly, our work shows that better document AI does not only require more data; it also requires understanding what each training method is actually good at.