RTPrune: Reading-Twice Inspired Token Pruning for Efficient DeepSeek-OCR Inference

DeepSeek-OCR leverages visual–text compression to reduce long-text processing costs and accelerate inference, yet visual tokens remain prone to redundant textual and structural information. Moreover, current token pruning methods for conventional vision–language models (VLMs) fail to preserve textual fidelity due to improper compression mechanisms. By analyzing the decoding process of DeepSeek-OCR, we find that a distinct two-stage reading trajectory: the model initially prioritizes the majority of high-norm tokens, then subsequently redistributes its attention to the remaining ones. Motivated by this insight, we propose RTPrune, a two-stage token pruning method tailored for DeepSeek-OCR. In the first stage, we prioritize high-norm visual tokens that capture salient textual and structural information. In the second stage, the remaining tokens are paired and merged based on optimal transport theory to achieve efficient feature aggregation. We further introduce a dynamic pruning ratio that adapts to token similarity and textual density for OCR tasks, enabling a better efficiency–accuracy trade-off. Extensive experiments demonstrate state-of-the-art performance, as evidenced by 99.47\% accuracy and 1.23× faster prefill on OmniDocBench, achieved with 84.25\% token retention when applied to DeepSeek-OCR-Large. Code is released.

DeepSeek-OCR accelerates large language models (LLMs) inference by a small amount of visual tokens, demonstrating the effectiveness of visual modality as an efficient compression medium for textual information. However, not all of these tokens are useful and many contain redundant background or layout details that slow the inference process down. We discover that during reading, the model acts like it reads twice: it first quickly grasps the most important parts and then carefully checks the rest to fill in the missing details. Based on this, we built RTPrune, a tool that mimics this behavior. First, it identifies and keeps only the most crucial tokens. Then, instead of just throwing the rest away, it smartly merges the remaining useful bits of information into the kept tokens using a mathematical matching method. It also automatically adjusts how much to prune based on how dense the text in the image is. Our method allows the OCR model to read documents about 1.23× faster while maintaining over 99% accuracy, saving significant computational power without sacrificing quality.