Read or Look? Textification Safety Boundaries for Efficient Multimodal Document QA

Question answering (QA) over multimodal documents must balance answer quality against inference cost. In efficient multimodal question answering (EMM-QA), this tradeoff appears early: vision-language models (VLMs) preserve page images but are expensive to invoke repeatedly, while optical character recognition (OCR) sidecars let large language models (LLMs) answer from cached text when that text remains faithful to the page. Existing document question answering evaluations compare models and architectures, but they do not identify the safety boundary where textification loses the evidence required by a question. We therefore compare image-preserving, text-only, and OCR-sidecar routes on visual-rich and text-only subsets derived from MMLongBench-Doc, and we measure both slice-level quality gaps and false-refusal behavior. The results reveal a clear boundary: visual-rich questions, especially those requiring figure evidence or cross-page grounding, still favor image-preserving inference, whereas single-page pure-text questions can often be answered safely from cached OCR text. These findings support a conservative policy for EMM-QA systems: use OCR+LLM only on validated low-risk slices, preserve images when visual evidence may be lost, and recheck suspicious OCR refusals with the image route.