MORE: A Multilingual Document Parsing Benchmark and Evaluation

Multilingual documents encapsulate rich regional cultures, scientific discoveries, and historical records. Parsing this content into structured, machine-readable formats is critical for unlocking global knowledge. However, existing benchmarks predominantly focus on high-resource languages like English and Chinese, creating an $\textit{evaluation blind spot}$ concerning model performance on other languages. While recent Vision-Language Models (VLMs) claim support for hundreds of languages, the lack of ground truth makes it impossible to empirically verify these capabilities. To bridge this gap, we introduce $\textbf{MORE}$, a large-scale benchmark designed for multilingual document parsing evaluation. MORE distinguishes itself through three key dimensions: (1) $\textbf{Unprecedented Scale}$: It covers $\textbf{149 languages}$, making it the most linguistically diverse benchmark to date; (2) $\textbf{Structural Complexity}$: Unlike previous works, it extends evaluation beyond plain text to include structural elements such as code blocks, tables, and catalogs; and (3) $\textbf{Data Authenticity}$: All samples are curated from real-world documents via a model-assisted, human-refined annotation pipeline. We evaluate state-of-the-art models using MORE, establishing new performance baselines for long-tail languages and validating the benchmark's effectiveness in diagnosing model capabilities in realistic, diverse scenarios. The MORE dataset will be available at https://github.com/zimoqingfeng/MORE.

(1) Problem: Documents are the ultimate vessels of human knowledge, and Artificial Intelligence is increasingly used to read them. However, most AI models are only tested on high-resource languages like English or Chinese. This creates a massive "blind spot": while modern AI claims to understand hundreds of languages, we actually have no reliable way to verify if it can accurately read a complex table in Welsh or a receipt in Amharic. (2) Solution: To fix this, we built MORE, the most diverse reading test for AI ever created. We collected real-world documents from the internet and carefully annotated them across 149 different languages. Unlike previous tests that only check simple text, MORE challenges AI to understand complex visual structures like tables, mathematical formulas, and computer code in these underrepresented languages. (3) Impact: By testing current state-of-the-art AI models on MORE, we discovered that while they are good at reading simple text, they still struggle heavily with complex layouts in less common scripts. Our benchmark provides a much-needed yardstick to help researchers build truly global, inclusive AI systems that leave no language behind.