MMClima: A Framework for Multimodal Climate Science Data and Evaluation

Climate change research increasingly requires AI systems that reason across text, dynamic visual content, and scientific figures, yet existing climate QA benchmarks are small, mostly textual, and cover a narrow range of models. We introduce MMClima, a large-scale multimodal climate question answering framework with over 104k expert-validated question–answer pairs spanning articles, video transcriptions, and figures across five core climate science domains. MMClima is constructed via automated claim extraction and QA synthesis with human-in-the-loop validation to ensure both scale and reliability. Using MMClima, we benchmark state-of-the-art multimodal language models on tasks requiring factual recall, visual interpretation, and cross-modal synthesis. We additionally fine-tune on the textual split to produce mmclima-70b-txt, a domain-adapted baseline that outperforms strong open- and closed-source models on textual QA. We release the dataset, evaluation pipeline, fine-tuned model weights, and data creation framework to support standardized multimodal evaluation for climate science.

Climate change information is often shared through articles, videos, reports, charts, and scientific figures. AI systems can now answer many climate questions, but we still need a clear way to check whether their answers are correct and reliable. We built MMClima, a large collection of climate science questions and answers. These questions come from text, video transcripts, and scientific figures, and they cover major climate topics such as air quality, oceans, ice, extreme events, and climate policy. Experts checked the questions and answers to make sure they are clear and trustworthy. We used MMClima to test many AI systems and found that some climate questions are still hard for them, especially questions that need exact details or understanding of figures. We also trained a climate-focused AI model that performs better on many text-based climate questions. MMClima can help researchers build better AI tools for climate education, science communication, and climate analysis.