VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning

Multi-model learning has attracted great attention in visual-text tasks. However, visual-tabular data, which plays a pivotal role in high-stakes domains like healthcare and industry, remains underexplored. In this paper, we introduce \textit{VT-Bench}, the first unified benchmark for standardizing vision-tabular discriminative prediction and generative reasoning tasks. VT-Bench aggregates 14 datasets across 9 domains (medical-centric, while covering pets, media, and transportation) with over 756K samples. We evaluate 23 representative models, including unimodal experts, specialized visual-tabular models, general-purpose vision-language models (VLMs), and tool-augmented methods, highlighting substantial challenges of visual-tabular learning. We believe VT-Bench will stimulate the community to build more powerful multi-modal vision-tabular foundation models. Benchmark: \url{https://github.com/LAMDA-NeSy/VT-Bench}

Many real-world AI applications need to use images together with tables, such as medical images with patient records or product images with structured information. However, most existing benchmarks focus on image-text tasks, while image-table learning is still less studied. In this paper, we introduce VT-Bench, a benchmark for evaluating how well AI models combine visual and tabular information. VT-Bench includes 14 datasets from 9 domains and more than 756,000 examples, covering both prediction and reasoning tasks. We evaluate 23 representative models and find that current systems still struggle with this setting. We hope VT-Bench will help researchers build more reliable AI systems for real-world multimodal problems.