CoDA-Bench: Can Code Agents Handle Data-Intensive Tasks?

Advanced agents are increasingly demonstrating the potential to operate as autonomous engineers, creating a growing demand for evaluation benchmarks that capture the complexity of real-world development. Such environments typically involve both complex code and large-scale data (i.e., file system). However, existing benchmarks usually evaluate code-centric or data-centric capabilities in isolation, leaving a clear gap with real development scenarios. In this paper, we bridge this gap by introducing CoDA-Bench, the first benchmark to jointly evaluate code and data intelligence in a data-intensive environment. We construct a data-intensive Linux sandbox based on the Kaggle ecosystem (containing hundreds of datasets), where agents must actively explore complex file hierarchies to identify relevant resources and generate code for data-driven analytical tasks. CoDA-Bench comprises 1,009 tasks spanning 31 communities, with each task environment containing an average of 980 files, simulating realistic data scale and noise. Evaluations of advanced agents reveal that even top-performing systems struggle to effectively integrate data discovery with code execution, achieving a success rate of only 61.1\%. These results highlight a substantial gap in current agentic capabilities for data-intensive tasks and point to promising directions for future research.

AI agents are increasingly being used to help people write code and complete complex computer-based tasks. However, in many real-world situations, solving a problem is not only about writing correct code. The agent must also search through a large collection of files, identify which data is relevant, understand different file formats, and then use the right data to produce the final answer. This paper introduces CODA-BENCH, a benchmark designed to test whether current AI coding agents can handle such data-intensive tasks. In CODA-BENCH, an agent is placed in a realistic computer environment containing hundreds of data files, many of which are related but not useful for the task. The agent receives only a natural-language question and must figure out which files matter, write code to analyze them, and return the correct result. We build CODA-BENCH from real data analysis workflows in the Kaggle ecosystem, so the tasks reflect practical problems that data analysts may encounter. The benchmark contains 1,009 tasks across many data communities, with each environment containing nearly one thousand files on average. Our experiments show that even strong AI coding agents still struggle in this setting. The best system solves only about 61% of the tasks, indicating that current agents often fail either to find the right data or to use it correctly. These results suggest that future AI agents need stronger abilities to explore messy data environments, select useful information, and combine data discovery with reliable code execution.