CausalGame: Benchmarking Causal Thinking of LLM Agents in Games

Recently, it has received growing attention in building AI Scientist agents with Large Language Models (LLMs). Since scientific discovery fundamentally relies on uncovering causal relationships from observations, the capability of causal thinking that distinguish causation from correlation and hidden biases, is essential to LLM agents. Despite a number of existing benchmarks for AI scientists, they do not explicitly incorporate challenges from hidden confounders, selection bias, and noisy measurements that widely exist in real-world scientific discovery. To this end, we present CausalGame, a benchmark that evaluates the causal thinking capabilities of LLM agents through interactive games. More specifically, we ask LLM agents to actively design experimental protocols, collect observation data and derive a final solution with an explanation report. To emulate realistic scientific discovery challenges, we design 14 game settings with the incorporation of selection bias, noisy measurements, and hidden confounders. The results with 29 frontier LLM agents show that they consistently fail to reason about and recover the underlying causal relationships required to solve the games. CausalGame provides a controlled testbed for evaluating causal thinking of AI Scientist agents. The project is available at causalgame.github.io .

There is growing interest in turning AI language models into automated "scientists" that form hypotheses and run experiments. But real discovery hinges on telling genuine cause and effect apart from coincidence, and history is full of scientists fooled by misleading data, such as early evidence that seemed to show high altitude protected against cholera when the real culprit was contaminated water lower down. To test whether today's AI agents have this skill, we built CausalGame, a set of interactive games where an AI plays a drone designer who must run its own small experiments to uncover the hidden rules that decide whether a drone survives. We deliberately planted the same traps that mislead real scientists, including only being able to observe drones that survived, noisy measurements, and important factors hidden from view. Testing 29 leading AI models across 14 scenarios, we found they consistently failed to recover the true cause-and-effect rules and were easily fooled by surface patterns. More "thinking time" did not help. This is a sobering reality check for anyone hoping to trust AI with scientific discovery.