Learning to Theorize the World from Observation

What does it mean to understand the world? Is it simply to predict future video frames? Developmental cognitive science suggests that understanding the world is fundamentally the process of constructing internal theories of how it works rather than mere prediction, even before language is acquired. However, in machine learning, it remains unclear how to endow AI systems with such theory-building capability from raw, non-textual observation alone. In this paper, we introduce Learning-to-Theorize (L2T), a learning paradigm in which an AI system acquires the ability to construct theories represented as executable programs directly from observation alone. To instantiate this paradigm, we propose the Neural Language-of-Thought Programmer, a neural model that induces and executes latent programs as explanations rather than task-specific predictors or policies. In experiments, we show that this formulation enables explanation-driven generalization, allowing observations to be understood in terms of the programs that generate them.

Today’s AI “world models” mostly measure understanding by how well they predict what happens next. But this tends to capture surface-level correlations, so these models break down the moment they face an unfamiliar situation. Human infants, by contrast, build little “theories” of how and why the world works long before they learn language, and that is how they come to understand it. Inspired by this idea, we propose Learning-to-Theorize, a way for an AI to construct its own explanations of the world purely from raw observations, and a model called the Neural Theorizer (NEO) that realizes it. Without any labels or language, NEO looks only at a before-and-after pair—say, an apple on a tree and then the same apple on the ground—and infers a short “recipe” of reusable basic operations that explains the change, preferring the simplest such explanation. Because each explanation is built from recombinable parts, NEO can explain situations it never saw in training, including changes longer and more complex than any it learned from, by reassembling familiar building blocks. This is a step toward AI that understands the world by explaining it, rather than merely predicting what comes next.