MetaOthello: A Controlled Study of Multiple World Models in Transformers

Foundation models must handle multiple generative processes, yet mechanistic interpretability largely studies capabilities in isolation; it remains unclear how a single transformer organizes multiple, potentially conflicting "world models". Previous experiments on Othello-playing neural networks test world-model learning, but focus on a single game with a single set of rules. We introduce MetaOthello, a controlled suite of Othello-like games with shared syntax but different rules or tokenizations, and train small GPTs on mixed-variant data. We show that transformers trained on multiple Othello variants learn shared world-state representations: linear probes trained on one game intervene on another's board state nearly as well as matched probes. When the games conflict, the model resolves the resulting ambiguity through a localized mechanism we identify and steer. For isomorphic games with token remapping, representations are equivalent up to a single orthogonal rotation that generalizes across layers, showing the shared structure is abstract rather than tied to surface form. Together, these results show that transformers reconcile conflicting world models by sharing structure and localizing conflict. MetaOthello thus offers a path toward understanding how transformers organize many world models at once.

Modern AI systems are trained on enormously varied data and have to juggle many different "rule systems" at once. For example, the grammar of different languages, the syntax of different programming languages, the conventions of different tasks, etc. We still understand very little about how a single AI keeps all of these straight internally, especially when two of them disagree about what should come next. To study this in a setting we can fully control, we built MetaOthello: a family of board games that look identical on the surface but follow different hidden rules, and we trained small AI models to play a mix of them. Because we know each game's true rules exactly, we can check what the model has actually worked out inside. We found that the model does not keep a separate picture for each game. It builds one shared understanding of the board and reuses it everywhere the games agree, spending extra effort only where the rules genuinely conflict. That decision turns out to be handled by a small, identifiable part of the network — one we can locate and even nudge to flip which game the model "thinks" it is playing. We also found that scrambling the labels (board co-ordinates) it sees doesn't change this underlying picture, showing the understanding is about the game itself, not the surface symbols. Because everything here has a known right answer, MetaOthello gives researchers a proving ground for tools meant to inspect the inner workings of much larger AI systems, where the right answer is hidden.