Invited Talk 3 (Jacob Andreas: Language Models as World Models?)

Title: Language Models as World Models?

Abstract: The extent to which language modeling induces representations of the world described by text—and the broader question of what can be learned about meaning from text alone—have remained a subject of ongoing debate across NLP and cognitive sciences. Some of these questions are terminological, and this talk will begin by trying to define a few different ways in which a predictor like a neural net might implicitly instantiate a world model. But the most important questions are empirical, so I'll conclude by describing a few pieces of evidence we have about how LMs represent the world described in their training data and the situations described in their input text.