Invited Talk 5 (Sendhil Mullainathan: Testing for Understanding Requires First Defining It)

Title: Testing for Understanding Requires First Defining It

Abstract: We need to take a step back in how we assess algorithmic understanding. The benchmarks and probes we have are intuitive, interesting and revealing. But without a rigorous foundation it is hard to know how much confidence we should have in what they have found. To gain that confidence we must (i) formally define "understanding", (ii) from that formalism define a test and (iii) prove conditions under which the test is valid. I will follow this procedure for three related notions of understanding: one for concepts; one for world models for a single task and one for foundation models for many tasks. These three exercises highlight that many of our current tests are problematic, having a bias in favor of algorithms; they are too quick to conclude that models have understood the world.