Plenary Speaker
in
Workshop: High-dimensional Learning Dynamics Workshop: The Emergence of Structure and Reasoning
Phase transition in high-dimensional learning, Lenka Zdeborová
Lenka Zdeborova
Abstract:
Emergence in LLMs is surrounded by a plethora of open questions. Emergence in physics is linked to phase transitions. We will describe recent progress in characterizing phase transitions in the performance of neural networks and their consequences for algorithmic hardness. In particular, we will discuss how the staircase picture changes when batches are reused. We also unveil a phase transition between semantic and positional learning in a toy model of dot-product attention.
Chat is not available.