When is theory useful in practice? A guide to pitching your work to LLM trainers, Stella Biderman

A commonly cited motivation for doing theoretical work in interpretability and learning dynamics is a desire to empower people who train models, so they can train models better. It's very common for work to fall short of this goal, not because it's bad research but because of the way it is scoped, framed, and designed. Drawing on her experience as both a theorist and a LLM trainer, Stella will discuss what common pitfalls she sees preventing high quality research from having real-world impact and detail how she designs theoretical research programs with an eye towards building tools that will be practically useful when training models.

Bio: Stella Biderman is the executive director of EleutherAI. Her research focuses on understanding how large language models and other large-scale AI systems behave with an eye towards empowering model trainers and model deployers to build systems that behave more desirably. She's also an advocate for free and open source AI technologies and works to ensure that there are public and transparent options for entire technology stack.