What to optimize for – from robot arms to frontier AI - Anca Dragan

How to move losses down, and rewards and metrics up: from a robot’s arm motion in my PhD, to the policy of a virtual assistant or of a self-driving car in my Berkeley lab and at Waymo later, to the Gemini model today at Google DeepMind, that’s been the name of the game. But throughout it all, what I cared about more was what those losses/rewards/metrics ought to be in the first place. What started as an intuition in grad school – that what to optimize was the deeper and harder question than how to optimize – became a central pursuit when I became faculty, as my lab and I sought to understand the ins and outs of how agents can accomplish what we want without unintended side effects. Now at the heart of frontier AI development, that experience is coming in handy as we work to make Gemini a useful and safe collaborator for humanity.