Proxy objectives in reinforcement learning from human feedback
John Schulman
2023 Invited Talk
Abstract
Proxy objectives are a fundamental concept in machine learning. That is, there's a true objective that we care about, but it's hard to compute or estimate, so instead we construct a locally-valid approximation and optimize that. I will examine reinforcement from human feedback with this lens, as a chain of approximations, each of which can widen the gap between the desired and achieved result.
Video
Chat is not available.
Successful Page Load