Removing Sandbagging in LLMs by Training with Weak Supervision

As AI systems begin to automate complex tasks, supervision increasingly relies on weaker models or limited human oversight that cannot fully verify output quality. A model more capable than its supervisors could exploit this gap through sandbagging, producing work that appears acceptable but falls short of its true abilities. Can training elicit a model's best work even without reliable verification? We study this using model organisms trained to sandbag, testing elicitation techniques on problem-solving math, graduate-level science, and competitive coding tasks. We find that training with weak supervision can reliably elicit sandbagging models when supervised fine-tuning (SFT) and reinforcement learning (RL) are combined: SFT on weak demonstrations breaks the sandbagging behavior, enabling RL to then fully elicit performance. Neither method succeeds reliably alone—RL without SFT almost always leads to reward hacking rather than genuine improvement, and SFT without RL fails to elicit full performance when the supervisor is much weaker than the untrusted model. Critically, this relies on training being indistinguishable from deployment; when models can distinguish between training and deployment, they can perform well during training while continuing to sandbag afterward. Our results provide initial evidence that training is a viable mitigation against sandbagging, while highlighting the importance of making training indistinguishable from deployment.

Future AI systems will be used for tasks where we cannot easily check whether the work has been done well — for instance, research on AI safety itself. A misaligned AI in such a setting could deliberately do worse than it is able to, a behaviour known as sandbagging, while producing work that looks acceptable to whoever is checking. A natural response is to train the model to do the task properly. But if our ability to evaluate its outputs is limited, can such training actually elicit the model's real capabilities? We study this question by deliberately training a capable model to sandbag on competition math, graduate-level science, and competitive programming problems, then attempt to recover its true performance using only weaker AI models as supervisors. When the supervisor is moderately weaker than the sandbagging model, supervised fine-tuning on its demonstrations alone elicits nearly the model's full performance. When the supervisor is substantially weaker, fine-tuning alone is insufficient, but following it with a stage of reinforcement learning closes the remaining gap. In all cases, success depends on the model being unable to tell whether it is being trained or deployed; otherwise it can comply during training and continue sandbagging afterwards. Our results provide initial evidence that training is a viable defence against sandbagging in capable AI systems, while highlighting the importance of designing training procedures that the model cannot distinguish from real deployment.