Scaling Trends for Lie Detector Oversight in Preference Learning

Deceptive behavior in LLMs is costly to monitor and prevent, motivating approaches such as Scalable Oversight via Lie Detectors (SOLiD)~\citep{cundy2025preferencelearningliedetectors}, which uses lie detectors to identify responses for review by high-cost labelers. In this paper, we scale SOLiD to larger models and evaluate it in more diverse and realistic preference-learning settings. We find favorable scaling: undetected deception drops from 34\% for 1B-parameter models to 14\% for 405B-parameter models at a detector true positive rate of 99\%, and expensive human labelers can be removed entirely from the finetuning phase without a statistically significant increase in deception. However, SOLiD is sensitive to distribution shift between detector training and preference-training data, which can drive detector false positive rates to impractical levels.