Overcoming the Incentive Collapse Paradox

Human labeling increasingly relies on AI assistance, raising incentive challenges when annotators’ effort is unobserved. Recent work by Bastani & Cachon (2025); Sambasivan et al. (2021) shows that accuracy-based payment schemes suffer from incentive collapse: as AI accuracy improves, sustaining positive human effort requires unbounded payments. We study this problem in a budget-constrained setting with strategic annotators whose labeling accuracy depends on unobserved effort. We propose a sentinel-auditing payment mechanism that enforces a strictly positive and controllable level of human effort at finite cost, independent of AI accuracy. Building on this incentive-robust foundation, we develop an incentive-aware active statistical inference framework that jointly optimizes (i) the auditing rate and (ii) active sampling and budget allocation across instances of varying difficulty to minimize the final statistical loss under a single budget. Experiments demonstrate improved cost–error tradeoffs relative to standard active learning and auditing-only baselines.