Human-AI Collaborative Decision-Making: Beyond Learning to Defer

Human-AI collaboration (HAIC) in decision-making aims to improve predictive performance, fairness and efficiency by creating synergistic teaming between human decision-makers and AI systems. A key challenge in HAIC is to determine who among AI and humans takes which decisions. Recently, learning to defer (L2D) has been presented as a promising approach to tackle this challenge. Nevertheless, L2D entails several often unfeasible requirements, such as the availability of predictions from human decision-makers for every instance or ground-truth labels independent from said decision-makers. Furthermore, neither L2D nor alternative approaches tackle fundamental issues of deploying HAIC into real-world scenarios, such as capacity management or non-stationarity. In this paper, we aim to identify these and other limitations, pointing to where opportunities for future research work in HAIC may lie.