The learning to defer (L2D) framework allows models to defer their decisions to human experts. For L2D, the Bayes optimality is the basic requirement on theoretical guarantees for the design of consistent surrogate loss functions, which means that the minimizer (i.e., learned classifier) by the surrogate loss is the Bayes optimality. However, we find that the Bayes optimality in the original form fails to consider the dependency between the model and the expert, and such a dependency could be further exploited to design a better consistent loss for L2D. In this paper, we provide a new formulation for the Bayes optimality called dependent Bayes optimality, which reveals the dependency pattern in determining whether to defer. Based on the dependent Bayes optimality, we further propose a novel consistent surrogate loss that can explicitly utilize this dependency pattern. Comprehensive experimental results on both synthetic and real-world datasets demonstrate the superiority of our method.