Decision-Focused Learning via Tangent-Space Projection of Prediction Error

Decision-Focused Learning (DFL) trains predictors to improve downstream decision quality, but computing regret gradients typically requires differentiating through solvers or relying on surrogate losses, which can be computationally expensive or deviate from the true objective. We show that, under standard regularity with locally stable active constraints, the regret gradient admits a closed-form geometric characterization, equivalent to the prediction error projected onto the tangent space of active constraints, scaled by local curvature. This reveals that regret gradients can be obtained by filtering decision-irrelevant components from the MSE gradient, providing a simpler and more direct alternative to existing approaches. We propose PEAR (Projected Error As Regret-gradient), which computes regret gradients via a reduced linear system over active constraints, avoiding differentiation through solver iterations or additional optimization solves. Experiments on LP benchmarks and a real-world QP task show that PEAR achieves the best decision quality among all baselines while being the most computationally efficient, with gains that persist under constraint shifts.

Machine learning models are typically trained to make accurate predictions. Yet in many real-world settings, predictions are only the first step in a decision, such as choosing where to invest, scheduling deliveries, or routing traffic. And not every prediction error affects the resulting decision equally. Some can be badly wrong without changing the chosen action at all, while others can be only slightly off and yet completely change the outcome. Decision-Focused Learning addresses this by training models with the decision in mind. But existing work has focused on how to compute such a training signal, not on its underlying structure. We found that this signal has a surprisingly simple geometric form. It is the prediction error with its decision-irrelevant parts filtered out. Only the part of the error that could change the chosen decision matters, and the rest can be safely ignored. Building on this insight, we developed PEAR, which extracts this filtered error through a single small computation, without differentiating through an optimization solver or approximating the true objective. On standard benchmarks and a real-world financial task, PEAR led to better decisions than existing methods while running faster, and its advantage held even when the rules of the decision problem changed.