A Penalty Approach For Differentiation Through Black-box Quadratic Programming Solvers

Differentiating through the solution of a quadratic program (QP) is a central problem in differentiable optimization. Most existing approaches differentiate through the Karush--Kuhn--Tucker (KKT) system, but their computational cost and numerical robustness can degrade at scale. To address these limitations, we propose dXPP, a penalty-based differentiation framework that decouples QP solving from differentiation. In the solving step (forward pass), dXPP is solver-agnostic and can leverage any black-box QP solver. In the differentiation step (backward pass), we map the solution to a smooth approximate penalty problem and implicitly differentiate through it, requiring only the solution of a much smaller linear system in the primal variables. This approach bypasses the difficulties inherent in explicit KKT differentiation and significantly improves computational efficiency and robustness. We evaluate dXPP on various tasks, including randomly generated QPs, large-scale sparse projection problems, and a real-world multi-period portfolio optimization task. Empirical results demonstrate that dXPP is competitive with KKT-based differentiation methods and achieves substantial speedups on large-scale problems. Our implementation is open source and available at https://github.com/mmmmmmlinghu/dXPP.

Many machine learning models need to make decisions by solving optimization problems, such as choosing a portfolio or allocating resources. To train these models end-to-end, we need to compute how the optimized decision changes when the model inputs change. Existing methods can be slow or unstable for large problems. This paper proposes dXPP, a method for differentiating through quadratic programs. It uses a standard black-box solver to find the optimal decision, then computes gradients through a smooth penalty-based reformulation instead of the usual optimality equations. This makes the backward pass smaller, more stable, and easier to scale. Experiments show that dXPP gives accurate gradients and is faster than existing methods on large optimization problems.