L-SR1: Learned Symmetric-Rank-One Preconditioning

End-to-end deep learning has achieved impressive results but often relies on large labeled datasets, exhibits limited generalization to unseen scenarios, and incurs substantial computational cost. Classical optimization methods, in contrast, are more data-efficient and lightweight but frequently suffer from slow convergence. Learned optimizers aim to bridge this gap, yet existing approaches have focused primarily on first-order methods, while learned second-order optimization has received much less attention. We introduce L-SR1, a learned second-order optimizer inspired by the classical Symmetric Rank-One (SR1) method. At its core, L-SR1 employs a Projection-Guided Secant Mechanism (PGSM) that generates positive semi-definite preconditioners and biases meta-training toward the quasi-Newton secant relation. Through controlled analytic benchmarks, we study stability, generalization across problem dimensions, and search direction quality, and further evaluate L-SR1 on Monocular Human Mesh Recovery (HMR), where it outperforms both classical and learned optimization-based baselines. With a compact model and no reliance on task-specific fine-tuning or annotated data, L-SR1 demonstrates strong generalization and can be integrated into a broad range of iterative optimization problems to accelerate convergence and reduce the required number of iterations.

Many computer-vision and graphics tasks still rely on iterative optimization: the computer repeatedly adjusts unknown parameters until a 3D model or other solution fits the data. Classical optimizers are data-efficient but often need many slow steps. Deep-learning “learned optimizers” can speed this up, but most work has focused on simple first-order updates rather than methods that use richer curvature information from second-order optimization. We introduce L-SR1, a compact learned optimizer inspired by a classical second-order method (SR1). It learns how to choose update directions while keeping them mathematically stable through a mechanism we call PGSM (Projection-Guided Secant Mechanism). We test L-SR1 on standard mathematical benchmarks and on recovering a 3D human body mesh from a single 2D image—a setting where each optimization step is expensive. L-SR1 reaches good solutions in fewer iterations than strong classical and learned baselines in our experiments, using a smaller model and without extra task-specific fine-tuning for the mesh-recovery application. We believe it can be integrated into a broad range of iterative optimization problems to accelerate convergence and reduce the number of steps required to reach a solution.