Iterative Double Sketching for Faster Least-Squares Optimization

Rui Wang · Yanyan Ouyang · Wangli Xu

Hall E #732

Keywords: [ OPT: Convex ] [ T: Optimization ] [ OPT: Sampling and Optimization ] [ OPT: Large Scale, Parallel and Distributed ]


This work is concerned with the overdetermined linear least-squares problem for large scale data. We generalize the iterative Hessian sketching (IHS) algorithm and propose a new sketching framework named iterative double sketching (IDS) which uses approximations for both the gradient and the Hessian in each iteration. To understand the behavior of the IDS algorithm and choose the optimal hyperparameters, we derive the exact limit of the conditional prediction error of the IDS algorithm in the setting of Gaussian sketching. Guided by this theoretical result, we propose an efficient IDS algorithm via a new class of sequentially related sketching matrices. We give a non-asymptotic analysis of this efficient IDS algorithm which shows that the proposed algorithm achieves the state-of-the-art trade-off between accuracy and efficiency.

Chat is not available.