Two-point deterministic equivalence for SGD in random feature models
Alexander Atanasov ⋅ Blake Bordelon ⋅ Jacob A Zavatone-Veth ⋅ Courtney Paquette ⋅ Cengiz Pehlevan
Abstract
We derive a novel deterministic equivalence for the two-point function of a random matrix resolvent. Using this result, we give a unified derivation of the performance of a wide variety of high-dimensional linear models trained with stochastic gradient descent. This includes high-dimensional linear regression, kernel regression, and random feature models. Our results include previously known asymptotics as well as novel ones.
Chat is not available.
Successful Page Load