Skip to yearly menu bar Skip to main content


Poster

Towards Theoretical Understanding of Learning Large-scale Dependent Data via Random Features

Chao Wang · Xin Bing · Xin HE · Caixing Wang


Abstract: Random feature (RF) mapping is an attractive and powerful technique for solving large-scale nonparametric regression. Yet, the existing theoretical analysis crucially relies on the i.i.d. assumption that individuals in the data are independent and identically distributed. It is still unclear whether learning accuracy will be compromised when such an assumption is violated. This paper aims to provide theoretical understanding of the kernel ridge regression (KRR) with RFs for large-scale dependent data. Specifically, we consider two types of data dependence structure, where one is the $\tau$-mixing process with exponential decay coefficient, and another is the $\tau$-mixing process with polynomial decay coefficient. Theoretically, we prove that the kernel ridge estimator with RFs achieves the minimax optimality under the exponential decay case, but yields a sub-optimal result under the polynomial decay case. Our analysis further reveals how the decay rate of the $\tau$-mixing coefficient impacts the learning accuracy of the kernel ridge estimator with RFs, which, to the best of our knowledge, is new. Extensive numerical experiments on both synthetic and real examples further validate our theoretical findings and support the effectiveness of the KRR with RFs in dealing with dependent data.

Live content is unavailable. Log in and register to view live content