THE CONFERENCE
FOR AUTHORS
FOR PARTICIPANTS

Timezone: »

 
Talk
Learning Latent Space Models with Angular Constraints
Pengtao Xie · Yuntian Deng · Yi Zhou · Abhimanu Kumar · Yaoliang Yu · James Zou · Eric Xing

Mon Aug 07 11:42 PM -- 12:00 AM (PDT) @ C4.9& C4.10
in Probabilistic inference 3 »

The large model capacity of latent space models (LSMs) enables them to achieve great performance on various applications, but meanwhile renders LSMs to be prone to overfitting. Several recent studies investigate a new type of regularization approach, which encourages components in LSMs to be diverse, for the sake of alleviating overfitting. While they have shown promising empirical effectiveness, in theory why larger "diversity" results in less overfitting is still unclear. To bridge this gap, we propose a new diversity-promoting approach that is both theoretically analyzable and empirically effective. Specifically, we use near-orthogonality to characterize "diversity" and impose angular constraints (ACs) on the components of LSMs to promote diversity. A generalization error analysis shows that larger diversity results in smaller estimation error and larger approximation error. An efficient ADMM algorithm is developed to solve the constrained LSM problems. Experiments demonstrate that ACs improve generalization performance of LSMs and outperform other diversity-promoting approaches.

[ PDF]  [ ss Summary/Notes]  [ Video] 

Author Information

Pengtao Xie (Carnegie Mellon University)
Yuntian Deng (Harvard University)
Yi Zhou (Syracuse University)
Abhimanu Kumar (IMC Financial Markets)
Yaoliang Yu (University of Waterloo)
James Zou (Stanford)
Eric Xing (Carnegie Mellon University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors