Oral
Nonoverlap-Promoting Variable Selection
Pengtao Xie · Hongbao Zhang · Yichen Zhu · Eric Xing

Wed Jul 11th 11:20 -- 11:30 AM @ K1 + K2
in Feature Selection 1 »

Variable selection is a classic problem in machine learning (ML), widely used to find important explanatory factors, and improve generalization performance and interpretability of ML models. In this paper, we consider variable selection for models where multiple responses are to be predicted based on the same set of covariates. Since each response is relevant to a unique subset of covariates, we desire the selected variables for different responses have small overlap. We propose a regularizer that simultaneously encourage orthogonality and sparsity, which jointly brings in an effect of reducing overlap. We apply this regularizer to four model instances and develop efficient algorithms to solve the regularized problems. We provide a formal analysis on why the proposed regularizer can reduce generalization error. Experiments on both simulation studies and real-world datasets demonstrate the effectiveness of the proposed regularizer in selecting less-overlapped variables and improving generalization performance.

Author Information

Pengtao Xie (Carnegie Mellon University)
Hongbao Zhang (Petuum Inc)
Yichen Zhu (Peking University)
Eric Xing (Petuum Inc. and CMU)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors