The Matrix Stick-Breaking Pro cess for Flexible Multi-Task Learning

The Matrix Stick-Breaking Pro cess for Flexible Multi-Task Learning
Ya Xue - Department of Electrical & Computer Engineering, Duke University, USA David Dunson - Biostatistics Branch, National Institute of Environmental Health Sciences, USA Lawrence Carin - Department of Electrical & Computer Engineering, Duke University, USA
In multi-task learning our goal is to design regression or classification models for each of the tasks and appropriately share information between tasks. A Dirichlet process (DP) prior can be used to encourage task clustering. However, the DP prior does not allow local clustering of tasks with respect to a subset of the feature vector without making independence assumptions. Motivated by this problem, we develop a new multitask-learning prior, termed the matrix stickbreaking process (MSBP), which encourages cross-task sharing of data. However, the MSBP allows separate clustering and borrowing of information for the different feature components. This is important when tasks are more closely related for certain features than for others. Bayesian inference proceeds by a Gibbs sampling algorithm and the approach is illustrated using a simulated example and a multi-national application.

Ya Xue - Department of Electrical & Computer Engineering, Duke University, USA
David Dunson - Biostatistics Branch, National Institute of Environmental Health Sciences, USA
Lawrence Carin - Department of Electrical & Computer Engineering, Duke University, USA

In multi-task learning our goal is to design regression or classification models for each of the tasks and appropriately share information between tasks. A Dirichlet process (DP) prior can be used to encourage task clustering. However, the DP prior does not allow local clustering of tasks with respect to a subset of the feature vector without making independence assumptions. Motivated by this problem, we develop a new multitask-learning prior, termed the matrix stickbreaking process (MSBP), which encourages cross-task sharing of data. However, the MSBP allows separate clustering and borrowing of information for the different feature components. This is important when tasks are more closely related for certain features than for others. Bayesian inference proceeds by a Gibbs sampling algorithm and the approach is illustrated using a simulated example and a multi-national application.