Bayesian Inference for Transductive Learning of Kernel Matrix Using the Tanner-Wong Data Augmentation Algorithm
Zhihua Zhang - Hong Kong University of Science and Technology Dit-Yan Yeung - Hong Kong University of Science and Technology James T. Kwok - Hong Kong University of Science and Technology
In kernel methods, an interesting recent development seeks to learn a goodkernel from empirical data automatically. In this paper, regarding thetransductive learning of the kernel matrix as a missing data problem, wepropose a Bayesian hierarchical model for the problem and devise the Tanner-Wong data augmentation algorithm for making inference on the model. The Tanner-Wong algorithm is closely related to Gibbs sampling, and it also bears a strongresemblance to the expectation-maximization (EM) algorithm. For an efficientimplementation, we also propose a simplified Bayesian hierarchical model andthe corresponding Tanner-Wong algorithm. We express the relationship betweenthe kernel on the input space and the kernel on the output space as asymmetric-definite generalized eigenproblem. Based on this eigenproblem, anefficient approach to choosing the base kernel matrices is presented. Theeffectiveness of the our Bayesian model with the Tanner-Wong algorithm isdemonstrated through some classification tasks showing promising results.