Supervised Feature Selection via Dep endence Estimation

Supervised Feature Selection via Dep endence Estimation
Le Song - National ICT Australia, Australia Alex Smola - National ICT Australia, Australia Arthur Gretton - MPI Tübingen, Germany Karsten M. Borgwardt - LMU München, Germany Justin Bedo - National ICT Australia, Australia
We introduce a framework for filtering features that employs the Hilbert-Schmidt Independence Criterion (HSIC) as a measure of dependence between the features and the labels. The key idea is that good features should maximise such dependence. Feature selection for various supervised learning problems (including classification and regression) is unified under this framework, and the solutions can be approximated using a backward-elimination algorithm. We demonstrate the usefulness of our method on both artificial and real world datasets.

Le Song - National ICT Australia, Australia
Alex Smola - National ICT Australia, Australia
Arthur Gretton - MPI Tübingen, Germany
Karsten M. Borgwardt - LMU München, Germany
Justin Bedo - National ICT Australia, Australia

We introduce a framework for filtering features that employs the Hilbert-Schmidt Independence Criterion (HSIC) as a measure of dependence between the features and the labels. The key idea is that good features should maximise such dependence. Feature selection for various supervised learning problems (including classification and regression) is unified under this framework, and the solutions can be approximated using a backward-elimination algorithm. We demonstrate the usefulness of our method on both artificial and real world datasets.