Sunday, June 16, 2013
Morning Tutorials: 8:30 – noon
Submodularity in Machine Learning: New Directions
Andreas Krause, Stefanie Jegelka
Room: Marquis 103, 104, 105
Numerous problems in machine learning are inherently discrete. More often than not, these lead to challenging optimization problems. While convexity is an important property when solving continuous optimization problems, submodularity, often viewed as a discrete analog of convexity, is key to solving many discrete problems. Its characterizing property, diminishing marginal returns, appears naturally in a multitude of settings. While submodularity has long been recognized in combinatorial optimization and game theory, it has seen a recent surge of interest in theoretical computer science and machine learning. This tutorial will introduce the concept of submodularity and its basic properties, and outline recent research directions — such as new approaches towards large-scale optimization, learning submodular functions and sequential decision making tasks. We will discuss recent applications to challenging machine learning problems such as high-order graphical model inference, sparsity, document summarization, active learning and recommendation. The tutorial will not assume any specific prior knowledge.
A list of further resources may be found at submodularity.org
Deep Learning
Yann Lecun, Marc’Aurelio Ranzato
Room: International 7,8,9
Tutorial slides: http://www.cs.nyu.edu/~yann/
Tensor Decomposition Algorithms for Latent Variable Model Estimation
Anima Anandkumar, Daniel Hsu, Sham M. Kakade
Room: International 1, 2, 3
This tutorial surveys algorithms for learning latent variable models based on the method-of-moments, focusing on algorithms based on low-rank decompositions of higher-order tensors. The target audiences of the tutorial include (i) users of latent variable models in applications, and (ii) researchers developing techniques for learning latent variable models. The only prior knowledge expected of the audience is a familiarity with simple latent variable models (e.g., mixtures of Gaussians), and rudimentary linear algebra and probability. The audience will learn about new algorithms for learning latent variable models, techniques for developing new learning algorithms based on spectral decompositions, and analytical techniques for understanding the aforementioned models and algorithms. Advanced topics such as learning overcomplete represenations may also be discussed.
Tutorial website: http://cseweb.ucsd.edu/~djhsu/
Multi-Target Prediction
Willem Waegeman, Krzysztof Dembczynski and Eyke Hullermeier
Room: International 10
Traditional methods in machine learning and statistics provide data-driven models for predicting one-dimensional targets, such as binary outputs in classification and real-valued outputs in regression. In recent years, novel application domains have triggered fundamental research on more complicated problems where multi-target predictions are required. Such problems arise in diverse application domains, such as document categorization, tag recommendation of images, videos and music, information retrieval, natural language processing, drug discovery, marketing, biology, etc. Specific multi-target prediction problems have been studied in a variety of subfields of machine learning and statistics, such as multi-label classification, multivariate regression, sequence learning, structured output prediction, preference learning, multi-task learning, recommender systems and collective learning. Despite their commonalities, work on solving problems in the above domains has typically been performed in isolation, without much interaction between the different sub-communities. The main goal of the tutorial is to present a unifying overview of the above-mentioned subfields of machine learning, by focusing on the simultaneous prediction of multiple, mutually dependent output variables. We will distinguish two different views on these problems. The individual-target view concerns improving the prediction quality of a single target by using information from other targets. The joint-target view concerns minimization of complex loss functions that cannot be decomposed to single targets.
Tutorial website: http://www.ngdata.com/
Afternoon Tutorials: 2:00 – 5:30
Topological Data Analysis
Primoz Skraba and Sayan Mukherjee
Room: International 9
This tutorial will provide an introduction to Topological Data Analysis (TDA) to the machine learning community. The idea behind TDA is to extract robust topological features from data and use these summaries for modeling the data. Topological features can quantify qualitative changes in data and formalize its global structure. We will cover one of the main tools in TDA, persistent homology, including the necessary background. In particular, no prior knowledge of algebraic topology will be required or assumed. The focus will be on how these methods relate to and may be applied in machine learning. The tutorial will also give a brief overview of some of the available tools and current relevant directions of research in the field.
Discovering Multiple Clustering Solutions: Grouping Objects in Different Views of Data
Emmanuel Muller, Stephan Gunnemann, Ines Farber, Thomas Seidl
Room: International 1, 2, 3
Traditional clustering algorithms identify just a single clustering of the data. Today’s complex data, however, allow multiple interpretations leading to several valid groupings hidden in different views of the database. Each of these multiple clustering solutions is valuable and interesting for the user as they provide different perspectives on the same data. Due to multiple views each object is represented in several meaningful groupings. Especially for high dimensional data, where each object is described by multiple attributes, alternative clusters in different attribute subsets are of major interest. In this tutorial, we describe several real world application scenarios for multiple clustering solutions. We abstract from these scenarios and provide the general challenges in this emerging research area. We describe state-of-the-art paradigms proposed by different communities, discuss specific techniques, and give an overview of this topic by providing a taxonomy of existing methods.
Tutorial website: http://dme.rwth-aachen.de/en/
Copulas in Machine Learning
Gal Elidan
Room: International 7, 8
From high-throughput biology to astronomy to medical diagnosis, a wide variety of complex high-dimensional domains are inherently continuous. The statistical copula framework is a powerful mechanism for constructing multivariate real-valued distribution by separating the choice of the univariate marginals and that of the dependency structure. This provides great modeling flexibility that often leads to substantial predictive advantages. Surprisingly, copulas are only now being discovered in machine learning.
In the first part of this tutorial I will present the basics of this “distribution generating” framework with an emphasis on concepts that are relevant to the machine learning community. In the second part of the tutorial, I will present several high-dimensional copula-based constructions that have emerged in the machine learning community in recent years.
Music Information Research Based on Machine Learning
Masataka Goto and Kazuyoshi Yoshii
Room: International 10
This tutorial is intended for an audience interested in the application of machine learning techniques to music domains. Audience members who are not familiar with music information research are welcome, and researchers working on music technologies are likely to find something new to study.
First, the tutorial serves as a showcase of music information research. The audience can enjoy and study many state-of-the-art demonstrations of music information research based on signal processing and machine learning. This tutorial highlights timely topics such as active music listening interfaces, singing information processing systems, web-related music technologies, crowdsourcing, and consumer-generated media (CGM).
Second, this tutorial explains the music technologies behind the demonstrations. The audience can learn how to analyze and understand musical audio signals, process singing voices, and model polyphonic sound mixtures. As a new approach to advanced music modeling, this tutorial introduces unsupervised music understanding based on nonparametric Bayesian models.
Third, this tutorial provides a practical guide to getting started in music information research. The audience can try available research tools such as music feature extraction, machine learning, and music editors. Music databases and corpora are then introduced. As a hint towards research topics, this tutorial also discusses open problems and grand challenges that the audience members are encouraged to tackle.
Tutorial website: http://staff.aist.go.jp/m.