THE CONFERENCE
FOR AUTHORS
FOR PARTICIPANTS

Timezone: »

 
Poster
Consistency Analysis for Binary Classification Revisited
Krzysztof Dembczynski · Wojciech Kotlowski · Sanmi Koyejo · Nagarajan Natarajan

Tue Aug 08 01:30 AM -- 05:00 AM (PDT) @ Gallery #104
in Posters Tue »

Statistical learning theory is at an inflection point enabled by recent advances in understanding and optimizing a wide range of metrics. Of particular interest are non-decomposable metrics such as the F-measure and the Jaccard measure which cannot be represented as a simple average over examples. Non-decomposability is the primary source of difficulty in theoretical analysis, and interestingly has led to two distinct settings and notions of consistency. In this manuscript we analyze both settings, from statistical and algorithmic points of view, to explore the connections and to highlight differences between them for a wide range of metrics. The analysis complements previous results on this topic, clarifies common confusions around both settings, and provides guidance to the theory and practice of binary classification with complex metrics.

Author Information

Krzysztof Dembczynski (Poznan University of Technology)

Krzysztof Dembczyński is an assistant professor at Poznań University of Technology. He received his B.Sc., M.Sc., and Ph.D. degrees in computer science from the same university. As a post-doctoral researcher he spent two years from 2009 to 2011 in the Knowledge Engineering & Bioinformatics Lab at Marburg University, Germany. His articles have been published at the main conferences (ECML,ICML, NIPS) and in the leading journals (JMLR, MLJ, DAMI) in the field of machine learning. As a co-author he won the best paper award at the European Conference on Artificial Intelligence 2012 and at the Asian Conference on Machine Learning 2015. He also gave a tutorial on multi-target prediction problems at the International Conference on Machine Learning 2013 and at Algorithmic Learning Theory/Discovery Science 2013. He serves as a member of the program committees of major conferences in the field of artificial intelligence (ICML, NIPS, IJCAI, AAAI, KDD) and as a reviewer for several international journals (MLJ, DAMI, JMLR). He is a laureate of a prestigious scholarship in the HOMING PLUS programme awarded by the Foundation for Polish Science (2012– 2014). He was also receiving a stipend for outstanding young scientists funded by the Polish Ministry of Science and Higher Education (2011–2013).

Wojciech Kotlowski (Poznan University of Technology)
Sanmi Koyejo (University of Illinois at Urbana-Champaign)

Sanmi (Oluwasanmi) Koyejo an Assistant Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign. Koyejo's research interests are in the development and analysis of probabilistic and statistical machine learning techniques motivated by, and applied to various modern big data problems. He is particularly interested in the analysis of large scale neuroimaging data. Koyejo completed his Ph.D in Electrical Engineering at the University of Texas at Austin advised by Joydeep Ghosh, and completed postdoctoral research at Stanford University with a focus on developing Machine learning techniques for neuroimaging data. His postdoctoral research was primarily with Russell A. Poldrack and Pradeep Ravikumar. Koyejo has been the recipient of several awards including the outstanding NCE/ECE student award, a best student paper award from the conference on uncertainty in artificial intelligence (UAI) and a trainee award from the Organization for Human Brain Mapping (OHBM).

Nagarajan Natarajan (Microsoft Research)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors