Registration Desk: Registration Check-in Desk Mon 18 Jul 07:00 a.m.
Registration Check-in Desk closing at 6 pm. Badge pickup.
Affinity Workshop: LatinX in AI (LXAI) LXAI Research Workshop Mon 18 Jul 08:00 a.m.
The LatinX in AI research workshop is a one-day event with invited speakers, oral presentations, and research posters. The event brings together faculty, graduate students, research scientists, and engineers for an opportunity to connect and exchange ideas. There will be a panel discussion and a mentoring session to discuss current research trends and career choices in artificial intelligence and machine learning, highlighting the unique challenges of LatinX identifying researchers. The workshop aims to create a platform for the work of Latinx researchers and we invite everyone to attend.We strongly encourage students, postdocs, and researchers who primarily identify as Latinx in all areas of machine learning to submit an abstract describing new, previously, or concurrently published research. We welcome abstract submissions, in theory, methodology, as well as applications. Abstracts may describe completed research or work-in-progress. While the presenting author need not be the first author of the work, we encourage authors to highlight the contribution of Latinx individuals — particularly the presenting author — in the abstract. The LatinX authors of accepted abstracts will be asked to present their work in a poster session. A few authors will be selected to give 15-minute oral presentations. Authors accepted to present will be offered presentation coaching. Submissions will be peer-reviewed. The authors are encouraged to sign up to review as part of the program committee for LXAI as well.
Affinity Workshop: Women in Machine Learning (WiML) Un-Workshop Mon 18 Jul 08:30 a.m.
Since ICML 2020, WiML organizes so-called "un-workshops" at ICML. The un-workshop is based on the concept of an “un-conference”, a form of discussion on a pre-selected topic that is primarily driven by participants. Different from the workshop, the un-workshop’s main focus is the topical "breakout sessions"--where attendees are split into smaller groups, and 2-3 participants per each session---who are preselected prior to the event---lead each discussion on a predefined topic. In addition to the breakout sessions, the un-workshop will include short invited talks, casual informal poster presentations, and a mentoring session. The overall goal of the un-workshop is to advance research through collaboration and increased interaction among participants from diverse backgrounds. Students, postdocs, and researchers in all areas of Machine Learning who primarily identify as a woman and/or nonbinary are encouraged to submit a one-page proposal to lead a breakout session on a certain research topic, and/or to submit a short abstract for the poster session. While all presenters will identify primarily as a woman and/or nonbinary, all genders are invited to attend and participate in the discussions.
Affinity Workshop: New In Machine Learning (NewInML) Mon 18 Jul 09:00 a.m.
Following the previous NewInML workshop editions, our goal is to welcome newcomers in the community and provide them with some guidance to contribute to Machine Learning research fully and effectively.
We intend to host presentations appealing to ICML audiences eager to learn how to conduct their research from experienced researchers. The following topics will be addressed:
- Communicating your results
- Collaborations with ML researchers
- Coding best practices
Attendees who submit an extended abstract will get peer-reviewed and best selected abstracts will be presented by their authors. This year, we are also hosting a presentation on academic writing support tailored on the submissions. It is your opportunity to refine your academic writing skills and boost your chances to see your paper published at the next ICML conference.
Tutorial: Stefan Riezler · Michael Hagmann
Scientific progress in machine learning is driven by empirical studies that evaluate the relative quality of models. The goal of such an evaluation is to compare machine learning methods themselves, not to reproduce single test-set evaluations of particular optimized instances of trained models. The practice of reporting performance scores of single best models is particularly inadequate for deep learning because of a strong dependence of their performance on various sources of randomness. Such an evaluation practice raises methodological questions of whether a model predicts what it purports to predict(validity), whether a model’s performance is consistent across replications of the training process (reliability), and whether a performance difference between two models is due to chance (significance). The goal oft his tutorial is to provide answers to these questions by concrete statistical tests. The tutorial is hands-on and accompanied by a textbook (Riezler and Hagmann,2021) and a webpage including R and Python code: https://www.cl.uni-heidelberg.de/statnlpgroup/empirical_methods/
Bio s:Tutorial: Chuan Guo · Reza Shokri
Quantitative Reasoning About Data Privacy in Machine Learning
Machine learning algorithms leak a significant amount of information about their training data. A legitimate user of a model can reconstruct sensitive information about the training data, by having access to its predictions or parameters. Given that all privacy policies and regulations require privacy auditing of (machine learning) algorithms, we are interested in a generic approach to perform quantitative reasoning about the privacy risks of various machine learning algorithms. Differentially private machine learning is currently the most widely accepted framework for privacy-preserving machine learning on sensitive data. The framework prescribes a rigorous accounting of information leakage about the training data through the learning algorithm using statistical divergences. However, it is often difficult to interpret this mathematical guarantee in terms of how a randomized algorithm limits how much an adversary can infer about one's data. For example, if a model is trained on my private emails containing personal information such as credit card number, does DP epsilon = 10 prevent my credit card number from being leaked by the model? If I am a patient participating in a personalized cancer treatment prediction study, does DP epsilon = 5 prevent others from identifying my membership (and hence my cancer positivity) in this study? In this tutorial, we present a unified view of recent works that translate privacy bounds to practical inference attacks and provide a rigorous quantitative understanding of DP machine learning. The objective is to link the underlying relation between privacy concepts, inference attacks, protection mechanisms, and tools, and to make the whole field more understandable to ML researchers and engineers.
Bio s:Tutorial: Nan Rosemary Ke · Stefan Bauer
Causality and Deep Learning: Synergies, Challenges and the Future
Deep neural networks have achieved outstanding success in many tasks ranging from computer vision, to natural language processing, and robotics. However such models are still pale in their ability to understand the world around us, as well as generalizing and adapting to new tasks or environments. One possible solution to this problem are models that comprehend causality, since such models can reason about the connections between causal variables and the effect of intervening on them. However, existing causal algorithms are typically not scalable nor applicable to highly nonlinear settings, and they also assume that the causal variables are meaningful and given. Recently, there has been an increased interest and research activity at the intersection of causality and deep learning in order to tackle the above challenges, which use deep learning for the benefit of causal algorithms and vice versa. This tutorial is aimed at introducing the fundamental concepts of causality and deep learning for both audiences, providing an overview of recent works, as well as present synergies, challenges and opportunities for research in both fields.
Bio s:Social: Oxford Wom*n in Computer Science: Highlighting Wom*n Researchers in ML Mon 18 Jul 11:00 a.m.
"Our session aims to highlight the distinguished work of several Wom*n Researchers in machine learning. From ML for Space, ML for healthcare, interpretability, climate change, and more, we’ll have you covered with this exciting research and discuss about further possibilities along with established researchers in the field."
Tutorial: Priya Donti · David Rolnick · Lynn Kaack
Climate Change and Machine Learning: Opportunities, Challenges, and Considerations
Climate change is one of the greatest challenges that society faces today, requiring rapid action from across society. In this tutorial, we will provide an introduction to climate change, what it means to address it, and how machine learning can play a role. From energy to agriculture to disaster response, we will describe high-impact problems where machine learning can help, e.g., by providing decision-relevant information, optimizing complex systems, and accelerating scientific experimentation. These problems encompass exciting opportunities for both methodological innovation and on-the-ground implementation. We will also describe avenues for machine learning researchers and practitioners to get involved, alongside key considerations for the responsible development and deployment of such work. While this tutorial will primarily discuss opportunities for machine learning to help address climate change, it is worth noting that machine learning is a general-purpose technology that can be used for applications that both help and hinder climate action. In addition, machine learning has its own computational and hardware footprint. We will therefore briefly present a framework for understanding and contextualizing machine learning’s overall climate impacts, and describe associated considerations for machine learning research and practice as a whole. Through the course of this tutorial, we hope that participants will gain a deeper understanding of how climate change and machine learning intersect, as well as how they can get involved by using their skills to help address the climate crisis.
Bio s:Tutorial: Dylan Foster · Alexander Rakhlin
Bridging Learning and Decision Making
This tutorial will give an overview of the theoretical foundations of interactive decision making (high-dimensional/contextual bandits, reinforcement learning, and beyond), a promising paradigm for developing AI systems capable of intelligently exploring unknown environments. The tutorial will focus on connections and parallels between supervised learning/estimation and decision making, and will build on recent research which provides (i) sample complexity measures for interactive decision making that are necessary and sufficient for sample-efficient learning, and (ii) unified algorithm design principles that achieve optimal sample complexity. Using this unified approach as a foundation, the main aim of the tutorial will be to give a bird’s-eye view of the statistical landscape of reinforcement learning (e.g., what modeling assumptions lead to sample-efficient algorithms). Topics covered will range from basic challenges and solutions (exploration in tabular RL, policy gradient methods, contextual bandits) to the current frontier of understanding. We will also highlight practical algorithms.
Bio s:Tutorial: Dorsa Sadigh · Anca Dragan
Learning for Interactive Agents
One of the key challenges in developing intelligent and autonomous learning agents is their ability to effectively interact with humans. In this tutorial, we plan to cover the theoretical and practical foundations of interactive agents. Specifically, in the first part of the tutorial, we will focus on models of human behavior in isolation, how these models can be used for effective coordination and how they can be optimized for influencing the partner. In the second part of the tutorial, we will continue by introducing co-adaptation settings, where the human preferences are non-stationary and they adapt, and we will discuss how this leads to emergence of new norms, conventions, and equilibria. Finally, we will wrap up by introducing approaches for inferring human partner preferences using a range of offline and online sources of data present in interactive domains. Throughout this tutorial, we will also go over concrete examples from applications in autonomous driving, mixed-autonomy traffic network, personal robotics, and multi-agent games.
Bio s:
Social: How to Negotiate Industry Offers in AI Mon 18 Jul 01:00 p.m.
"Webinar and Q&A on How to Negotiate Industry Offers in AI. Some of the topics we discuss are: * What the standard recruiting process looks like * How to choose the best job offer for career growth * When/how you should negotiate * When should you walk away from a job offer * When can an offer be rescinded from negotiating"
Social: It’s all about Transformers! Mon 18 Jul 02:00 p.m.
"Did you read about LaMDA, what do you think about the claims that it has become sentient! Will the transformer models save the world just that Transformer Optimus Prime did? What are your opinions about the BLOOM model? Seems like the era of NLP is here with these super successful large language models breaking all benchmark scores that the big tech companies are producing, some even in association with other big techs! Long live this collaboration! In this social, we will aim to look at the rapid rise, the progress of NLP models, the tasks Transformers had made the greatest impact, and the road ahead! This will be an informal session where I will try to walk through a few examples from different industries like adtech/fintech and use-cases and would appreciate an open discussion on other use-cases from the participants!"
Tutorial: Anna Korba · Adil Salim
Sampling as First-Order Optimization over a space of probability measures
Sampling from a target probability distribution whose density is only known up to a normalisation constant is a fundamental problem in statistics and machine learning. While the literature on optimization for machine learning has developed widely in the past decade, with fine convergence rates for some methods, the literature on sampling remained mainly asymptotic until very recently. Since then, the Machine Learning community has been increasingly interested in the non asymptotic analysis of sampling algorithms, or in designing new schemes to improve the complexity of sampling. Interestingly, approximating a target probability distribution can be cast as an optimization problem where the objective functional measures the dissimilarity to the target distribution. In particular, the Kullback-Leibler divergence (or relative entropy) with respect to the target distribution is a suitable objective functional when the normalisation constant is intractable, as it is commonly the case in Bayesian inference. This optimization problem can be addressed using optimization techniques over a space of probability measures. The theory of Wasserstein gradient flows provides tools to solve this optimization problem. Indeed, Wasserstein gradient flows are continuous paths of distributions that decrease the objective functional. Moreover, several sampling algorithms such as Langevin Monte Carlo or Stein Variational Gradient Descent can be seen as discretisations of a Wasserstein gradient flow. In this tutorial, we will show how one can leverage optimization techniques to design and analyze sampling algorithms. We will first review fundamental optimization concepts such as Euclidean gradient flows (i.e., continuous time gradient descent), before introducing their optimal transport analogue such as Wasserstein gradient flows. Then, we will present an optimization point of view on both standard and novel sampling algorithms, and how this point of view has led to new convergence results and the design of new schemes.
Bio s:Tutorial: Hao Zhang · Lianmin Zheng · Zhuohan Li · Ion Stoica
Welcome to the "Big Model" Era: Techniques and Systems to Train and Serve Bigger Models
In recent years, researchers in ML and systems have been working together to bring big models -- such as GPT-3 with 175B parameters -- into research and production. It has been revealed that increasing model sizes can significantly boost ML performance, and even lead to fundamentally new capabilities.
However, experimenting and adopting big models call for new techniques and systems to support their training and inference on big data and large clusters. This tutorial identifies research and practical pain points in model-parallel training and serving. In particular, this tutorial introduces new algorithmic techniques and system architectures for addressing the training and serving of popular big models, such as GPT-3, PaLM, and vision transformers. The tutorial also consists of a session on how to use the latest open-source system toolsets to support the training and serving of big models. Through this tutorial, we hope to lower the technical barrier of using big models in ML research and bring the big models to the masses.
Tutorial: Elias Bareinboim · Drago Plecko
AI plays an increasingly prominent role in modern society since decisions that were once made by humans are now delegated to automated systems. These systems are currently in charge of deciding bank loans, criminals' incarceration, and the hiring of new employees, and it is not hard to envision that soon they will underpin most of the society's decision infrastructure. Despite the high stakes entailed by this task, there is still a lack of formal understanding of some basic properties of such systems, including issues of fairness, accountability, and transparency. In this tutorial, we introduce a framework of causal fairness analysis, with the intent of filling in this gap, i.e., understanding, modelling, and possibly solving issues of fairness in decision-making settings. The main insight of our approach will be to link the quantification of the disparities present in the observed data with the underlying, and often unobserved causal mechanisms that generate the disparity in the first place. We will study the problem of decomposing variations, which results in the construction of empirical measures of fairness that attribute such variations to causal mechanisms that generated them. Such attribution of disparity to specific causal mechanisms will allow us to propose a formal and practical framework for assessing legal doctrines of disparate treatment and impact, allowing also for considerations of business necessity. Finally, through the newly developed framework we will draw important connections with previous literature, both in and outside the causal inference arena. This effort will culminate in the "Fairness Map", which is the first cohesive and systematic classification device of multiple measures of fairness in terms of their causal properties, including admissibility, decomposability, and power. We hope this new set of principles, measures, and tools can help guide AI researchers and engineers when analyzing and/or developing decision-making systems that will be aligned with society's goals, expectations, and aspirations.
Bio s:
Social: Quantum ML Mon 18 Jul 07:00 p.m.
An opportunity to meet others in the community who work on quantum machine learning and/or quantum-inspired machine learning. It will be very informal, with a few pointers placed on the tables to help start discussions.
Social: Designing an RL system toward AGI Mon 18 Jul 07:00 p.m.
An artificial general intelligence (AGI) agent is capable of achieving general goals. An agent that reasons about generality is complicated. The world the AGI is interacting with, however, is much more complicated than the agent itself. Further, the agent only observes a part of the world at a time and thus needs to construct its own summary of the past and the summary is the agent’s subjective state. All components that the agent has, except the one that generates the agent’s state, take the agent’s state as input and generate desired outputs. What components the agent should maintain and how the specific components interact with each other are two fundamental questions. Specific questions arise from these two fundamental questions. For example, what are good agent states and what are bad ones? What should the world model take and produce? Are sub-tasks necessary? What sub-tasks are good and what are bad? These questions are about designing architecture and identifying the purposes of each component in the architecture, rather than specific ways to implement each component. Our social welcomes everyone who is interested in brainstorming such an architecture design.