A social with keynote speakers, and a panel discussion. The main objective would be to learn more about African Languages, related ongoing research, and initiatives.

Registration Check-in Desk closing at 7 pm. Badge pickup.

In recent years, breakthroughs in sample-efficient RL algorithms like Contextual Bandits enabled new solutions to personalization and optimization scenarios. Unbiased off-policy evaluation gave Data Scientists superpowers on real-world data volumes, giving them confidence in putting machine learning into production. Vowpal Wabbit is an open source machine learning toolkit and research platform, used extensively across the industry, providing fast, scalable machine learning.

Personalizer, an Azure Cognitive Service, powered by Vowpal Wabbit, aims to democratize real world reinforcement learning for content personalization.

Come and learn about reinforcement learning, the theory behind contextual bandits and how this applies to content personalization:

- Quick introduction into Vowpal Wabbit and contextual bandits learning
- How to succeed using Vowpal Wabbit
- What is new in Vowpal Wabbit 9
- Keeping up with Azure Personalizer

Graph Neural Networks (GNNs) are a tantalizing way of modeling data which doesn't have a fixed structure. However, getting them to work as expected has had some twists and turns over the years.

This talk will have four components.

1. First, we'll briefly describe the Graph Mining team at Google.
2. Next, we'll focus on the Graph Mining team's work to make GNNs useful. We'll focus on challenges that we've identified and the solutions we've developed for them. Specifically, we'll highlight work that's led to more expressive graph convolutions, more robust models, and better graph structure. In addition, we'll highlight some new features available in our open source library for GNNs in TensorFLow, TF-GNN.
3. Second, we'll talk about other advances in Graph Mining from the group (including clustering, graph building, and privacy).
4. Finally, we'll have a tutorial section with quick demos covering TF-GNN, synthetic evaluation of GNNs with GraphWorld, and benchmarking for clustering tasks.

Model customization unlocks several advantages in wearable devices, including better memorability, higher interaction efficiency, and enhanced accessibility for people with special needs. We present a framework for gesture customization requiring minimal examples from users, all without degrading the performance of existing gesture sets. To achieve this, we first deployed a large-scale study (N=500+) to collect data and train an accelerometer-gyroscope recognition model with a cross-user accuracy of 95.7% and a false-positive rate of 0.6 per hour when tested on everyday non-gesture data. Next, we design a few-shot learning framework which derives a lightweight model from our pre-trained model, enabling knowledge transfer without performance degradation. We validate our approach through a series of user studies, examining on-device customization from several new gestures, resulting in an average accuracy of 55.3%, 83.1%, and 87.2% on using one, three, or five shots when adding a new gesture, while maintaining the same recognition accuracy and false-positive rate from the pre-existing gesture set. Our results highlight the effectiveness, learnability, and usability of our customization framework. Our approach paves the way for a future where users are no longer bound to pre-existing gestures, freeing them to creatively introduce new gestures tailored to their preferences and abilities.

AEPsych is a platform for modeling and active learning targeting live, human-in-the-loop experiments related to human perception and preferences. It is built on state-of-the-art tools including PyTorch, GPyTorch and BoTorch, and provides both a platform for new algorithm development and benchmarking (e.g. Letham et al, AISTATS 2022) and the deployment of models and algorithms into real user studies (e.g. Wu et al. SID Display Week 2022; Guan et al., SIGGRAPH to appear). Our demonstration will showcase a suite of interactive preference-based tuning and psychometric field estimation demos, including in VR.

We present the alpha version of TorchRL, the Reinforcement-Learning dedicated PyTorch domain library.

TorchRL is an open-source Reinforcement Learning (RL) library for PyTorch.

It provides pytorch and python-first, low and high level abstractions for RL that are intended to be efficient, documented and properly tested. The code is aimed at supporting research in RL. Most of it is written in python in a highly modular way, such that researchers can easily swap components, transform them or write new ones with little effort.

TorchRL provide low-level primitives to efficiently collect data across a wide range of libraries and efficiently train algorithms on these data. We provide data-carrying structures that make it easy to write efficient codes in parallel and distributed settings.

We will present a few examples of basic usage of the library and some results on specific tasks (robotics, games and others).

Resistance to EGFR inhibitors (EGFRi) presents a major obstacle in treating non-small cell lung cancer (NSCLC). One of the most exciting new ways to find potential resistance markers involves running functional genetic screens, such as CRISPR, followed by manual triage of significantly enriched genes. This triage process to identify ‘high value’ hits resulting from the CRISPR screen involves manual curation that requires specialized knowledge and can take even experts several months to comprehensively complete. To find key drivers of resistance faster we build a recommendation system on top of a heterogeneous biomedical knowledge graph integrating pre-clinical, clinical, and literature evidence. The recommender system ranks genes based on trade-offs between diverse types of evidence linking them to potential mechanisms of EGFRi resistance. This unbiased approach identifies 57 resistance markers from >3,000 genes, reducing hit identification time from months to minutes. In addition to reproducing known resistance markers, our method identifies previously unexplored resistance mechanisms that we prospectively validate.

AstraZeneca leverages Weights & Biases as a core technology in their Machine Learning Engineering Foundation platform which enables their scientists and engineers to develop reproducible ML models faster, including applications in computer vision, natural language processing, and audio.

In this live coding demo, you will learn how W&B can help track your experiments, review model performance and edge cases, select optimal hyperparameters and share your results easily with colleagues.

Teams that train the best models will win prizes!

Model customization unlocks several advantages in wearable devices, including better memorability, higher interaction efficiency, and enhanced accessibility for people with special needs. We present a framework for gesture customization requiring minimal examples from users, all without degrading the performance of existing gesture sets. To achieve this, we first deployed a large-scale study (N=500+) to collect data and train an accelerometer-gyroscope recognition model with a cross-user accuracy of 95.7% and a false-positive rate of 0.6 per hour when tested on everyday non-gesture data. Next, we design a few-shot learning framework which derives a lightweight model from our pre-trained model, enabling knowledge transfer without performance degradation. We validate our approach through a series of user studies, examining on-device customization from several new gestures, resulting in an average accuracy of 55.3%, 83.1%, and 87.2% on using one, three, or five shots when adding a new gesture, while maintaining the same recognition accuracy and false-positive rate from the pre-existing gesture set. Our results highlight the effectiveness, learnability, and usability of our customization framework. Our approach paves the way for a future where users are no longer bound to pre-existing gestures, freeing them to creatively introduce new gestures tailored to their preferences and abilities.

With deep learning models rapidly growing in size, training large models becomes one of the most important challenges in modern AI. We present Amazon SageMaker model parallelism (SMP), a software library that integrates with PyTorch and enables easy training of large models using model parallelism and memory-saving features. The implementation of the SMP library is generic and flexible, in that it can automatically partition arbitrary model architectures, is modular enough to be easily applied to new training scripts, and preserves the native PyTorch experience to a larger degree than existing solutions. In this talk, we will dive deep into the techniques used in SMP, which includes pipeline parallelism, tensor parallelism, activation checkpointing/offloading and optimizer state sharding. We will also demonstrate how to integrate SMP with a native Pytorch script.

Recent advances in machine learning have made significant progress in fields such as computer vision, speech recognition, and natural language processing. Machine learning is, however, yet to have a major impact on drug discovery and development. It is not surprising that the progress is slow because drug development is a complex problem, spanning target identification, lead discovery and optimization, pre-clinical studies, and clinical trials. In contrast to computer vision and speech recognition, machine learning experts also do not have a good understanding of the problem setting and there is generally a lack of well curated benchmark datasets that would drive the state-of-the-art across different stages. All of this leads to the underutilization of machine learning techniques in drug development. This panel brings together distinguished academics in machine learning and accomplished leaders in the pharmaceutical industry working across different stages of drug discovery and development pipelines to discuss the challenges and opportunities from their own perspectives. The aim of the panel is to draw a contrast between different views and discover potential misalignments; thus, identifying opportunities for collaboration and driving the research while optimizing for the impact on the drug discovery and development pipelines.

Are there going to be publicly available pharma-specific ImageNet/CIFAR/MNIST/TIMIT benchmark datasets? Are machine learning labs focusing on the problems that could make a difference in terms of costs, effectiveness, and intelligence augmentation? What challenges should be prioritized to drive the field forward and utilize modern machine learning tools and models? How do improve synergies between the two fields that could be mutually beneficial?

Our demo will show how AWS enables customers to automatically screen and monitor user-generated online content on their platforms. This demoed solution aims to help customers maintain a safe environment for their users by detecting and removing toxic and offensive content using machine learning. For example, gaming customers can use this solution to detect toxic comments in gamers' chat or audio stream.

Speakers’ names:
Xiang, Yi yxxan@amazon.com; Bhabesh, Sourav sbhabesh@amazon.coml ; Yanjun Qi (Jane) yanjunqi@amazon.com

In this talk, we propose a novel approach for learning robust acoustic models in adverse environments, characterized by a significant mismatch between training and test conditions. This problem is of paramount importance for the deployment of speech recognition systems that need to perform well in unseen environments. The proposed approach is an instance of vicinal risk minimization, which aims to improve risk estimates during training by replacing the delta functions that define the empirical density over the input spacen with an approximation of the marginal population density in the vicinity of the training samples. More specifically, we assume that local neighborhoods centered at training samples can be approximated using a mixture of Gaussians, and demonstrate theoretically that this can incorporate robust inductive bias into the learning process. We characterize the individual mixture components implicitly via data augmentation schemes, designed to address common sources of spurious correlations in acoustic models. To avoid potential confounding effects on robustness due to information loss, which has been associated with standard feature extraction techniques (e.g., FBANK and MFCC features), we focus our evaluation on the waveform-based setting. The empirical results show that the proposed approach can generalize to unseen noise conditions, with 150% relative improvement in out-of-distribution generalization compared to training using the standard risk minimization principle. Moreover, the results demonstrate competitive performance relative to models learned using a training sample designed to match the acoustic conditions characteristic of test utterances (i.e., optimal vicinal densities).

Please join us to celebrate the opening of ICML 2022!