PaddlePaddle (PArallel Distributed Deep LEarning) is an easy-to-use, efficient, flexible, and scalable deep learning platform, which was originally developed by Baidu scientists and engineers for the purpose of applying deep learning to many products at Baidu such as Computer Vision (CV), NLP, and Speech. PaddlePaddle supports various neural network architectures and optimization algorithms. With PaddlePaddle, it is possible to leverage many CPUs/GPUs and machines to speed up training, achieving high throughput and performance via optimized communication. In this workshop, Baidu scientists and engineers will present a wide range of PaddlePaddle-based research and projects, from CV, NLP, graph learning, federated learning, few shot learning, to quantum computing.
Federated learning is a recent and rapidly expanding area of machine learning that allows parties to benefit from joint training of models whilst respecting the privacy of each party's data. IBM Research has a broad effort in federated learning comprising novel methods, models and paradigms and offers an enterprise-strength federated learning platform free to use for non-commercial purposes, the IBM Federated Learning Community Edition.
The session will give an overview through a series of 7 short talks on the most exciting new research results from IBM Research in federated learning. Questions shall be collected using the Chat window and addressed after the lightning talks as well as after the live-interactive demo.
There has been a discrepancy between academic research and industrial applications. Academic research weighs more on developing new models, but industrial application weighs more on the data. Open data such as ImageNet, KITTI, and MNIST has been at the core of AI research in the last several decades. With the rise of open data, more researchers began to realize the importance of data in AI development. Industry expert Andrew Ng and many other developers are advocating for the transition from Model-centric AI to Data-centric AI development.
In this talk, we will discuss the rationale of Data-centric AI development from an academic perspective and explain some of the ways to improve data quality. We will also talk about some current pain points of open data and introduce Graviti Open Dataset --- our solution to these problems by showcasing a demo on its usage.
Entity matching in Master Data Management (MDM) is the task of determining if two entities represent the same real world entity. Entities are typically people, organizations, locations, and events represented as attributed nodes in a graph, though they can also be represented as relational data. While artificial neural network models and probabilistic matching engines exist for this task, explaining entity matching has received less attention. In this presentation, we describe three entity matching scenarios in the real world and present explainability solutions for them.
Talk on AI for biodiversity and closing the gap between academic research and real-world impact. Nontraditional paths to research and interdisciplinary education. Register for the socials here.
In this tutorial, we will present a brief overview of responsible AI, highlighting model explainability, fairness, and privacy in AI, key regulations/laws, and techniques/tools for providing understanding around web-based AI/ML systems. Then, we will focus on the application of explainability, fairness assessment/unfairness mitigation, and privacy techniques in industry, wherein we present practical challenges/guidelines for using such techniques effectively and lessons learned from deploying models for several web-scale machine learning and data mining applications. We will present case studies across different companies, spanning application domains such as search and recommendation systems, hiring, sales, lending, and fraud detection. We will emphasize that topics related to responsible AI are socio-technical, that is, they are topics at the intersection of society and technology. The underlying challenges cannot be addressed by technologists alone; we need to work together with all key stakeholders — such as customers of a technology, those impacted by a technology, and people with background in ethics and related disciplines — and take their inputs into account while designing these systems. Finally, based on our experiences in industry, we will identify open problems and research directions for the machine learning community.Bios:
In this tutorial, we will present the emerging direction of explainability that we will refer to as Natural-XAI. Natural-XAI aims to build AI models that (1) learn from natural language explanations for the ground-truth labels at training time, and (2) provide such explanations for their predictions at deployment time. For example, a self-driving car would not only see at training time that it has to stop in a certain environment, but it would additionally be told why this is the case, e.g., “Because the traffic light in front is red.”. At usage time, the self-driving car would also be able to provide such natural language explanations for its actions, thus reassuring the passengers. This direction has recently received increasingly large attention.Bios:
This tutorial will perform an detailed overview of the work on sparsity in deep learning, covering sparsifi- cation techniques for neural networks, from both the mathematical and implementation perspectives. We specifically aim to cover the significant recent advances in the area, and put them in the context of the foundational work performed on this topic in the 1990s.Bios:
Scientists in the field of machine learning (ML) – including deep learning (DL) -- aspire to build better models (usually judged by beating SOTA in well-defined tasks and datasets); successful applications of such models, on the other hand, are about product-market fit (PMF) in environments with ever-growing complexities. As many expect ML to play a bigger role in our society, ML scientists’ ability to influence this journey will depend on putting ML research in a PMF context and vice versa (i.e., optimising for market.fit(model.fit())+⍺*model.fit(market.fit()) instead of optimising for model.fit() alone). Therefore, in this tutorial we aim to cover the general principals of building AI products in the “real world”, covering topics such as product design/management, achieving product-market fit, and ML R&D in this context.
All times are EST
Session 1 (11:00 a.m. - 11:15 a.m): Overview of tutorial and the core idea (R. Khorshidi)
Session 2 (11:15 a.m. - 11:45 a.m): Product Market Fit (R. Khorshidi)
- Break (11:45 a.m. - 12:00 p.m)
Session 3 (12:15 p.m. - 12:30 p.m): Build Measure Learn (R. Khorshidi)
Session 4 (12:30 p.m. - 1:00 p.m): Experiments and Metrics (R. Khorshidi)
- Break (1:00 p.m. - 1:15 p.m)
Session 5 (1:15 p.m. - 2:00 p.m): Examples (P. Faratin)
Q&A (2:00 p.m. - 2:15 p.m)
Humans have the extraordinary ability to learn continually from experience. Not only we can apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of Artificial Intelligence (AI) is building an artificial “continual learning” agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex knowledge and skills (Parisi, 2019).
However, despite early speculations and few pioneering works (Ring, 1998; Thrun, 1998; Carlson, 2010), very little research and effort has been devoted to address this vision. Current AI systems greatly suffer from the exposure to new data or environments which even slightly differ from the ones for which they have been trained for (Goodfellow, 2013). Moreover, the learning process is usually constrained on fixed datasets within narrow and isolated tasks which may hardly lead to the emergence of more complex and autonomous intelligent behaviors. In essence, continual learning and adaptation capabilities, while more than often thought as fundamental pillars of every intelligent agent, have been mostly left out of the main AI research focus.
In this tutorial, we propose to summarize the application of these ideas in light of the more recent advances in machine learning research and in the context of deep architectures for AI (Lomonaco, 2019). Starting from a motivation and a brief history, we link recent Continual Learning advances to previous research endeavours on related topics and we summarize the state-of-the-art in terms of major approaches, benchmarks and key results.
In the second part of the tutorial we plan to cover more exploratory studies about Continual Learning with low supervised signals and the relationships with other paradigms such as Unsupervised, Semi-Supervised and Reinforcement Learning. We will also highlight the impact of recent Neuroscience discoveries in the design of original continual learning algorithms as well as their deployment in real-world applications.
Finally, we will underline the notion of continual learning as a key technological enabler for Sustainable Machine Learning and its societal impact, as well as recap interesting research questions and directions worth addressing in the future.
In this tutorial we provide an overview of state-of-the-art techniques for synthesizing the two most common types of clinical data; namely tabular (or multidimensional) data and time-series data. In particular we discuss various generative modeling approaches based on generative adversarial networks (GANs) normalizing flows and state-space models for cross-sectional and time-series data demonstrating the use cases of such models in creating synthetic training data for machine learning algorithms and highlighting the comparative strengths and weaknesses of these different approaches. In addition we discuss the issue of evaluating the quality of synthetic data and the performance of generative models; we highlight the challenges associated with evaluating generative models as compared to discriminative predictions and present various metrics that can be used to quantify different aspects of synthetic data quality.Bios:
This tutorial will address the wider social and economic implications of large language models, such as ELMO (Peters et al., 2018), BERT (Devlin et al., 2019), GPT-2 and -3 (Radford et al., 2019; Brown et al., 2020), FlauBERT (Le et al., 2020), XLNet (Yang et al., 2019), CPM (Zhang et al., 2020), PALM (Bi et al., 2020), Switch C (Fedus et al., 2021) and others. Over the past few years the resources put into developing bigger language models trained on more data has been unparalleled. And yet, the full repercussions of this record concentration of resources has been little discussed. In this tutorial, we aim to address concerns around the economic, political, social, and legal impacts of the development of large language models.
Our tutorial includes guest presentations by:
Su Lin Blodgett
Thanks to these five scholars for providing their expertise!