Timezone: »
Many of the world's most pressing issues, such as climate change, pandemics, financial market stability and fake news, are emergent phenomena that result from the interaction between a large number of strategic or learning agents. Understanding these systems is thus a crucial frontier for scientific and technology development that has the potential to permanently improve the safety and living standards of humanity. Agent-Based Modelling (ABM) (also known as individual-based modelling) is an approach toward creating simulations of these types of complex systems by explicitly modelling the actions and interactions of the individual agents contained within. However, current methodologies for calibrating and validating ABMs rely on human expert domain knowledge and hand-coded behaviours for individual agents and environment dynamics. Recent progress in AI has the potential to offer exciting new approaches to learning, calibrating, validation, analysing and accelerating ABMs. This interdisciplinary workshop is meant to bring together practitioners and theorists to boost ABM method development in AI, and stimulate novel applications across disciplinary boundaries - making ICML the ideal venue.Our inaugural workshop will be organised along two axes. First, we seek to provide a venue where ABM researchers from a variety of domains can introduce AI researchers to their respective domain problems. To this end, we are inviting a number of high-profile speakers across various application domains. Second, we seek to stimulate research into AI methods that can scale to large-scale agent-based models with the potential to redefine our capabilities of creating, calibrating, and validating such models. These methods include, but are not limited to, simulation-based inference, multi-agent learning, causal inference and discovery, program synthesis, and the development of domain-specific languages and tools that allow for tight integration of ABMs and AI approaches.
Sat 5:30 a.m. - 5:40 a.m.
|
Introduction by the Organizers
(
Live intro
)
SlidesLive Video » |
Christian Schroeder de Witt 🔗 |
Sat 5:40 a.m. - 6:10 a.m.
|
What will be the ImageNet moment for ABMs?
(
Invited Talk
)
|
Stephan Zheng 🔗 |
Sat 6:10 a.m. - 6:40 a.m.
|
The intersection of simulation-based inference and agent based modelling
(
Invited Talk
)
SlidesLive Video » |
Kyle Cranmer 🔗 |
Sat 6:40 a.m. - 7:10 a.m.
|
Reservoir Computing for Predicting Complex Network Dynamics
(
Invited Talk
)
SlidesLive Video » |
Michelle Girvan 🔗 |
Sat 7:10 a.m. - 7:40 a.m.
|
Adding AI to Agent-Based Models – Applications in infectious disease epidemiology
(
Invited Talk
)
SlidesLive Video » |
Theresa Reiker 🔗 |
Sat 7:40 a.m. - 7:50 a.m.
|
Generating Diverse Cooperative Agents by Learning Incompatible Policies
(
Spotlight (Contributed)
)
SlidesLive Video » Effectively training a robust agent that can cooperate with unseen agents requires diverse training partner agents. Nonetheless, obtaining cooperative agents with diverse behaviors is a challenging task. Previous work proposes learning a diverse set of agents by diversifying the state-action distribution of the agents. However, without information about the task's goal, the diversified behaviors are not motivated to find other important, albeit non-optimal, solutions, resulting in only local variations of a solution. In this work, we propose to learn diverse behaviors by looking at policy compatibility while using state-action information to induce local variations of behaviors. Conceptually, policy compatibility measures whether policies of interest can collectively solve a task. We posit that incompatible policies can be behaviorally different. Based on this idea, we propose a novel objective to learn diverse behaviors. We theoretically show that our novel objective can generate a unique policy, which we incorporate into a population-based training scheme. Empirically, the proposed method outperforms the baselines in terms of the number of discovered solutions given the same number of agents. |
Rujikorn Charakorn 🔗 |
Sat 7:50 a.m. - 8:00 a.m.
|
High Performance Simulation for Scalable Multi-Agent Reinforcement Learning
(
Spotlight (Contributed)
)
SlidesLive Video » Multi-agent reinforcement learning experiments and open-source training environments are typically limited in scale, supporting tens or sometimes up to hundreds of interacting agents. In this paper we demonstrate the use of Vogue, a high performance agent based modelling framework. Vogue serves as a multi-agent training environment, supporting thousands to tens of thousands of interacting agents while maintaining high training throughput by running both the environment and reinforcement learning agents on the GPU. High performance multi-agent environments at this scale have the potential to enable the learning of robust and flexible policies for use in agent based models and simulations of complex systems. We demonstrate training performance with two newly developed, large scale multi-agent training environments. Moreover, we show that these environments can train shared reinforcement learning policies on time-scales of minutes and hours. |
Jordan Langham-Lopez 🔗 |
Sat 8:00 a.m. - 8:20 a.m.
|
Evology: an Empirically-Calibrated Market Ecology Agent-Based Model for Trading Strategy Search
(
Oral (Contributed)
)
SlidesLive Video » Market ecology views financial markets as ecosystems of diverse, interacting and evolving trading strategies. We present a heterogeneous, empirically calibrated multi-agent market ecology agent-based model. We outline its potential as a valuable and challenging training ground for optimising trading strategies using machine learning algorithms and defining research tasks. |
Aymeric Vie 🔗 |
Sat 8:20 a.m. - 8:40 a.m.
|
Exploring social theory integration in agent-based modelling using multi-objective grammatical evolution
(
Oral (Contributed)
)
SlidesLive Video » In Generative Social Science, modellers design agents at the micro-level to generate macro-level a target social phenomenon. In the Inverse Generative Social Science (iGSS), from a target phenomenon, the goal is to search for possible explanatory model structures. This model discovery process is a promising tool to improve the explanatory capability and theory exploration of computational social science. This paper presents a framework for iGSS and applies Grammatical Evolution to an empirically-calibrated agent-based model of alcohol use. Results of the model discovery process find many alternative rules for agent behaviours with different trade-offs. Future work should involve domain experts to evaluate the discovered structures in terms of theoretical credibility and knowledge contribution. |
Tuong Manh Vu 🔗 |
Sat 8:40 a.m. - 9:00 a.m.
|
Differentiable agent-based epidemiological modeling for end-to-end learning
(
Oral (Contributed)
)
SlidesLive Video » Mechanistic simulators are an indispensable tool for epidemiology to explore the behavior of complex, dynamic infections under varying conditions and navigate uncertain environments. ODE-based models are the dominant paradigm that enable fast simulations and are tractable to gradient-based optimization, but make simplifying assumptions about population homogeneity. Agent-based models (ABMs) are an increasingly popular alternative paradigm that can represent the heterogeneity of contact interactions with granular detail and agency of individual behavior. However, conventional ABM frameworks are not differentiable and present challenges in scalability; due to which it is non-trivial to connect them to auxiliary data sources easily. In this paper we introduce GradABM which is a new scalable, fast and differentiable design for epidemiological ABMs. GradABM runs simulations with million-size populations in few seconds on commodity hardware and enables fast forward and differentiable inverse simulations. This makes it amenable to be merged with deep neural networks and seamlessly integrate heterogeneous data sources to help with calibration, forecasting and policy evaluation. We demonstrate the efficacy of GradABM via extensive experiments with real COVID-19 and influenza datasets. We are optimistic that the current work will bring ABM and AI communities closer together. |
Ayush Chopra 🔗 |
Sat 9:00 a.m. - 10:30 a.m.
|
Poster Session and Lunch Break
(
Poster Session
)
|
🔗 |
Sat 10:30 a.m. - 11:00 a.m.
|
Estimating Policy Functions in Payments Systems Using Reinforcement Learning
(
Invited Talk
)
SlidesLive Video » High-value payments systems (HVPSs) are used to settle transactions between large financial institutions and are considered the core national financial infrastructure. In collaboration with the Bank of Canada, we have been exploring the use of reinforcement learning techniques to understand the behaviour of banks participating in the Canadian HVPS. This understanding could help regulators design policies to ensure the safety and efficiency of these systems. |
Pablo Samuel Castro 🔗 |
Sat 11:00 a.m. - 11:30 a.m.
|
Latent state estimation for agent-based models using data assimilation
(
Invited Talk
)
SlidesLive Video » Scientists have recognized the need to build bottom-up models for socio-economic systems. Such models are often framed as heterogeneous agents interacting in a network following the rules of a dynamical system. However, the available data is often aggregated and incomplete, so even if we have a good model of reality, inferring the state of the individual agents remains a big open challenge. Moreover, these models are usually costly to simulate because one has to compute the individual interactions of all the agents in the system. We present a methodology to infer the latent states of agents embedded in a network when the data available is sparse, noisy, and low-dimensional. The methodology is based on the ensemble Kalman filter extended with a network localization technique that uses the system’s topology to improve the accuracy of the estimations. Our methodology has the following desired properties for bottom-up socioeconomic models: i) it treats the model as a black box, so it does not assume any closed-form mathematical form of the model a priori, ii) it requires a minimal number of simulations compared to state-of-the-art methods, iii) it exploits the underlying topology of the system to improve its predictions, iv) it works for nonlinear systems, v) it is well-justified from a Bayesian perspective, and vi) it is easy to implement. We validate our methodology in two informative examples: 1) a high-dimensional approximation of the Mackey-Glass chaotic system and 2) the Hegselmann-Krause bounded confidence (nonlinear) model of opinion dynamics embedded in a social network. While we do not use real-world data to showcase our methodology, we add noise and exogenous shocks to the observations, obtaining accurate predictions in both the observation and the latent state spaces. We aim to help bridge the gap between bottom-up modeling and data assimilation techniques in a computationally efficient way. |
Blas Kolic 🔗 |
Sat 11:30 a.m. - 12:00 p.m.
|
Physics-infused learning with ABM
(
Invited Talk
)
Research at the intersection of physics and machine learning has outsized potential to advance both fields: models and algorithms can be embedded with, or informed by, physics knowledge, and learned models as well as simulators of complex processes and systems can advance experimentation towards understanding. In practice, this physics-ML intersection is almost entirely ML surrogates for accelerating an existing numerical simulator. But can we look to ML & AI to discover physics knowledge we don't yet have governing equations for, to recover missing physics and fill gaps in human-expert understanding? It is this perspective we explore in this talk, in particular the use of agent-based modeling (ABM) as a new abstraction for computational fluid dynamics (CFD), pulling in advanced reinforcement learning (RL) methods from the AI field. We introduce multi-agent RL as an automated discovery tool of turbulence and other fluid dynamics models, leveraging the emergent phenomena of ABM to surface the unresolved subgrid-scale physics. These methods, although nascent, can significantly advance prediction and control of industrial aerodynamics and environmental flows in critical areas like nuclear fusion plasma and atmospheric transport of contaminants. |
Alexander Lavin 🔗 |
Sat 12:00 p.m. - 12:10 p.m.
|
Calibrating Agent-based Models to Microdata with Graph Neural Networks
(
Spotlight (Contributed)
)
SlidesLive Video » Calibrating agent-based models to data is among the most fundamental requirements to ensure the model fulfils its desired purpose. In recent years, simulation-based inference methods have emerged as powerful tools for performing this task when the model likelihood function is intractable, as is often the case for agent-based models. In some real-world use cases of agent-based models, both the observed data and the agent-based model output consist of the agents' states and their interactions over time. In such cases, there is a tension between the desire to make full use of the rich information content of such granular data on the one hand, and the need to reduce the dimensionality of the data to prevent difficulties associated with high-dimensional learning tasks on the other. A possible resolution is to construct lower-dimensional time-series through the use of summary statistics describing the macrostate of the system at each time point. However, a poor choice of summary statistics can result in an unacceptable loss of information from the original dataset, dramatically reducing the quality of the resulting calibration. In this work, we instead propose to learn parameter posteriors associated with granular microdata directly using temporal graph neural networks. We will demonstrate that such an approach offers highly compelling inductive biases for Bayesian inference using the raw agent-based model microstates as output. |
Joel Dyer 🔗 |
Sat 12:10 p.m. - 12:20 p.m.
|
The StarCraft Multi-Agent Challenges+ : Learning of Sub-tasks and Environmental Benefits without Precise Reward Functions
(
Spotlight (Contributed)
)
SlidesLive Video » Coming Soon |
Mingyu Kim 🔗 |
Sat 12:20 p.m. - 12:40 p.m.
|
Estimating the Impact of Coordinated Inauthentic Behavior on Content Recommendations in Social Networks
(
Oral (Contributed)
)
SlidesLive Video » Online disinformation is a dynamic and pervasive problem on social networks as evidenced by a spate of public disasters in light of active efforts to combat it. Since the massive amounts of content generated each day on these platforms is impossible to manually curate, ranking and recommendation algorithms are a key apparatus that drive user interactions. However, the vulnerability of ranking and recommendation algorithms to attack from coordinated campaigns spreading misleading information has been established both theoretically and anecdotally. Unfortunately it is unclear how effective countermeasures to disinformation are in practice due to the limited view we have into the operation of such platforms. We develop a multiagent simulation of a popular social network, Reddit, that aligns with the state-action space available to real users based on the platform's affordances. We collect millions of real-world interactions from Reddit to estimate the network for each user in our dataset and utilise Reddit's self-described content ranking strategies to compare the impact of coordinated activity on content spread by each algorithm. We expect that this will inform the design of robust content distribution systems that are resilient against targeted attacks by groups of malicious actors. |
Swapneel Mehta 🔗 |
Sat 12:40 p.m. - 1:00 p.m.
|
Coffee Break
|
🔗 |
Sat 1:00 p.m. - 2:00 p.m.
|
Panel Discussion
SlidesLive Video » |
Christian Schroeder de Witt 🔗 |
Sat 2:00 p.m. - 2:30 p.m.
|
Award Ceremony & Closing Remarks
SlidesLive Video » |
Christian Schroeder de Witt 🔗 |
-
|
Low-Loss Subspace Compression for Clean Gains against Multi-Agent Backdoor Attacks
(
Poster (Contributed)
)
SlidesLive Video » Recent exploration of the multi-agent backdoor attack demonstrated the backfiring effect, a natural defense against backdoor attacks where backdoored inputs are randomly classified. This yields a side-effect of low accuracy w.r.t. clean labels, which motivates this paper's work on the construction of multi-agent backdoor defenses that maximize accuracy w.r.t. clean labels and minimize that of poison labels. Founded upon agent dynamics and low-loss subspace construction, we contribute three defenses that yield improved multi-agent backdoor robustness. |
Siddhartha Datta 🔗 |
-
|
Deep Learning-based Spatially Explicit Emulation of an Agent-Based Simulator for Pandemic in a City
(
Poster (Contributed)
)
SlidesLive Video » Agent-Based Models are very useful for simulation of physical or social processes, such as the spreading of a pandemic in a city. Such models proceed by specifying the behavior of individuals (agents) and their interactions, and parameterizing the process of infection based on such interactions based on the geography and demography of the city. However, such models are computationally very expensive, and the complexity is often linear in the total number of agents. This seriously limits the usage of such models for simulations, which often have to be run hundreds of times for policy planning and even model parameter estimation. An alternative is to develop an emulator, a surrogate model that can predict the Agent-Based Simulator's output based on its initial conditions and parameters. In this paper, we discuss a Deep Learning model based on Dilated Convolutional Neural Network that can emulate such an agent based model with high accuracy. We show that use of this model instead of the original Agent-Based Model provides us major gains in the speed of simulations, allowing much quicker calibration to observations, and more extensive scenario analysis. The models we consider are spatially explicit, as the locations of the infected individuals are simulated instead of the gross counts. Another aspect of our emulation framework is its divide-and-conquer approach that divides the city into several small overlapping blocks and carries out the emulation in them parallelly, after which these results are merged together. This ensures that the same emulator can work for a city of any size, and also provides significant improvement of time complexity of the emulator, compared to the original simulator. |
Varun Madhavan 🔗 |
-
|
Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Games
(
Poster (Contributed)
)
SlidesLive Video » |
Dingyang Chen 🔗 |
-
|
A Variational Approach to Mutual Information-Based Coordination for Multi-Agent Reinforcement Learning
(
Poster (Contributed)
)
SlidesLive Video » |
WOOJUN KIM 🔗 |
-
|
Risk Perspective Exploration in Distributional Reinforcement Learning
(
Poster (Contributed)
)
SlidesLive Video » Distributional reinforcement learning shows state-of-the-art performance in continuous and discrete control settings with the properties of variance and risk, which can be utilized as a means of exploration. However, the exploration method using risk property is hard to find, while various exploration methods use the variance of return distribution per action in Distributional RL. In this paper, we propose risk scheduling methods, a risk perspective exploration, that explore risk levels and optimistic actions. We show the performance improvement of DMIX algorithm by risk scheduling in a multi-agent setting with various experiments. |
Jihwan Oh 🔗 |
-
|
The AI Macroeconomy: A New Set of Benchmarks for Multiagent Reinforcement Learning Models
(
Poster (Contributed)
)
SlidesLive Video » We propose a new set of challenging benchmark gym environments for testing single- and multi-agent reinforcement learning environments. Single-agent environments are based on a simple consumption-saving decision problem. In each period, agents face an exogenous positive draw that represents how much income they will have in this period. In response, agents may choose what fraction of that income they would like to consume immediately for a reward, or save and get a return going forward on it. In the full version of the problem, all agents' saving decisions generate a price via market clearing. Agents then must learn what their value will be conditioned on the current state. This environment will provide a challenging, potentially non-stationary environment where agents' actions have critical effects on other agents' actions, albeit via a common observation. This environment will be made publicly available via a Github repository and open-source. |
Brandon Kaplowitz 🔗 |
-
|
The Robustness of Inverse Reinforcement Learning
(
Poster (Contributed)
)
Reinforcement learning research experienced substantial jumps in its progress after the first achievement on utilizing deep neural networks to approximate the state-action value function in high-dimensional states. While deep reinforcement learning algorithms are currently being employed in many different tasks from industrial control to biomedical applications, yet the fact that an MDP has to provide a clear reward function limits the tasks that can be achieved via reinforcement learning. In this line of research, some studies proposed to directly learn a policy from observing expert trajectories (i.e. imitation learning), and others proposed to learn a reward function from the expert demonstrations (i.e. inverse reinforcement learning). In this paper we will focus on robustness and vulnerabilities of deep imitation learning and deep inverse reinforcement learning policies. Furthermore, we will layout non-robust features learnt by the deep inverse reinforcement learning policies. We conduct experiments in the Arcade Learning Environment (ALE), and compare the non-robust features learnt by the deep inverse reinforcement learning algorithms to vanilla trained deep reinforcement learning policies. We hope that our study can provide a basis for the future discussions on the robustness of both deep inverse reinforcement learning and deep reinforcement learning. |
Ezgi Korkmaz 🔗 |
-
|
Introducing our mentee Aymeric Vie
(
Mentorship Program Presentation
)
SlidesLive Video » |
Aymeric Vie 🔗 |
-
|
Introducing our mentor Yongli Zhu
(
Mentorship Program Presentation
)
SlidesLive Video » |
Yongli Zhu 🔗 |
-
|
The StarCraft Multi-Agent Challenges+ : Learning of Sub-tasks and Environmental Benefits without Precise Reward Functions (Poster)
(
Poster (Contributed)
)
|
Mingyu Kim 🔗 |
-
|
Calibrating Agent-based Models to Microdata with Graph Neural Networks
(
Poster (Contributed)
)
Calibrating agent-based models to data is among the most fundamental requirements to ensure the model fulfils its desired purpose. In recent years, simulation-based inference methods have emerged as powerful tools for performing this task when the model likelihood function is intractable, as is often the case for agent-based models. In some real-world use cases of agent-based models, both the observed data and the agent-based model output consist of the agents' states and their interactions over time. In such cases, there is a tension between the desire to make full use of the rich information content of such granular data on the one hand, and the need to reduce the dimensionality of the data to prevent difficulties associated with high-dimensional learning tasks on the other. A possible resolution is to construct lower-dimensional time-series through the use of summary statistics describing the macrostate of the system at each time point. However, a poor choice of summary statistics can result in an unacceptable loss of information from the original dataset, dramatically reducing the quality of the resulting calibration. In this work, we instead propose to learn parameter posteriors associated with granular microdata directly using temporal graph neural networks. We will demonstrate that such an approach offers highly compelling inductive biases for Bayesian inference using the raw agent-based model microstates as output. |
Joel Dyer 🔗 |
Author Information
Christian Schroeder (University of Oxford)
Yang Zhang (Banque Canada)
Anisoara Calinescu (University of Oxford)
Dylan Radovic (McKinsey)
Prateek Gupta (University of Oxford)
Jakob Foerster (University of Oxford)
More from the Same Authors
-
2023 : Some challenges of calibrating differentiable agent-based models »
Arnau Quera Bofarull · Joel Dyer · Anisoara Calinescu · Michael Wooldridge -
2022 : Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS »
Christian Schroeder · Yongchao Huang · Phil Torr · Martin Strohmeier -
2022 : Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS »
Christian Schroeder · Yongchao Huang · Phil Torr · Martin Strohmeier -
2022 Poster: Communicating via Markov Decision Processes »
Samuel Sokota · Christian Schroeder · Maximilian Igl · Luisa Zintgraf · Phil Torr · Martin Strohmeier · Zico Kolter · Shimon Whiteson · Jakob Foerster -
2022 Spotlight: Communicating via Markov Decision Processes »
Samuel Sokota · Christian Schroeder · Maximilian Igl · Luisa Zintgraf · Phil Torr · Martin Strohmeier · Zico Kolter · Shimon Whiteson · Jakob Foerster -
2021 Poster: Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning »
Shariq Iqbal · Christian Schroeder · Bei Peng · Wendelin Boehmer · Shimon Whiteson · Fei Sha -
2021 Oral: Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning »
Shariq Iqbal · Christian Schroeder · Bei Peng · Wendelin Boehmer · Shimon Whiteson · Fei Sha -
2020 Poster: Revisiting Training Strategies and Generalization Performance in Deep Metric Learning »
Karsten Roth · Timo Milbich · Samrath Sinha · Prateek Gupta · Bjorn Ommer · Joseph Paul Cohen -
2019 : "Ideas" mini-spotlights »
Kevin McCloskey · Nikola Milojevic-Dupont · Jonathan Binas · Christian Schroeder · Sasha Luccioni -
2019 : Networking Lunch (provided) + Poster Session »
Abraham Stanway · Alex Robson · Aneesh Rangnekar · Ashesh Chattopadhyay · Ashley Pilipiszyn · Benjamin LeRoy · Bolong Cheng · Ce Zhang · Chaopeng Shen · Christian Schroeder · Christian Clough · Clement DUHART · Clement Fung · Cozmin Ududec · Dali Wang · David Dao · di wu · Dimitrios Giannakis · Dino Sejdinovic · Doina Precup · Duncan Watson-Parris · Gege Wen · George Chen · Gopal Erinjippurath · Haifeng Li · Han Zou · Herke van Hoof · Hillary A Scannell · Hiroshi Mamitsuka · Hongbao Zhang · Jaegul Choo · James Wang · James Requeima · Jessica Hwang · Jinfan Xu · Johan Mathe · Jonathan Binas · Joonseok Lee · Kalai Ramea · Kate Duffy · Kevin McCloskey · Kris Sankaran · Lester Mackey · Letif Mones · Loubna Benabbou · Lynn Kaack · Matthew Hoffman · Mayur Mudigonda · Mehrdad Mahdavi · Michael McCourt · Mingchao Jiang · Mohammad Mahdi Kamani · Neel Guha · Niccolo Dalmasso · Nick Pawlowski · Nikola Milojevic-Dupont · Paulo Orenstein · Pedram Hassanzadeh · Pekka Marttinen · Ramesh Nair · Sadegh Farhang · Samuel Kaski · Sandeep Manjanna · Sasha Luccioni · Shuby Deshpande · Soo Kim · Soukayna Mouatadid · Sunghyun Park · Tao Lin · Telmo Felgueira · Thomas Hornigold · Tianle Yuan · Tom Beucler · Tracy Cui · Volodymyr Kuleshov · Wei Yu · yang song · Ydo Wexler · Yoshua Bengio · Zhecheng Wang · Zhuangfang Yi · Zouheir Malki -
2018 Poster: QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning »
Tabish Rashid · Mikayel Samvelyan · Christian Schroeder · Gregory Farquhar · Jakob Foerster · Shimon Whiteson -
2018 Oral: QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning »
Tabish Rashid · Mikayel Samvelyan · Christian Schroeder · Gregory Farquhar · Jakob Foerster · Shimon Whiteson