Timezone: »
As global demand for electricity increases, operating power networks has become more complex. Power network operation can be posed as a reinforcement learning (RL) task, and there is increasing interest in developing RL agents that can automate operation. The Learning To Run Power Network (L2RPN) environment models a real-world electric grid, posing as a test bed for these RL agents. Agents must be robust, i.e., ensure reliable electricity flow even when some power lines are disconnected. Because of the large state and action space of power grids, robustness is hard to achieve and has become a key technical obstacle in widespread adoption of RL for power networks. To improve the robustness of L2RPN agents, we propose adversarial training. We make the following contributions: 1) we design an agent-specific \emph{adversary MDP} to train an adversary that minimizes a given agent's reward; 2) we demonstrate the potency of our adversarial policies against winning agent policies from the L2RPN challenge; 3) we improve the robustness of a winning L2RPN agent by adversarially training it against our learned adversary. To the best of our knowledge, we provide the first evidence that learned adversaries for power network agents are potent. We also demonstrate a novel, real-world application of adversarial training: improving the robustness of RL agents for power networks.
Author Information
Alexander Pan (California Institute of Technology)
Yongkyun Lee (California Institute of Technology)
Huan Zhang (UCLA)
More from the Same Authors
-
2021 : Empirical robustification of pre-trained classifiers »
Mohammad Sadegh Norouzzadeh · Wan-Yi Lin · Leonid Boytsov · Leslie Rice · Huan Zhang · Filipe Condessa · Zico Kolter -
2021 : Fast Certified Robust Training with Short Warmup »
Zhouxing Shi · Yihan Wang · Huan Zhang · Jinfeng Yi · Cho-Jui Hsieh -
2021 : Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Neural Network Robustness Verification »
Shiqi Wang · Huan Zhang · Kaidi Xu · Xue Lin · Suman Jana · Cho-Jui Hsieh · Zico Kolter -
2023 Poster: Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the Machiavelli Benchmark »
Alexander Pan · Jun Shern Chan · Andy Zou · Nathaniel Li · Scott Emmons · Hanlin Zhang · Steven Basart · Thomas Woodside · Dan Hendrycks -
2023 Poster: Towards Robust and Safe Reinforcement Learning with Benign Off-policy Data »
Zuxin Liu · Zijian Guo · Zhepeng Cen · Huan Zhang · Yihang Yao · Hanjiang Hu · Ding Zhao -
2023 Oral: Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the Machiavelli Benchmark »
Alexander Pan · Jun Shern Chan · Andy Zou · Nathaniel Li · Scott Emmons · Hanlin Zhang · Steven Basart · Thomas Woodside · Dan Hendrycks -
2023 Workshop: 2nd Workshop on Formal Verification of Machine Learning »
Mark Müller · Brendon G. Anderson · Leslie Rice · Zhouxing Shi · Shubham Ugare · Huan Zhang · Martin Vechev · Zico Kolter · Somayeh Sojoudi · Cho-Jui Hsieh -
2022 Affinity Workshop: Queer in AI @ ICML 2022 Affinity Workshop »
Huan Zhang · Arjun Subramonian · Sharvani Jha · William Agnew · Krunoslav Lehman Pavasovic -
2022 : Paper 15: On the Robustness of Safe Reinforcement Learning under Observational Perturbations »
Zuxin Liu · Zhepeng Cen · Huan Zhang · Jie Tan · Bo Li · Ding Zhao -
2022 : Characterizing Neural Network Verification for Systems with NN4SysBench »
Haoyu He · Tianhao Wei · Huan Zhang · Changliu Liu · Cheng Tan -
2022 Workshop: Workshop on Formal Verification of Machine Learning »
Huan Zhang · Leslie Rice · Kaidi Xu · aditi raghunathan · Wan-Yi Lin · Cho-Jui Hsieh · Clark Barrett · Martin Vechev · Zico Kolter -
2022 Poster: A Branch and Bound Framework for Stronger Adversarial Attacks of ReLU Networks »
Huan Zhang · Shiqi Wang · Kaidi Xu · Yihan Wang · Suman Jana · Cho-Jui Hsieh · Zico Kolter -
2022 Poster: Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness »
Tianlong Chen · Huan Zhang · Zhenyu Zhang · Shiyu Chang · Sijia Liu · Pin-Yu Chen · Zhangyang “Atlas” Wang -
2022 Spotlight: A Branch and Bound Framework for Stronger Adversarial Attacks of ReLU Networks »
Huan Zhang · Shiqi Wang · Kaidi Xu · Yihan Wang · Suman Jana · Cho-Jui Hsieh · Zico Kolter -
2022 Spotlight: Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness »
Tianlong Chen · Huan Zhang · Zhenyu Zhang · Shiyu Chang · Sijia Liu · Pin-Yu Chen · Zhangyang “Atlas” Wang -
2022 Social: Black in AI and Queer in AI Joint Social Event »
Victor Silva · Huan Zhang · Nathaniel Rose · Arjun Subramonian · Krunoslav Lehman Pavasovic · Ana Da Hora -
2020 Poster: On Lp-norm Robustness of Ensemble Decision Stumps and Trees »
Yihan Wang · Huan Zhang · Hongge Chen · Duane Boning · Cho-Jui Hsieh -
2019 Poster: Robust Decision Trees Against Adversarial Examples »
Hongge Chen · Huan Zhang · Duane Boning · Cho-Jui Hsieh -
2019 Oral: Robust Decision Trees Against Adversarial Examples »
Hongge Chen · Huan Zhang · Duane Boning · Cho-Jui Hsieh