Timezone: »
Characterizing Neural Network Verification for Systems with NN4SysBench
Haoyu He · Tianhao Wei · Huan Zhang · Changliu Liu · Cheng Tan
We present NN4SysBench, a benchmark suite for neural network verification, comprised of benchmarks from neural networks for systems (or NN4Sys). NN4Sys is booming: there are hundreds of proposals of using neural networks in computer systems—databases, OSes, and networked systems—that are safety critical. We observe that NN4Sys has some unique characteristics that to- day’s neural network verification tools overlooked. This benchmark aims at bridging the gap between NN4Sys and NN-verification by tailoring impactful NN4Sys instances to benchmarks that today’s NN-verification tools can work on.
Author Information
Haoyu He (Northeastern University)
Tianhao Wei (CMU)
Huan Zhang (CMU)
Changliu Liu (Carnegie Mellon University)
Cheng Tan (Northeastern University)
More from the Same Authors
-
2021 : Empirical robustification of pre-trained classifiers »
Mohammad Sadegh Norouzzadeh · Wan-Yi Lin · Leonid Boytsov · Leslie Rice · Huan Zhang · Filipe Condessa · Zico Kolter -
2021 : Fast Certified Robust Training with Short Warmup »
Zhouxing Shi · Yihan Wang · Huan Zhang · Jinfeng Yi · Cho-Jui Hsieh -
2021 : Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Neural Network Robustness Verification »
Shiqi Wang · Huan Zhang · Kaidi Xu · Xue Lin · Suman Jana · Cho-Jui Hsieh · Zico Kolter -
2021 : Automating Power Networks: Improving RL Agent Robustness with Adversarial Training »
Alexander Pan · Yongkyun Lee · Huan Zhang -
2021 : IADA: Iterative Adversarial Data Augmentation Using Formal Verification and Expert Guidance »
Ruixuan Liu · Changliu Liu -
2022 : An Optical Pulse Stacking Environment and Reinforcement Learning Benchmarks »
Abulikemu Abuduweili · Changliu Liu -
2022 : Characterizing Neural Network Verification for Systems with NN4SysBench »
Haoyu He · Cheng Tan -
2023 : Robustness Verification for Perception Models against Camera Motion Perturbations »
Hanjiang Hu · Changliu Liu · Ding Zhao -
2023 : Formal Verification for Neural Networks with General Nonlinearities via Branch-and-Bound »
Zhouxing Shi · Qirui Jin · Huan Zhang · Zico Kolter · Suman Jana · Cho-Jui Hsieh -
2023 Workshop: 2nd Workshop on Formal Verification of Machine Learning »
Mark Müller · Brendon G. Anderson · Leslie Rice · Zhouxing Shi · Shubham Ugare · Huan Zhang · Martin Vechev · Zico Kolter · Somayeh Sojoudi · Cho-Jui Hsieh -
2023 Poster: Towards Robust and Safe Reinforcement Learning with Benign Off-policy Data »
Zuxin Liu · Zijian Guo · Zhepeng Cen · Huan Zhang · Yihang Yao · Hanjiang Hu · Ding Zhao -
2022 Affinity Workshop: Queer in AI @ ICML 2022 Affinity Workshop »
Huan Zhang · Arjun Subramonian · Sharvani Jha · William Agnew · Krunoslav Lehman Pavasovic -
2022 : Invited Talk 6 (Changliu Liu): Applications of Neural Verification on Robotics »
Changliu Liu -
2022 : Paper 15: On the Robustness of Safe Reinforcement Learning under Observational Perturbations »
Zuxin Liu · Zhepeng Cen · Huan Zhang · Jie Tan · Bo Li · Ding Zhao -
2022 : New adversarial ML applications on safety-critical human-robot systems »
Changliu Liu -
2022 Workshop: Workshop on Formal Verification of Machine Learning »
Huan Zhang · Leslie Rice · Kaidi Xu · aditi raghunathan · Wan-Yi Lin · Cho-Jui Hsieh · Clark Barrett · Martin Vechev · Zico Kolter -
2022 Poster: A Branch and Bound Framework for Stronger Adversarial Attacks of ReLU Networks »
Huan Zhang · Shiqi Wang · Kaidi Xu · Yihan Wang · Suman Jana · Cho-Jui Hsieh · Zico Kolter -
2022 Poster: Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness »
Tianlong Chen · Huan Zhang · Zhenyu Zhang · Shiyu Chang · Sijia Liu · Pin-Yu Chen · Zhangyang “Atlas” Wang -
2022 Spotlight: A Branch and Bound Framework for Stronger Adversarial Attacks of ReLU Networks »
Huan Zhang · Shiqi Wang · Kaidi Xu · Yihan Wang · Suman Jana · Cho-Jui Hsieh · Zico Kolter -
2022 Spotlight: Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness »
Tianlong Chen · Huan Zhang · Zhenyu Zhang · Shiyu Chang · Sijia Liu · Pin-Yu Chen · Zhangyang “Atlas” Wang -
2022 Social: Black in AI and Queer in AI Joint Social Event »
Victor Silva · Huan Zhang · Nathaniel Rose · Arjun Subramonian · Krunoslav Lehman Pavasovic · Ana Da Hora -
2020 Poster: On Lp-norm Robustness of Ensemble Decision Stumps and Trees »
Yihan Wang · Huan Zhang · Hongge Chen · Duane Boning · Cho-Jui Hsieh -
2019 Poster: Robust Decision Trees Against Adversarial Examples »
Hongge Chen · Huan Zhang · Duane Boning · Cho-Jui Hsieh -
2019 Oral: Robust Decision Trees Against Adversarial Examples »
Hongge Chen · Huan Zhang · Duane Boning · Cho-Jui Hsieh -
2018 Poster: Towards Fast Computation of Certified Robustness for ReLU Networks »
Tsui-Wei Weng · Huan Zhang · Hongge Chen · Zhao Song · Cho-Jui Hsieh · Luca Daniel · Duane Boning · Inderjit Dhillon -
2018 Oral: Towards Fast Computation of Certified Robustness for ReLU Networks »
Tsui-Wei Weng · Huan Zhang · Hongge Chen · Zhao Song · Cho-Jui Hsieh · Luca Daniel · Duane Boning · Inderjit Dhillon -
2017 Poster: Gradient Boosted Decision Trees for High Dimensional Sparse Output »
Si Si · Huan Zhang · Sathiya Keerthi · Dhruv Mahajan · Inderjit Dhillon · Cho-Jui Hsieh -
2017 Talk: Gradient Boosted Decision Trees for High Dimensional Sparse Output »
Si Si · Huan Zhang · Sathiya Keerthi · Dhruv Mahajan · Inderjit Dhillon · Cho-Jui Hsieh