Characterizing Neural Network Verification for Systems with NN4SysBench

We present NN4SysBench, a benchmark suite for neural network verification, comprised of benchmarks from neural networks for systems (or NN4Sys). NN4Sys is booming: there are hundreds of proposals of using neural networks in computer systems—databases, OSes, and networked systems—that are safety critical. We observe that NN4Sys has some unique characteristics that to- day’s neural network verification tools overlooked. This benchmark aims at bridging the gap between NN4Sys and NN-verification by tailoring impactful NN4Sys instances to benchmarks that today’s NN-verification tools can work on.