Timezone: »

 
Oral
TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
Augustus Odena · Catherine Olsson · David Andersen · Ian Goodfellow

Thu Jun 13 09:40 AM -- 10:00 AM (PDT) @ Grand Ballroom
in Interpretability »

Neural networks are difficult to interpret and debug. We introduce testing techniques for neural networks that can discover errors occurring only for rare inputs. Specifically, we develop coverage-guided fuzzing (CGF) methods for neural networks. In CGF, random mutations of inputs are guided by a coverage metric toward the goal of satisfying user-specified constraints. We describe how approximate nearest neighbor (ANN) algorithms can provide this coverage metric for neural networks. We then combine these methods with techniques for property-based testing (PBT). In PBT, one asserts properties that a function should satisfy and the system automatically generates tests exercising those properties. We then apply this system to practical goals including (but not limited to) surfacing broken loss functions in popular GitHub repositories and making performance improvements to TensorFlow. Finally, we release an open source library called TensorFuzz that implements the described techniques.

Author Information

Augustus Odena (Google Brain)
Catherine Olsson (Open Philanthropy Project)
David Andersen (Google)
Ian Goodfellow (Google Brain)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors