Can neural networks help us better exploit the inherent structure in the neural network verification problem? In this talk, I will present some recent efforts to speed-up formal verification of neural networks using deep learning. Specifically, we use deep learning to make branching decisions for branch-and-bound, compute descent directions for bound computation, and reduce the search space for counterexample generation. The common methodology behind all these approaches is a graph neural network (GNN) whose architecture closely resembles the network we wish to verify. Using shared parameters, the GNN can be trained on smaller networks and tested on larger ones. Using standard verification tasks, I will show some promising results for our approach.