The acclaimed successes of neural networks often overshadow their tremendous complexity. We focus on numerical precision - a key parameter defining the complexity of neural networks. First, we present theoretical bounds on the accuracy in presence of limited precision. Interestingly, these bounds can be computed via the back-propagation algorithm. Hence, by combining our theoretical analysis and the back-propagation algorithm, we are able to readily determine the minimum precision needed to preserve accuracy without having to resort to time-consuming fixed-point simulations. We provide numerical evidence showing how our approach allows us to maintain high accuracy but with lower complexity than state-of-the-art binary networks.
Charbel Sakr (University of Illinois at Urbana-Champaign)
Yongjune Kim (UIUC)
Naresh Shanbhag (University of Illinois)
Related Events (a corresponding poster, oral, or spotlight)
2017 Talk: Analytical Guarantees on Numerical Precision of Deep Neural Networks »
Mon Aug 7th 04:06 -- 04:24 AM Room C4.8