Timezone: »

Analytical Guarantees on Numerical Precision of Deep Neural Networks
Charbel Sakr · Yongjune Kim · Naresh Shanbhag

Mon Aug 07 01:30 AM -- 05:00 AM (PDT) @ Gallery #65

The acclaimed successes of neural networks often overshadow their tremendous complexity. We focus on numerical precision - a key parameter defining the complexity of neural networks. First, we present theoretical bounds on the accuracy in presence of limited precision. Interestingly, these bounds can be computed via the back-propagation algorithm. Hence, by combining our theoretical analysis and the back-propagation algorithm, we are able to readily determine the minimum precision needed to preserve accuracy without having to resort to time-consuming fixed-point simulations. We provide numerical evidence showing how our approach allows us to maintain high accuracy but with lower complexity than state-of-the-art binary networks.

Author Information

Charbel Sakr (University of Illinois at Urbana-Champaign)
Yongjune Kim (UIUC)
Naresh Shanbhag (University of Illinois)

Related Events (a corresponding poster, oral, or spotlight)