Skip to yearly menu bar Skip to main content

Workshop: Differentiable Almost Everything: Differentiable Relaxations, Algorithms, Operators, and Simulators

A Short Review of Automatic Differentiation Pitfalls in Scientific Computing

Jan Hueckelheim · Harshitha Menon · William Moses · Bruce Christianson · Paul Hovland · Laurent Hascoet


Automatic differentiation, also known as backpropagation, AD, autodiff, or algorithmic differentiation, is a popular technique for computing derivatives of computer programs. While AD has been successfully used in countless engineering, science and machine learning applications, it can sometimes nevertheless produce surprising results. In this paper we categorize problematic usages of AD and illustrate each category with examples such as chaos, time-averages, discretizations, fixed-point loops, lookup tables, linear solvers, and probabilistic programs, in the hope that readers may more easily avoid or detect such pitfalls.

Chat is not available.