Plenary Talk
in
Workshop: Beyond first-order methods in machine learning systems
Recent trends in regularization methods with adaptive accuracy requirements
Stefania Bellavia
In this talk we present regularization methods employing inexact function and derivatives evaluations. They feature a very flexible adaptive mechanism for determining the inexactness which is allowed, at each iteration, when computing objective function and derivatives, in order to preserve the complexity results of their exact counterpart. The complexity analysis covers arbitrary optimality order and arbitrary degree of available approximate derivatives.
In most applications the accuracy requirements can be satisfied only within a certain probability. As a first step to cope with this non-deterministic aspect, complexity results for the case of exact functions and randomly perturbed derivatives are provided. We also analyse the key computational aspects related to an efficient implementation of the method of this class employing a cubic regularized second-order model and provide some numerical results showing the behaviour of the method.
Authors: Stefania Bellavia, Gianmarco Gurioli, Benedetta Morini, Philippe Toint