Convex Analysis at Infinity: An Introduction to Astral Space

"Optimization is centrally important to machine learning, including optimization of convex functions. A particular challenge arises, however, when the function being minimized, even if convex, has no finite minimizer, and so can only be minimized by a sequence as it heads to infinity, as can certainly occur in practice, for instance, when minimizing objectives based on log loss (or cross entropy). Analyzing statistical properties and proving the convergence of algorithms in such cases is considerably more difficult.

This tutorial presents a new theory for studying minimizers of convex functions at infinity, introducing astral space, an extension of Euclidean space that includes points at infinity, and that has many favorable properties. In extending convex analysis, astral space provides a mathematical foundation for the study of optimization algorithms when minimizers exist only at infinity. We will look at how some of the most important topics studied in convex analysis extend to astral space. We also look at applications of particular relevance to machine learning, such as in the analysis of descent methods."