Skip to yearly menu bar Skip to main content


Generalized Majorization-Minimization

Sobhan Naderi Parizi · Kun He · Reza Aghajani · Stan Sclaroff · Pedro Felzenszwalb

Pacific Ballroom #102

Keywords: [ Structured Prediction ] [ Optimization ] [ Non-convex Optimization ] [ Computer Vision ] [ Clustering ]


Non-convex optimization is ubiquitous in machine learning. Majorization-Minimization (MM) is a powerful iterative procedure for optimizing non-convex functions that works by optimizing a sequence of bounds on the function. In MM, the bound at each iteration is required to touch the objective function at the optimizer of the previous bound. We show that this touching constraint is unnecessary and overly restrictive. We generalize MM by relaxing this constraint, and propose a new optimization framework, named Generalized Majorization-Minimization (G-MM), that is more flexible. For instance, G-MM can incorporate application-specific biases into the optimization procedure without changing the objective function. We derive G-MM algorithms for several latent variable models and show empirically that they consistently outperform their MM counterparts in optimizing non-convex objectives. In particular, G-MM algorithms appear to be less sensitive to initialization.

Live content is unavailable. Log in and register to view live content