Oral
Generalized Majorization-Minimization
Sobhan Naderi Parizi · Kun He · Reza Aghajani · Stan Sclaroff · Pedro Felzenszwalb

Wed Jun 12th 02:20 -- 02:25 PM @ Room 104

Non-convex optimization is ubiquitous in machine learning. Majorization-Minimization (MM) is a powerful iterative procedure for optimizing non-convex functions that works by optimizing a sequence of bounds on the function. In MM, the bound at each iteration is required to touch the objective function at the optimizer of the previous bound. We show that this touching constraint is unnecessary and overly restrictive. We generalize MM by relaxing this constraint, and propose a new optimization framework, named Generalized Majorization-Minimization (G-MM), that is more flexible. For instance, G-MM can incorporate application-specific biases into the optimization procedure without changing the objective function. We derive G-MM algorithms for several latent variable models and show empirically that they consistently outperform their MM counterparts in optimizing non-convex objectives. In particular, G-MM algorithms appear to be less sensitive to initialization.

Author Information

Sobhan Naderi Parizi (Google Inc.)
Kun He (Facebook Reality Labs)
Reza Aghajani (University of California San Diego)
Stan Sclaroff (Boston University)
Pedro Felzenszwalb (Brown University)

Related Events (a corresponding poster, oral, or spotlight)