Skip to yearly menu bar Skip to main content

Workshop: The First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward

PARS-Push: Personalized, Asynchronous and Robust Decentralized Optimization

Mohammad Taha Toghani · Mohammad Taha Toghani · Soomin Lee · Soomin Lee · Cesar Uribe

Abstract: We study the decentralized multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of $n$ agents seeks to find a common point that enables ``few-shot'' learning (personalization) via local stochastic gradient steps on their local functions. We formulate the personalized optimization problem under the MAML framework and propose PARS-Push, a decentralized asynchronous algorithm robust to message failures, communication delays, and directed message sharing. We characterize the convergence rate of PARS-Push for smooth and strongly convex and smooth and non-convex functions under arbitrary multi-step personalization. Moreover, we provide numerical experiments showing its performance under heterogeneous data setups.

Chat is not available.