Timezone: »

 
Poster
Quantized Decentralized Stochastic Learning over Directed Graphs
Hossein Taheri · Aryan Mokhtari · Hamed Hassani · Ramtin Pedarsani

Wed Jul 15 10:00 AM -- 10:45 AM & Wed Jul 15 09:00 PM -- 09:45 PM (PDT) @ Virtual
in Poster Session 24 »

We consider a decentralized stochastic learning problem where data points are distributed among computing nodes communicating over a directed graph. As the model size gets large, decentralized learning faces a major bottleneck that is the heavy communication load due to each node transmitting large messages (model updates) to its neighbors. To tackle this bottleneck, we propose the quantized decentralized stochastic learning algorithm over directed graphs that is based on the push-sum algorithm in decentralized consensus optimization. More importantly, we prove that our algorithm achieves the same convergence rates of the decentralized stochastic learning algorithm with exact-communication for both convex and non-convex losses. Furthermore, our numerical results illustrate significant speed-up compared to the exact-communication methods.

Keywords: Convex Optimization · Non-convex Optimization · Parallel and Distributed Learning · Optimization - Large Scale, Parallel and Distributed
[ Slides] 

Author Information

Hossein Taheri (UC Santa Barbara)
Aryan Mokhtari (UT Austin)
Hamed Hassani (University of Pennsylvania)
Hamed Hassani

I am an assistant professor in the Department of Electrical and Systems Engineering (as of July 2017). I hold a secondary appointment in the Department of Computer and Information Systems. I am also a faculty affiliate of the Warren Center for Network and Data Sciences. Before joining Penn, I was a research fellow at the Simons Institute, UC Berkeley (program: Foundations of Machine Learning). Prior to that, I was a post-doctoral scholar and lecturer in the Institute for Machine Learning at ETH Zürich. I received my Ph.D. degree in Computer and Communication Sciences from EPFL.

Ramtin Pedarsani (University of California, Santa Barbara)

More from the Same Authors