Lessons of Local Learning in Training LLMs
Stephen Gou
2023 Invited Talk
in
Workshop: Localized Learning: Decentralized Model Updates via Non-Global Objectives
in
Workshop: Localized Learning: Decentralized Model Updates via Non-Global Objectives
Abstract
This presentation summarizes the main idea from interlocking backpropagation paper and discusses the intriguing intersection of large language models and local learning, investigating the unique challenges and opportunities. Additionally we'll touch on the state-of-the-art training paradigm of LLM and local learning's potential.
Video
Chat is not available.
Successful Page Load