Timezone: »

 
Spotlight
Domain Adaptation for Time Series Forecasting via Attention Sharing
Xiaoyong Jin · Youngsuk Park · Danielle Robinson · Hao Wang · Yuyang Wang

Thu Jul 21 10:35 AM -- 10:40 AM (PDT) @ Hall G
in Applications »

Recently, deep neural networks have gained increasing popularity in the field of time series forecasting. A primary reason for their success is their ability to effectively capture complex temporal dynamics across multiple related time series. The advantages of these deep forecasters only start to emerge in the presence of a sufficient amount of data. This poses a challenge for typical forecasting problems in practice, where there is a limited number of time series or observations per time series, or both. To cope with this data scarcity issue, we propose a novel domain adaptation framework, Domain Adaptation Forecaster (DAF). DAF leverages statistical strengths from a relevant domain with abundant data samples (source) to improve the performance on the domain of interest with limited data (target). In particular, we use an attention-based shared module with a domain discriminator across domains and private modules for individual domains. We induce domain-invariant latent features (queries and keys) and retrain domain-specific features (values) simultaneously to enable joint training of forecasters on source and target domains. A main insight is that our design of aligning keys allows the target domain to leverage source time series even with different characteristics. Extensive experiments on various domains demonstrate that our proposed method outperforms state-of-the-art baselines on synthetic and real-world datasets, and ablation studies verify the effectiveness of our design choices.

Keywords: APP: Time Series · DL: Attention Mechanisms

Author Information

Xiaoyong Jin (UCSB)
Youngsuk Park (Amazon Research)
Danielle Robinson (Amazon Web Services)
Hao Wang (Rutgers University)
Hao Wang

Dr. Hao Wang is currently an assistant professor in the department of computer science at Rutgers University. Previously he was a Postdoctoral Associate at the Computer Science & Artificial Intelligence Lab (CSAIL) of MIT, working with Dina Katabi and Tommi Jaakkola. He received his PhD degree from the Hong Kong University of Science and Technology, as the sole recipient of the School of Engineering PhD Research Excellence Award in 2017. He has been a visiting researcher in the Machine Learning Department of Carnegie Mellon University. His research focuses on statistical machine learning, deep learning, and data mining, with broad applications on recommender systems, healthcare, user profiling, social network analysis, text mining, etc. His work on Bayesian deep learning for recommender systems and personalized modeling has inspired hundreds of follow-up works published at top conferences such as AAAI, ICML, IJCAI, KDD, NIPS, SIGIR, and WWW. It has received over 1000 citations, becoming the most cited paper at KDD 2015. In 2015, he was awarded the Microsoft Fellowship in Asia and the Baidu Research Fellowship for his innovation on Bayesian deep learning and its applications on data mining and social network analysis.

Yuyang Wang (AWS AI Labs)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors