Timezone: »

Bidirectional learning for time-series models with hidden units
Takayuki Osogami · Hiroshi Kajino · Taro Sekiyama

Tue Aug 08 08:30 PM -- 08:48 PM (PDT) @ C4.6 & C4.7

Hidden units can play essential roles in modeling time-series having long-term dependency or on-linearity but make it difficult to learn associated parameters. Here we propose a way to learn such a time-series model by training a backward model for the time-reversed time-series, where the backward model has a common set of parameters as the original (forward) model. Our key observation is that only a subset of the parameters is hard to learn, and that subset is complementary between the forward model and the backward model. By training both of the two models, we can effectively learn the values of the parameters that are hard to learn if only either of the two models is trained. We apply bidirectional learning to a dynamic Boltzmann machine extended with hidden units. Numerical experiments with synthetic and real datasets clearly demonstrate advantages of bidirectional learning.

Author Information

Takayuki Osogami (IBM Research - Tokyo)
Hiroshi Kajino (IBM Research - Tokyo)
Taro Sekiyama (IBM Research - Tokyo)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors