Timezone: »

 
Poster
Countering Language Drift with Seeded Iterated Learning
Yuchen Lu · Soumye Singhal · Florian Strub · Aaron Courville · Olivier Pietquin

Thu Jul 16 06:00 AM -- 06:45 AM & Thu Jul 16 06:00 PM -- 06:45 PM (PDT) @
in Poster Session 41 »

Supervised learning methods excel at capturing statistical properties of language when trained over large text corpora. Yet, these models often produce inconsistent outputs in goal-oriented language settings as they are not trained to complete the underlying task. Moreover, as soon as the agents are finetuned to maximize task completion, they suffer from the so-called language drift phenomenon: they slowly lose syntactic and semantic properties of language as they only focus on solving the task. In this paper, we propose a generic approach to counter language drift called Seeded iterated learning(SIL). We periodically refine a pretrained student agent by imitating data sampled from a newly generated teacher agent. At each time step, the teacher is created by copying the student agent, before being finetuned to maximize task completion.SIL does not require external syntactic constraint nor semantic knowledge, making it a valuable task-agnostic finetuning protocol. We evaluate SIL in a toy-setting Lewis Game, and then scale it up to the translation game with natural language. In both settings, SIL helps counter language drift as well as it improves the task completion compared to baselines.

Keywords: Algorithms · Supervised Learning · Natural Language Processing / Dialogue · Deep Learning - Algorithms

Author Information

Yuchen Lu (Mila & University of Montreal)
Soumye Singhal (Mila, University of Montreal)
Florian Strub (DeepMind)
Aaron Courville (Université de Montréal)
Olivier Pietquin (Google Brain)

More from the Same Authors