Research at the intersection of vision and language has been attracting a lot of attention in recent years. Topics include the study of multi-modal representations, translation between modalities, bootstrapping of labels from one modality into another, visually-grounded question answering, segmentation and storytelling, and grounding the meaning of language in visual data. An ever-increasing number of tasks and datasets are appearing around this recently-established field.
At NeurIPS 2018, we released the How2 data-set, containing more than 85,000 (2000h) videos, with audio, transcriptions, translations, and textual summaries. We believe it presents an ideal resource to bring together researchers working on the previously mentioned separate tasks around a single, large dataset. This rich dataset will facilitate the comparison of tools and algorithms, and hopefully foster the creation of additional annotations and tasks. We want to foster discussion about useful tasks, metrics, and labeling techniques, in order to develop a better understanding of the role and value of multi-modality in vision and language. We seek to create a venue to encourage collaboration between different sub-fields, and help establish new research directions and collaborations that we believe will sustain machine learning research for years to come.
|08:45 AM||Welcome (Break)|
|09:00 AM||The How2 Database and Challenge (Presentation) Video »||Lucia Specia, Ramon Sanabria|
|10:15 AM||Coffee Break (Break)|
|10:30 AM||Forcing Vision + Language Models To Actually See, Not Just Talk (Invited Talk 1) Video »||Devi Parikh|
|11:00 AM||Topics in Vision and Language: Grounding, Segmentation and Author Anonymity (Invited Talk 2 (Bernt Schiele)) Video »|
|11:30 AM||Learning to Reason: Modular and Relational Representations for Visual Questions and Referring Expressions (Invited Talk 3) Video »||Kate Saenko|
|01:30 PM||Multi-agent communication from raw perceptual input: what works, what doesn't and what's next (Invited Talk 4) Video »||Angeliki Lazaridou|
|02:00 PM||Overcoming Bias in Captioning Models (Invited Talk 5) Video »||Lisa Anne Hendricks|
|02:30 PM||Embodied language grounding (Invited Talk 6 (Katerina Fragkiadaki)) Video »||Katerina Fragkiadaki|
|03:00 PM||Poster Session and Coffee (Poster Session and Break)||Ramon Sanabria, Tejas Srinivasan, Vikas Raunak, Luowei Zhou, Aradhyo Kundu, Roma Patel, Lucia Specia, Sang Keun Choe, Anna Belova|
|04:30 PM||Unsupervised Bilingual Lexicon Induction from mono-lingual multimodal data (Invited Talk 7 (remote))||Qin Jin|
|05:00 PM||New Directions for Vision & Language (Discussion Panel)||Florian Metze, Shruti Palaskar|