Unified Modal Learning: Motivation, Practice and Beyond

Existed pre-training methods either focus on single-modal tasks or multi-modal tasks, and cannot effectively adapt to each other. However, human beings are very good at learning from multi-source heterogeneous data to better understand the physical concept. In this talk, we introduce unified modal learning, whose target is to learn from different modalities of information simultaneously in a more general way and has the ability to boost both single-modal and multi-modal tasks. Based on PaddlePaddle, we present a unified modal pre-training architecture namely UNIMO. It achieves SOTAs on several NLP and multi-modal benchmarks. We hope that unified model learning will provide a possible way to Artificial General Intelligence(AGI) and can be built together by community.