Timezone: »

Improving Generative Imagination in Object-Centric World Models
Zhixuan Lin · Yi-Fu Wu · Skand Peri · Bofeng Fu · Jindong Jiang · Sungjin Ahn

Tue Jul 14 07:00 AM -- 07:45 AM & Tue Jul 14 06:00 PM -- 06:45 PM (PDT) @ Virtual #None

The remarkable recent advances in object-centric generative world models raise a few questions. First, while many of the recent achievements are indispensable for making a general and versatile world model, it is quite unclear how these ingredients can be integrated into a unified framework. Second, despite using generative objectives, abilities for object detection and tracking are mainly investigated, leaving the crucial ability of temporal imagination largely under question. Third, a few key abilities for more faithful temporal imagination such as multimodal uncertainty and situation-awareness are missing. In this paper, we introduce Generative Structured World Models (G-SWM). The G-SWM achieves the versatile world modeling not only by unifying the key properties of previous models in a principled framework but also by achieving two crucial new abilities, multimodal uncertainty and situation-awareness. Our thorough investigation on the temporal generation ability in comparison to the previous models demonstrates that G-SWM achieves the versatility with the best or comparable performance for all experiment settings including a few complex settings that have not been tested before. https://sites.google.com/view/gswm

Author Information

Zhixuan Lin (Zhejiang University)
Yi-Fu Wu (Rutgers University)
Skand Peri (Rutgers University, New Jersey)
Bofeng Fu (Tianjin University)
Jindong Jiang (Rutgers University)
Sungjin Ahn (Rutgers University)

More from the Same Authors