VSCD: Video-based Scene Change Detection in Unaligned Scenes

Detecting what has changed in an environment is essential for long-term autonomy, yet most change detection settings assume fixed viewpoints, mild misalignment, or only a few changed objects. We introduce Video-based Scene Change Detection (VSCD), which predicts a pixel-wise change mask for each query frame, given a reference and a query RGB video of the same indoor space recorded at different times under unconstrained camera motion. The two videos are not temporally synchronized, and many object instances may appear or disappear. To study this setting, we build a large-scale benchmark with over 1.1 million frames annotated with pixel-accurate change masks, together with a real-world test set for evaluating transfer beyond simulation. We propose a query-centric multi-reference model that learns temporal matching implicitly from change-mask supervision, aligns candidate reference features to the query via local patch correspondence, and fuses per-candidate change features using frame-level and patch-level confidence before decoding a high-resolution mask once per frame. Our approach achieves state-of-the-art performance against strong image- and video-based baselines, and we validate its real-world impact by deploying it on a mobile robot for two downstream applications—visual surveillance and object incremental learning.

Robots that operate for a long time in the same indoor space need to notice when the environment has changed, such as when objects appear, disappear, or move. This paper studies how to detect these changes by comparing two videos of the same room recorded at different times, even when the camera paths and viewpoints are not aligned. We introduce a large benchmark for this problem and develop a model that finds changed regions in each video frame. We also test the method in real-world robot scenarios, showing that it can support applications such as visual surveillance and learning newly appeared objects.