Smoothing Slot Attention Iterations and Recurrences

Slot Attention (SA) lies at the heart of mainstream Object-Centric Learning (OCL). Image features can be aggregated into object-level representations by SA iteratively refining cold-start query slots. For video, such aggregation proceeds by SA recurrently shared across frames, with queries cold-started on the first frame while transitioned from the previous frame's slots thereafter. However, cold-start queries lack sample-specific cues thus hindering precise aggregation on image or video's first frame; Non-first frames' queries are already sample-specific thus requiring aggregation transforms different from the first frame. We address these issues with our SmoothSA: (1) To smooth SA iterations on image or video's first frame, we preheat cold-start queries with rich input-feature information, by a tiny module self-distilled inside OCL; (2) To smooth SA recurrences across video's first and non-first frames, we differentiate the homogeneous aggregation transforms by using full and single iterations respectively. Comprehensive experiments on object discovery, recognition and visual reasoning validate our method's effectiveness. Further visual analyses illuminate the underline mechanisms. Our source code, model checkpoints and training logs are provided on https://github.com/Genera1Z/SmoothSA.

This paper improves how AI systems identify and track objects in images and videos. Existing methods often struggle at the start because they begin with generic guesses and process all video frames in the same way. We propose SmoothSA, a simple method that gives the model a better starting understanding of the scene and treats the first video frame differently from later frames. This makes object learning more stable and accurate over time. Experiments on object discovery, recognition, and visual reasoning show consistent improvements over existing approaches. We also release our code, trained models, and training logs on GitHub https://github.com/Genera1Z/SmoothSA