RELO: Reinforcement Learning to Localize for Visual Object Tracking

Conventional visual object trackers localize targets using handcrafted spatial priors, often in the form of heatmaps. Such priors provide only surrogate supervision and are poorly aligned with tracking optimization and evaluation metrics, such as intersection over union (IoU) and area under the success curve (AUC). Here, we introduce RELO, a REinforcement-learning-to-LOcalize method for visual object tracking that formulates target localization as a Markov decision process. Specifically, RELO replaces handcrafted spatial priors with a localization policy learned over spatial positions via reinforcement learning, with rewards combining frame-level IoU and sequence-level AUC. We additionally introduce layer-aligned temporal token propagation to improve semantic consistency across frames, with negligible computational overhead. Across multiple benchmarks, RELO achieves superior results, attaining $57.5$\% AUC on LaSOT$_\mathrm{ext}$ without template updates. This confirms that reward-driven localization provides an effective alternative to prior-driven localization for visual object tracking.

When a video tracking system is given an object in the first frame, it must keep finding that same object as it moves, changes appearance, becomes partly hidden, or leaves the center of the image. Many current trackers are trained with manually designed hints about where the object is likely to be, rather than learning directly from whether their final tracking results are good. This paper introduces RELO, a tracker that learns where to look by trial and feedback. During training, it tries different possible object locations and is rewarded when its chosen location matches the true object well over both individual frames and whole video clips. This makes the training goal closer to the way tracking quality is actually judged. We also add a lightweight way to carry useful visual information from one frame to the next. Across several standard video tracking tests, RELO tracks objects more accurately than previous methods while still running in real time on a graphics processor, showing that feedback-driven learning is a practical alternative to manually designed location hints.