Linguistic Relative Policy Optimization for Video Anomaly Reasoning

Video anomaly detection (VAD) with multimodal large language models has shown strong potential, yet most existing methods still depend on large-scale annotations or expert-designed priors, limiting their ability to acquire anomaly knowledge with as little human intervention as possible. To address this, we propose Linguistic Relative Policy Optimization (LRPO), which distills group-relative semantic advantages from multiple reasoning trajectories into a linguistically expressed anomaly experience prior, and adapts the model by injecting this prior into the context to steer its output distribution without any parameter updates. LRPO builds two complementary experience representations: general experience captures transferable anomaly preferences across scenarios, while scenario experience models context-dependent anomaly rules for targeted refinement. To further improve the learned experience, we introduce an anomaly alignment reward that guides trajectory optimization to match human risk preferences and reinforce temporally grounded reasoning. Extensive experiments on XD-Violence, UCF-Crime, and UBNormal demonstrate that LRPO significantly outperforms existing state-of-the-art methods under tuning-free settings.

Keeping public spaces, transport systems, and campuses safe is important, but these places produce much video, making it hard for people to continuously watch for fights, accidents, or fires. Intelligent systems could help find these events, but such events are rare, and teaching a system to recognize them usually requires people to mark abnormal examples for training, which is costly. As a result, many existing methods rely on human marking or expert-written rules, making them hard to adapt to new scenes. To address this, we propose LRPO, a way to help an intelligent system learn anomaly judgment experience from a small number of examples instead of retraining the whole system. Specifically, LRPO turns useful differences between the system's reasoning attempts into plain-language experience, then provides relevant experience when the system analyzes a new video. Experiments show that LRPO improves video anomaly detection with limited data and without fine-tuning the system. It also makes the reasoning easier to inspect and may reduce the human effort needed to adapt anomaly detection systems to new environments. At the same time, because such systems can affect privacy and may produce false alarms, responsible deployment still requires legal safeguards and human oversight.