PRISM: Training-Free Video Anomaly Detection via Intrinsic Statistical Modeling

While recent training-free video anomaly detection (VAD) methods offer advantages such as interpretability and ease of deployment, they often suffer from computational inefficiency due to complex memory retrieval mechanisms or high-latency visual-language models (VLMs). To address this issue, we propose PRISM (Parameter-less Recognition Based on Intrinsic Statistical Modeling), a novel framework for efficient open-set anomaly detection with minimal computational cost. Built on a pre-trained multimodal embedding model, PRISM introduces differential amplification and whitening mechanisms to statistically suppress common-mode background noise in the embedding space, thereby improving the signal-to-noise ratio of anomalous events. Extensive experiments on three widely datasets demonstrate that PRISM achieves state-of-the-art performance among training-free methods while maintaining real-time inference capability. Furthermore, our statistical analysis offers a complementary perspective on why training-free methods may suffer from lower Average Precision (AP) on complex datasets such as XD-Violence.Code is released at https://github.com/ytC2026/ICML2026-PRISM.

Video anomaly detection aims to find unusual or dangerous events, such as fights, accidents, or violence, in large collections of video. This is important for safety and monitoring, but many existing methods either require expensive training on labeled videos or rely on large models that are too slow for real-time use. We developed PRISM, a lightweight method that detects video anomalies without additional training. Instead of learning from labeled videos, PRISM uses short text descriptions of normal and abnormal events as references. It then applies a simple statistical correction to reduce the influence of common background information, such as scene appearance, crowds, or camera motion. This makes abnormal events stand out more clearly from normal video content. Our experiments show that PRISM achieves strong results on several video anomaly detection benchmarks while remaining efficient enough for real-time use with suitable visual encoders. This work may help make video anomaly detection faster and easier to deploy, while still requiring careful privacy protection, human oversight, and validation in real-world settings.