FlowNar: Scalable Streaming Narration for Long-Form Videos

Recent Large Multimodal Models (LMMs), primarily designed for offline settings, are ill-suited for the dynamic requirements of streaming video. While recent online adaptations improve real-time processing, they still face critical scalability challenges, with resource demands typically growing at least linearly with video duration. To overcome this bottleneck, we propose FlowNar, a novel framework for scalable streaming video narration. The core of FlowNar is a dynamic context management strategy for historical visual context removal, combined with our CLAM (Cross Linear Attentive Memory) module for streaming visual history retention, ensuring bounded visual memory usage and computational complexity, crucial for efficient streaming. We also introduce a realistic self-conditioned evaluation protocol and complementary evaluation metrics to assess streaming narration models under deployment-like conditions. Experiments on the Ego4D, EgoExo4D, and EpicKitchens100 datasets demonstrate that FlowNar substantially improves narration quality over strong baselines while being highly efficient, supporting processing of 10$\times$ longer videos and achieving 3$\times$ higher throughput (FPS). The code is available at https://github.com/zeyun-zhong/FlowNar.

Imagine an AI assistant that watches a live video feed and describes what is happening in real time — useful for helping the visually impaired, augmented-reality glasses, or live commentary on long events. Today's systems handle short clips well but stumble on long videos: they remember every frame they have seen, so their memory keeps growing until the computer runs out of space or grinds to a halt. We built FlowNar, a system that watches video without this memory growth. After it describes a moment, it discards the detailed image data for those seconds, but first hands a short summary to a fixed-size "memory notebook" we designed, called CLAM. This lets the assistant draw on the distant past without re-storing every detail. We also propose a more honest test: instead of feeding the AI the correct past descriptions as a crutch, we make it rely on its own earlier guesses, as it would in practice. On three video datasets, FlowNar handles videos about ten times longer than prior methods, runs roughly three times faster, and produces more accurate descriptions. By keeping its memory use steady as the video grows, it stays fast and reliable where older systems would slow down or crash.