Think in Cloud, Look at Edges: Semantic-Driven Query Decomposition for Efficient Video Reasoning

Long video understanding faces a critical dilemma: cloud-based Large Multimodal Models (LMMs) offer superior reasoning but suffer from prohibitive bandwidth costs and latency, while edge-based solutions sacrifice perception accuracy for speed. Current collaborative approaches attempt to bridge this gap via similarity-based filtering, yet they treat complex queries as flat semantic vectors. We identify this as a fundamental flaw leading to "Semantic Submergence," where dominant visual features drown out subtle but logically critical cues. To solve this, we introduce SCOPE (Semantic Cloud-Orchestrated Perception at Edge). Shifting the paradigm to "Think in Cloud, Look at Edges," SCOPE utilizes a cloud LMM to decompose complex queries into a structured Directed Acyclic Graph (DAG). This "observation plan" guides the edge to retrieve evidence based on logical necessity rather than mere statistical similarity. Experiments on Video-MME and LongVideoBench demonstrate that SCOPE redefines the Pareto frontier, matching cloud-level accuracy with significantly lower transmission costs and outperforming state-of-the-art baselines on complex reasoning tasks.

Modern AI systems can answer surprisingly complex questions about videos—for example, "What does the person do after finishing cooking?"—but doing this well requires very large AI models that usually run in the cloud. Sending an entire long video to the cloud is slow and expensive, while running smaller models directly on a phone or laptop sacrifices accuracy. Existing compromise solutions try to pick a few "important" frames on the device and only send those, but they often misjudge which frames matter, getting distracted by visually obvious scenes and missing the subtle moments that actually answer the question. We propose SCOPE, a method that asks the powerful cloud model to first plan what to look for before any video is examined. The cloud breaks the question into a small set of simpler sub-questions (for example, "find the cooking activity" and "find the moment the light is turned off") and tells the device how important each one is. The device then searches the video for evidence of each sub-question separately, and only uploads those targeted frames. In experiments on standard long-video benchmarks, SCOPE matches the accuracy of running the full cloud model on the whole video while reducing total response time by roughly 85%. This makes high-quality video understanding practical in everyday situations with limited network bandwidth, such as smart-home assistants, surveillance, and mobile applications.