Mitigating Surgical Data Imbalance with Dual-Prediction Video Diffusion Model

Surgical video datasets are essential for scene understanding, enabling procedural modeling and intra-operative support. However, these datasets are often heavily imbalanced, with rare actions and tools under-represented, which limits the robustness of downstream models. We address this challenge with *SurgiFlowVid*, a sparse and controllable video diffusion framework for generating surgical videos of under-represented classes. Our approach introduces a dual-prediction diffusion module that jointly denoises RGB frames and optical flow, providing temporal inductive biases to improve motion modeling from limited samples. In addition, a sparse visual encoder conditions the generation process on lightweight signals (e.g., sparse segmentation masks or RGB frames), enabling controllability without dense annotations. We validate our approach on three surgical datasets across tasks including action recognition, tool presence detection, and laparoscope motion prediction. Synthetic data generated by our method yields consistent gains of $10$–$20$% over competitive baselines, establishing *SurgiFlowVid* as a promising strategy to mitigate data imbalance and advance surgical video understanding methods.

During minimally invasive surgery, surgeons operate through a camera rather than looking directly at the patient. Recordings from these cameras are valuable for training AI systems that could one day assist surgeons in real time, for example, by recognizing which step of a procedure is underway or which instruments are in use. However, surgical video datasets are heavily skewed: common actions appear thousands of times, while rare but clinically important actions may appear only a handful of times. AI models trained on such uneven data tend to perform poorly on these rare events, precisely the situations where automated support could matter most. We address this problem by generating realistic synthetic surgical videos of the underrepresented actions using a video generation model called SurgiFlowVid. Our model learns not only what surgical scenes look like but also how instruments and tissue move over time. It can also be guided by a few real frames or instrument outlines, giving users control over what appears in the generated videos without requiring expensive frame-by-frame annotations. When we combined our synthetic videos with real training data across three different surgical datasets and tasks, AI performance on rare actions improved by 10–20%, suggesting that targeted video generation is a promising approach for building more reliable surgical AI systems.