DisPOSE: Projected Polystochastic Diffusion for Self-Supervised Multi-View 3D Human Pose Estimation

Recovering 3D human poses for multiple individuals from different camera views is a fundamental bottleneck for analyzing interacting behaviors. Existing self-supervised approaches leverage synthetic catalogues of 3D poses; however, this leads to poor generalization in real-world scenarios due to distribution shifts. We therefore introduce DisPOSE, a self-supervised framework that approximates the inherently discrete multi-view person-assignment problem as a generative diffusion process over the space of polystochastic tensors. By employing differentiable Sinkhorn projections during denoising, our model learns to guide solutions toward valid and feasible assignments based on 2D image priors. The complete 3D skeletons of localized individuals are then regressed using a Hypergraph-Convolutional Decoder that explicitly models relational structures and articulated joints across multiple views. The proposed approach outperforms current state-of-the-art self-supervised methods on standard datasets and demonstrates strong performance on a newly proposed benchmark featuring highly occluded scenes from surgical operating rooms. Our diffusion-based localization demonstrates high label efficiency, retaining 99\% of its performance with only 10\% of the pseudo-labels. Notably, disentangling the assignment and root regression components while maintaining differentiability makes DisPOSE nearly agnostic to different camera arrangements.

Understanding how groups of people move together in 3D is important for many safety- and health-related tasks, such as surgery, robotics, and factory work. But teaching computers to do this is difficult. Methods for multi-camera systems often require expensive 3D labels or rely on fake poses that do not work well for real movements, such as a toddler crawling or a clinician kneeling at the operating table. We created DisPOSE, a system that learns from multiple cameras without needing 3D labels or fake poses. It works in two steps. First, it figures out which person in one camera view is the same person in the other views. Then, it builds that person's full 3D skeleton. The first step, inspired by AI image generators, begins with rough guesses about who matches who and gradually improves these guesses until they are geometrically consistent. DisPOSE achieves the highest accuracy among methods that learn without 3D labels. It also works with new camera setups it has not seen before and can handle challenging cases, like a crawling toddler or clinicians kneeling in a cluttered surgical room. This makes human-motion analysis possible in places where collecting labels is difficult but safety is very important.