CURVE: Learning Causality-Inspired Invariant Representations for Robust Scene Understanding via Uncertainty-Guided Regularization

Scene graphs provide structured abstractions for scene understanding, yet they often overfit to spurious correlations, severely hindering out-of-distribution generalization. To address this limitation, we propose CURVE, a causality-inspired framework that integrates variational uncertainty modeling with uncertainty-guided structural regularization to suppress high-variance, environment-specific relations. Specifically, we apply prototype-conditioned debiasing to disentangle invariant interaction dynamics from environment-dependent variations, promoting a sparse and domain-stable topology. Empirically, we evaluate CURVE in zero-shot transfer and low-data sim-to-real adaptation, verifying its ability to learn domain-stable sparse topologies and provide reliable uncertainty estimates to support risk prediction under distribution shifts.

Modern AI systems need to understand complex scenes, such as which objects are present and how they interact. Scene graphs help by representing a scene as objects and relationships, like car near pedestrian or person crossing road. However, models often learn shortcuts from training data. For example, they may wrongly assume that certain objects always appear together or that certain relationships only happen in familiar environments. These shortcuts can make the model unreliable when it is used in new or changing conditions. We propose CURVE, a method that teaches the model to focus on stable relationships and reduce the influence of uncertain, environment-specific ones. It also makes the scene graph simpler by keeping only the most reliable connections. CURVE helps AI systems understand new or changing environments more reliably. It is especially useful when little target-domain data is available and can support safer risk prediction under distribution shifts.