Geometric Collapse: When Vision Models Fail to Verify Physical Causality

Recent progress in large-scale self-supervised learning has improved dense geometric prediction, but it remains unclear whether such scaling yields inference-time physical plausibility checks. We propose Scrambled Edges, a controlled counterfactual that injects salient edge-like cues while violating surface continuity, illumination coherence, and occlusion ordering. With energy-matched and structure-matched controls, we isolate the effect of unsupported edge evidence from high-frequency energy and edge sparsity. Across CNN/ViT/SSL depth predictors on NYU Depth v2 and KITTI, Scrambled Edges induce up to 3.2× larger deviation from clean predictions than energy-matched noise. The resulting Geometric Collapse propagates globally: even with oracle knowledge of the corrupted region, output-level repair recovers only 47%, with substantial error outside the mask. These findings provide controlled behavioral evidence that current dense predictors lack reliable mechanisms to quarantine physically unsupported edge cues, motivating explicit plausibility scoring and selective cue integration.

Vision models can estimate 3D scene structure from a single image, but they may trust visual edges even when those edges do not make physical sense. We test this by adding misleading edge-like patterns that look strong but do not match real surfaces, lighting, or object boundaries. We find that several depth prediction models treat these false edges as real structure, causing errors that spread across the whole scene rather than staying near the added edges. This failure, which we call Geometric Collapse, is not simply ordinary noise sensitivity. It shows that current models may lack a reliable way to reject physically unsupported visual cues. This matters for applications such as robotics and autonomous navigation, where wrong 3D predictions could lead to unsafe decisions. Our work provides a simple test for this problem and suggests that future models should check whether visual cues are physically plausible before using them.