Spherical SO(3) Equivariant Local Attention

Spherical signals provide a natural representation for omnidirectional perception and often benefit from equivariance to 3D rotations. Recent spherical vision transformers implement local self-attention on spherical grids, but most retain only partial $\mathrm{SO}(3)$ equivariance and rely on $\textit{location-dependent}$ local positional embeddings (LPEs). Such LPEs can degrade robustness to camera tilt or object reorientation and introduce additional memory and computational overhead. We propose $\textit{Spherical $\mathrm{SO}(3)$-Equivariant Local Attention}$ (SoLA), an LPE-free local attention mechanism for spherical signals. SoLA achieves full $\mathrm{SO}(3)$ equivariance through a distance-preserving positional modulation that couples query/key features with each token’s unit direction. Specifically, the modulation lifts queries and keys using an outer-product with the 4D direction dependent vector. The induced similarity of the modulated queries and keys depends on content affinity and great-circle distance while remaining invariant to global $\mathrm{SO}(3)$ rotations. The same formulation admits a softmax-free linear variant that computes local attention via key-value aggregation without per-query neighbor materialization. We integrate SoLA into a U-shaped spherical transformer for $360^\circ$ depth estimation and semantic segmentation, demonstrating substantially improved robustness to arbitrary 3D rotations compared to prior spherical transformers with similar computational costs.

Many modern technologies, such as autonomous vehicles, robots, weather forecasting systems, and 360-degree cameras, collect information that naturally covers an entire sphere rather than a flat image. However, most AI systems are designed for flat images and can become unreliable when the camera or the observed scene is rotated. For example, the same object may be interpreted differently simply because it appears at a different orientation. We introduce a new AI method that helps models understand spherical data more consistently. Instead of relying on location-specific rules that depend on where an object appears, our approach represents spatial relationships using the geometry of the sphere itself. As a result, the model responds similarly when the entire scene is rotated, making it more robust to changes in viewpoint and orientation. We incorporate this idea into a transformer-based architecture, a widely used type of AI model. Our method is also computationally efficient, avoiding several costly operations used in previous approaches. Experiments on 360-degree scene understanding, scientific simulation, and image classification tasks show that the proposed method maintains strong performance even when evaluated on rotations that were never seen during training. These results suggest that AI systems can become more reliable and adaptable by respecting the underlying geometry of spherical data. This may benefit applications ranging from robotics and autonomous driving to Earth observation and scientific computing.