nD-RoPE: A Generalized RoPE for n-Dimensional Position Embedding

Rotary Position Embedding (RoPE) is widely adopted in Transformer models, yet its extension to high-dimensional domains lacks a unified theoretical formulation. Most existing approaches either apply rotations independently along each axis or empirically mix frequencies, which limits cross-dimensional interactions and yields direction-dependent representations. To address these limitations, we propose *nD-RoPE*, a decomposition-free generalization of RoPE to arbitrary dimensions. From a translation-invariant formulation in continuous Hilbert space, we derive a spectral condition for isotropy that requires treating positions and frequencies as coupled $n$-dimensional vectors. We instantiate this formulation with a multi-scale regular-simplex wave-vector design, which provides non-degenerate spatial coverage and a symmetric, directionally balanced second-order response. Experiments across images, videos, and point clouds demonstrate consistent performance gains and improved generalization in high-dimensional settings.

People understand the world by organizing relationships in space: we identify objects and people in images, track motion in videos, and navigate 3D environments using cognitive maps. Current AI models often encode position separately along each dimension, which can make it harder to capture rich spatial relationships. We propose nD-RoPE, a way to give AI models position information in spaces of any dimension. Instead of encoding positions only along fixed horizontal, vertical, or depth axes, our method spreads positional signals more evenly in all directions using a symmetric geometric pattern. This helps models handle high-dimensional data with a more unified sense of space.