Affine-Equivariant Kernel Space Encoding for NeRF Editing

Neural scene representations achieve high-fidelity rendering by encoding 3D scenes as continuous functions, but their latent spaces are typically implicit and globally entangled, making localized editing and physically grounded manipulation difficult. While several works introduce explicit control structures or point-based latent representations to improve editability, these approaches often suffer from limited locality, sensitivity to deformations, or visual artifacts. In this paper, we introduce Affine-Equivariant Kernel Space Encoding (EKS), a spatial encoding for neural radiance fields that provides localized, deformation-aware feature representations. Instead of querying latent features directly at discrete points or grid vertices, our encoding aggregates features through a field of anisotropic Gaussian kernels, each defining a localized region of influence. This kernel-based formulation enables stable feature interpolation under spatial transformations while preserving continuity and high reconstruction quality. To preserve detail without sacrificing editability, we further propose a training-time feature distillation mechanism that transfers information from multi-resolution hash grid encodings into the kernel field, yielding a compact and fully grid-free representation at inference. This enables intuitive, localized scene editing directly via Gaussian kernels without retraining, while maintaining high-quality rendering.

AI methods can now create realistic 3D scenes from photographs, but changing these scenes afterward is still hard. For example, if we want to move an object, bend it, or use a physics simulator to make it fall, stretch, or deform, the scene should continue to look natural from every camera view. Existing methods often struggle with this because the visual information inside the model is not stored in a way that is easy to move or reshape. We introduce a new way of organizing this visual information. Instead of keeping it in a fixed structure, our method stores it in many small, movable regions spread through the 3D scene. Each region carries information about the local appearance of the scene around it. When part of the scene is moved, rotated, or deformed, these regions can move with it, helping the edited scene remain smooth and visually consistent. During training, our method first learns detailed visual information from a powerful existing representation. After training, it no longer needs that fixed structure and can be edited more directly. This makes it possible to keep high-quality 3D rendering while supporting local edits, object manipulation, and physics-based effects such as rigid motion, soft deformation, and cloth-like motion. This could be useful for robotics, simulation, animation, and interactive 3D design.