Spatially-Adaptive Gradient Re-parameterization for 3D Large Kernel Optimization

Large kernel convolutions offer a scalable alternative to vision transformers for high-resolution 3D volumetric analysis, yet naively increasing kernel size often leads to optimization instability. Motivated by the spatial bias inherent in effective receptive fields (ERFs), we theoretically demonstrate that structurally re-parameterized blocks induce spatially varying learning rates that are crucial for convergence. Leveraging this insight, we introduce Rep3D, a framework that employs a lightweight modulation network to generate receptive-biased scaling masks, adaptively re-weighting kernel updates within a plain encoder architecture. This approach unifies spatial inductive bias with optimization-aware learning, avoiding the complexity of multi-branch designs while ensuring robust local-to-global convergence. Extensive evaluations on five 3D segmentation benchmarks demonstrate that Rep3D consistently outperforms state-of-the-art transformer and fixed-prior baselines. The source code is publicly available at https://github.com/leeh43/Rep3D.

AI tools are increasingly used to identify organs and tumors in 3D medical scans such as CT and MRI, but reliable segmentation requires understanding each region in the context of the surrounding anatomy. A natural way to give an AI model this wider view is to enlarge the small sliding window it uses to inspect a scan. In practice, however, simply enlarging this window into a large 3D cube of voxels often stops helping, because large 3D filters become difficult to train effectively. We identify an important reason for this failure: during training, the center of a large 3D window receives stronger learning signals than its edges, leaving the outer parts under-trained. We introduce Rep3D, which uses a lightweight training-time helper to rebalance how different parts of the large window learn. After training, this helper is removed, so the deployed model remains a plain convolutional network. Rep3D achieves state-of-the-art accuracy on five public 3D medical segmentation benchmarks covering CT and MRI scans of organs, kidneys, pancreas, tumors, and liver vessels. These results suggest a practical path toward medical AI models that can use larger anatomical context while keeping deployment simple.