Butterworth as Attention: Anisotropic Spectral Gating for Pansharpening

Pansharpening fuses high-resolution panchromatic (PAN) images with low-resolution multispectral (LMS) images. For spatial-spectral fusion, Fast Fourier Transform (FFT)-based methods provide a global receptive field to capture long-range dependencies and naturally separate frequency components. However, most existing approaches directly transplant spatial operators like convolution or self-attention, while disregarding the fundamental structure of the spectrum: a strict spatial correspondence where each coordinate represents a specific frequency component, and a highly non-uniform, radially decaying energy distribution. To address this, we revisit the classical Butterworth filter, a frequencydomain operator defined directly on spectral coordinates that is inherently suited for processing such structured representations. We generalize the standard isotropic Butterworth filter into an anisotropic, learnable frequency-domain gating mechanism, establishing an efficient alternative to self-attention, and propose the Anisotropic Butterworth Fusion Network (ABFNet). Its core is a novel dual-branch gating module that employs learnable anisotropic Butterworth filters to perform adaptive direction-aware feature selection, integrating global context and local details with linear complexity. Extensive experiments show that ABFNet achieves state-of-the-art (SOTA) performance on pansharpening benchmarks with low computational overhead. Furthermore, its superior accuracy on CIFAR-100 classification validates the broader applicability of this frequency-domain learning paradigm.

Satellite images are widely used in environmental monitoring, urban planning, agriculture, and disaster response. However, different satellite sensors often capture different types of information: one image may show fine spatial details, while another may contain richer color or material information but with lower resolution. Pansharpening aims to combine these images into a clearer and more informative high-resolution image. This paper proposes a new way to improve this image-combination process. Instead of directly using common deep learning operations designed for ordinary images, we process the image information in a frequency-based representation, where smooth regions and fine details can be more naturally separated. Inspired by a classical signal-processing filter, our method learns how to selectively preserve and enhance important image details, especially directional structures such as roads, building edges, and field boundaries. The proposed model produces sharper and more accurate satellite images while keeping the computational cost low. Experiments on several widely used satellite datasets show that it performs better than existing methods in many settings. The same core idea also works well on a general image classification task, suggesting that this frequency-based design may be useful beyond satellite image fusion.