Deep Discriminative Structure Proxy Hashing for Cross-modal Retrieval

Existing proxy-based hashing methods optimize samples toward independently learned proxies using isolated similarity constraints. Although efficient, this design overlooks the fact that proxies are learned jointly but lack explicit relational or competitive interactions during optimization. Consequently, proxy responses to a sample are often accumulated rather than contrasted, leading to weakly defined decision regions and limited discriminative structure in the Hamming space. In contrast, our method organizes multiple proxies into sample-specific relational structures, enabling proxies to interact and compete when responding to each sample. Through structure-guided learning, these interactions explicitly contrast positive and negative proxy responses, thereby shaping clearer and more discriminative decision boundaries. Extensive experiments on standard cross-modal benchmarks demonstrate that this structured discrimination consistently improves retrieval accuracy and embedding separability. The source code is available at .

When you search for images using a text description — or find related text given a photo — the system must bridge two fundamentally different types of data. Modern large-scale systems handle this by converting both images and text into short binary strings (sequences of 0s and 1s), so that similar content lands close together in this compact space. A key challenge is making these strings genuinely discriminative: semantically unrelated content should be clearly separated, not lumped together. Existing methods use learnable "proxies" — one representative anchor per category — to supervise this process. However, when a single image belongs to multiple categories simultaneously (a dog running by a lake, for instance), all relevant proxies independently pull the image toward themselves without any coordination, causing their influence to pile up rather than compete. This blurs the boundaries between categories in the learned space. We propose organizing the relevant proxies for each image into a small relational structure, where their responses are coupled and implicitly compete for influence rather than simply accumulating. This structured interaction produces cleaner category boundaries without requiring more proxies or heavier computation — in fact, our method trains faster than simpler baselines. Better-separated representations mean more accurate and reliable cross-modal search, which matters for applications ranging from medical image retrieval to large-scale multimedia databases.