Hyperbolic Associative Memory Networks

Modern Hopfield Networks (MHNs) have achieved widespread success across various domains but are confined to Euclidean/Hilbert spaces, failing to preserve the hierarchical structure of data due to geometric constraints—arbitrary tree structures cannot be embedded with low distortion, while hyperbolic spaces can naturally accommodate hierarchical structures through exponential volume growth. To address this issue, we propose Hyperbolic Associative Memory Networks (HAMNs), the first framework to embed modern associative memory into hyperbolic space: we map query and memory vectors from Euclidean space to a constant negative curvature manifold via exponential maps, define a regularized energy function based on the Minkowski inner product, and adopt curvature-aware Riemannian optimization combined with exponential map updates to achieve stable on-manifold retrieval. We put forward a hierarchy-sensitivity hypothesis—HAMNs outperform Euclidean MHNs on data with deep hierarchies but exhibit comparable performance on data with weak or shallow hierarchies, which is validated by depth-controlled experiments and cross-level metrics. As a plug-and-play, model-agnostic module, HAMNs are suitable for the storage and retrieval of representations in task architectures requiring hierarchical understanding, instantiated with the Poincar\'e ball in experiments, and also applicable to any hyperbolic model with constant negative curvature. The code is available at https://github.com/hbl66/HAMNs.

Many AI systems need to recognize not only individual items, but also how those items are organized in a hierarchy, such as animals, plants, categories in a knowledge base, or levels in a taxonomy. Standard memory-like components in AI usually work in flat spaces, which can make it hard to preserve these tree-like relationships. In this work, we design Hyperbolic Associative Memory Networks, or HAMNs, a memory component that stores and retrieves information in a curved space that is better suited to hierarchical data. The main idea is to let the model remember related concepts in a way that respects both their similarity and their position in a hierarchy. Our experiments show that this design helps most when the data have deep hierarchical structure, while it performs similarly to standard methods when such structure is weak or shallow. This makes HAMNs a useful add-on for AI systems that need to reason over taxonomies, knowledge graphs, hierarchical labels, or other tree-like information. The results suggest that choosing a geometry that matches the structure of the data can make machine learning models more reliable for hierarchical understanding.