Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory

We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical analyses of pairwise associative memory. We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity $M = e^{\alpha N}$, comparing Gaussian (LSE) and Epanechnikov (LSR) kernels. For continuous neurons on an $N$-sphere, the geometric entropy depends solely on the spherical geometry, not the kernel. In the sharp-kernel regime, the maximum theoretical capacity $\alpha = 0.5$ is achieved at zero temperature; below this threshold, a critical line separates retrieval from non-retrieval. The two kernels differ qualitatively in their phase boundary structure: for LSE, a critical line exists at all loads $\alpha > 0$. For LSR, the finite support introduces a threshold $\alpha_{\text{th}}$ below which no spurious patterns contribute to the noise floor, and no critical line exists -- retrieval is perfect at any temperature. These results advance the theory of high-capacity associative memory and clarify fundamental limits of retrieval robustness in modern attention-like memory architectures.

When you hear a few notes of a song, you can often recall the whole tune. Modern AI does the same: give it a snippet of text, and it pulls up related information using a mechanism called "attention." A simpler math model, Dense Associative Memory, describes this same recall from a partial cue, and one step of it works exactly like one step of attention — so it is a clean way to study how attention behaves. We asked how well such a memory holds up against random noise from the environment, and found that the answer depends on one design choice: how the model decides what counts as "close" to a stored memory. The smooth rule used in normal attention can always be broken by enough noise. But a rule with a hard cutoff — one that ignores any memory that is too far away — cannot be broken by noise at all, as long as the memory is not overloaded. This points to a concrete way to build noise-proof memory in AI.