Symmetry Acquisition in Predictive Coding Networks

Predictive coding networks (PCNs) perform inference through recurrent, bidirectional dynamics, requiring representations that support both abstraction and approximate inversion. We study how PCNs learn representations through the lens of weight-space symmetry, using persistent homology to track when and where PCN layers merge previously distinct regions of the data manifold. This provides a topological probe of representation-space identifications induced by the network’s learned dynamics. We find that this merging occurs across layers, but its timing is strongly controlled by model capacity ($0.72 \leq \rho \leq 0.79$) and activation smoothness. Critically, networks that merge representations earlier perform worse in input reconstruction tasks ($\rho = -0.58$). This reveals a measurable cost of early representational symmetry acquisition in generative architectures, and demonstrates that topological measures can expose structural properties of learned representations that loss values alone cannot.