Skip to yearly menu bar Skip to main content


Poster
in
Affinity Workshop: New In ML

Geometry of Rank Constraints in Shallow Polynomial Neural Networks

Param Mody · Maksym Zubkov


Abstract:

We study shallow quadratic polynomial networks of width 2. Across 500 random initialisations, we observe a universal two-phase dynamic: rapid projection onto the rank-2 variety followed by slow diffusion along its flat directions. Rank-mismatch traps optimisation on a 2-D ellipse and imposes a non-zero loss floor but when the target rank matches capacity, the manifold collapses to three minima clusters where one reaches zero loss.

Chat is not available.