Beyond Pairwise: Diagnosing Higher-Order Merge Failures via Hodge Decomposition
Dongzhe Zheng ⋅ Christine Allen-Blanchette
Abstract
Model merging implicitly assumes pairwise mergeability extends to higher-order combinations. On 24 weight-matched PlainCNNs (CIFAR-10), this fails: over 90\% of pairwise-mergeable triples are not jointly mergeable ($\beta_1$ peaks at 122). Hodge decomposition of the merge-barrier signal separates incompatibility into gradient, curl, and harmonic components. Star-topology alignment leaves 25.5\% of energy in the harmonic component; adjusting merge weights within the fixed alignment cannot reduce it. Re-aligning along a minimum spanning tree weighted by harmonic energy drives this to 1.8\% and collapses $\beta_1$ from 75 to 5, at a cost of 15\% higher total barrier. The composition moves out of the harmonic component into gradient and curl, which existing merging strategies already address. Whether these patterns hold at larger scales remains open.
Chat is not available.
Successful Page Load