Task-Restricted Symmetries in Recurrent Weight Space

Weight-space symmetries are usually studied as exact transformations that preserve a network's function, but recurrent networks also contain task symmetries over time whose learned implementations need not form exact parameter symmetries. We study this issue in one-layer tanh RNNs trained on a fixed-length copy task and three neuroscience-style recurrent tasks. Our analysis uses an ordered real Schur decomposition to place each recurrent matrix in a common coordinate system that separates the leading spectral rotation, its complement, and directed nonnormal interactions. We then ablate these Schur-coordinate interactions without refitting input or readout weights and measure the effect on autonomous rollout. In the copy task, a dense representative solves the task through complement-linked interactions, so some Schur directions are nearly redundant while others sharply change replay behavior. A Cayley-transform representative is nearly invariant to the same interventions. Across flip-flop, sine generation, and context-dependent integration, the fragile Schur directions depend on the task. Together, the copy and cross-task results support a task-restricted view of recurrent weight-space symmetry in which approximate equivalences are defined by rollout behavior and fragile directions vary with the computation.