The Gradient Does Not See Rank: Rank-Indifference in Matrix-CODI on ProsQA

Continuous chain-of-thought models compress reasoning into latent tokens. Matrix-valued variants introduce rank as a single-sample structural observable on the latent matrix Z. If matrix latents carry parallel reasoning paths via superposition, rank should track them, and truncating Z to low rank should hurt accuracy on tasks whose solutions plausibly require multiple components. Across four training regimes of a matrix-CODI model (three on ProsQA, one on GSM8K-Aug below the learning threshold), the rank-k projection ablation curve is flat to within 0.6 percentage points. A three-seed replication yields 81.5 plus or minus 1.2 percentage points accuracy while the final effective rank of Z spans the set {4, 12, 13}; the loss does not reward any particular rank. To test whether rank-blindness arises from the flatten-then-project readout alone, we trained four readouts: a bilinear reparametrization, a bilinear-plus-GELU readout nonlinear in Z, an SVD-augmented readout feeding singular values through an MLP, and a quadratic readout in Z Z transpose. All four rank-k curves remain flat with Spearman p-values of 0.63, 0.14, 0.82, and 0.46. The flat curves persist for readouts nonlinear in Z. A linear probe on Z underperforms a raw pretrained hidden state at target prediction, with AUC 0.673 versus 0.846. A negative control on vanilla GPT-2 SFT, with no matrix bottleneck and no Z, run over three seeds and 500 problems per seed, reproduces a flat rank-k curve under the same intervention paradigm with pooled-mean range 0.20 percentage points, and a random-h sensitivity floor lands at the same accuracy. The rank-k ablation alone conflates rank-blindness with position-irrelevance.