Riemannian stochastic optimization for sufficient dimension reduction

Gradient-based sufficient dimension reduction methods face a fundamental tradeoff: computing local gradients in the ambient space yields closed-form solutions but suffers from the curse of dimensionality, while iterative refinement in the projected space improves statistical efficiency at $O(n^2 p)$ cost per iteration. We show that minimizers of the Minimum Average Variance Estimation (MAVE) criterion recover the span of the regression function's gradients---the same target as the Outer Product of Gradients method---but through local regression in the projected space. We then reformulate MAVE as a Riemannian maximization problem on the Stiefel manifold and derive a closed-form gradient, enabling efficient stochastic optimization. The resulting algorithm, SMAVE, combines mini-batch Riemannian gradient ascent with adaptive $k$-nearest neighbor localization that evolves with the subspace estimate. On synthetic benchmarks, SMAVE matches or exceeds the accuracy of existing methods while running 10--50$\times$ faster; on real regression tasks, these gains translate to improved prediction with speedups exceeding three orders of magnitude.