CVarGD: Coreset-Variance-Aware Gradient Descent for Data-Scarce Image Classification

Training deep neural networks under data scarcity is fundamentally challenging: gradient estimates from small coresets exhibit variance scaling as O(1/k), destabilizing optimization. We propose CVarGD (Coreset-Variance-aware Gradient Descent), combining (i) AMSGrad-based variance stabilization, (ii) two-timescale gradient memory blending fast and slow EMAs, and (iii) a coreset-aware learning-rate boost scaling by $\mathcal{O}(1/k)$ and $\rho^{-0.25}$ (capped at 4x). Evaluated on CIFAR-10, BloodMNIST, and EuroSAT with ResNet-18 against vanilla GD, SGD, mini-batch GD (5 batch sizes), and AdamW, CVarGD outperforms all baselines on BloodMNIST at 5% coreset (87.81% vs. 86.67%) and 25% coreset (94.62%). An ablation confirms variance stabilization is the most critical component.