Machine unlearning, the process of selectively removing knowledge from trained models, emerges as a crucial mechanism for maintaining model relevance and privacy. However, the effectiveness of unlearning hinges on the quality of training, a challenge exacerbated by sensitivity to outlier data. We introduce the first robust training approach to unlearning, called TrimGrad, tailored to address this challenge by minimizing the training loss on the worst-case retain set to ensure a sturdy initialization for subsequent unlearning. Our method comes with theoretical guarantees for losses satisfying the Polyak-Lojasiewicz inequality, whereas most prior machine unlearning guarantees apply only to convex losses. Through empirical evaluations, we demonstrate the seamless integration of our approach with various unlearning techniques, resulting in accelerated processes and enhanced overall performance.