Abstract: In this work, we consider the Hypothesis Transfer Learning (HTL) setting under adversarial attacks, where the learner has access to a training dataset of size $n$ from underlying distribution $\mathcal{D}$ and a set of auxiliary hypotheses. These auxiliary hypotheses can be viewed as the prior information, originating either from expert knowledge or other tasks, such as well-trained foundation models, and are employed as an initialization for the learning process. The goal is to obtain an adversarially robust model for $\mathcal{D}$. Our approach begins by exploring Adversarial Regularized Empirical Risk Minimization (ARERM). Assuming a non-negative smooth loss function with a strongly convex regularizer, we initially establish a robust generalization bound on the hypothesis returned by ARERM that depends on the quality of the initialization. If the initialization is good -- there exists a combination of auxiliary hypotheses with a small robust generalization loss -- then the robust generalization exhibits a fast rate of $\mathcal{O}(1/n)$, otherwise (for sub-optimal initialization) we recover the original $\mathcal{O}(1/\sqrt{n})$ rate. We also provide a robust excess risk bound with a slightly worst but similar nature rate. Our findings suggest that a curriculum-style adversarial training potentially yield a slightly tighter bound. We then consider solving the problem via a practical algorithm, namely proximal stochastic adversarial training, and present a initialization-dependent robust generalization bound that matches the same rate as applying ARERM in terms of the sample size but introduces an additional payoff that depends on the perturbation size.