In the real world, more and more customers view privacy as a concern when using an AI service, especially when the customer content consists of sensitive data. Recent research demonstrates that large language model like GPT-2 can memorize content, which can be extracted by an adversary. This poses high privacy risk in deployed scenarios when models are trained on customer data. Differential privacy is widely recognized as a golden standard of privacy protection due to its mathematical rigor. To alleviate the privacy concern in machine learning, many research works have studied the machine learning with differential privacy guarantee. It is the time to clarify the challenge and opportunity for learning with differential privacy. In this tutorial, we first describe the potential privacy risk in machine learning models and introduce the background of differential privacy, then present the popular approaches of guaranteeing differential privacy in machine learning. In the rest of the tutorial, we highlight the interplay between learning and privacy. In the second section, we show how to utilize the learning property to improve the utility of private learning, especially with recent advances towards solving these challenges by exploiting the correlation across data points and the low-rank property of the deep learning models. In the third section, we present the other direction of research, i.e., using the tools in differential privacy to tackle the classical generalization problem and we also present concrete scenarios of using ideas in differential privacy to resist attacks in machine learning.