A key aspect of human intelligence is their ability to convey their knowledge to others in succinct forms. However, current machine learning models are largely blackboxes that are hard for humans to learn from. We study the problem of whether we can design machine learning algorithms capable of conveying their insights to humans in the context of a sequential decision making task. In particular, we propose a novel machine learning algorithm for extracting interpretable tips from a policy trained to solve the task using reinforcement learning. In particular, it searches over a space of interpretable decision rules to identify the one that most improves human performance. Then, we perform an extensive user study to evaluate our approach, based on a virtual kitchen-management game we designed that requires the participant to make a series of decisions to minimize overall service time. Our experiments show that (i) the tips generated by our algorithm are effective at improving performance, (ii) they significantly outperform the two baseline tips, and (iii) they successfully help participants build on their own experience to discover additional strategies and overcome their resistance to exploring counterintuitive strategies.