This talk will be about some recent works on the sample complexity of pure-exploration in an episodic MDP, when we sequentially collect trajectories, with or without rewards, and aim at identifying good policies. In particular, I will describe an optimal algorithm for reward-free exploration called RF-Express, which interestingly features some exploration bonuses that scale in 1/n instead of the usual 1/\sqrt{n}.