I will present our recent works related to the three AI pillars of a self-driving car: perception, prediction, and planning. For the perception pillar, I will present new human pose estimation and monocular distance estimation methods that use a loss that learns its own confidence, the Laplace loss. For prediction, I will show our investigations on interpretable models where we apply deep learning techniques within structured and hand-crafted classical models for path prediction in social contexts. For the third pillar, planning, I will show our crowd-robot interaction module that uses attention-based representation learning suitable for planning in an RL environment with multiple people.