We systematically investigate the expressive power of path-based graph neural networks. While it has been shown that they can achieve strong empirical results, an investigation into their expressive power is lacking. Therefore, we propose PATH-WL, a general class of color refinement algorithms based on paths and geodesic distance information. We characterize families of graphs that can be distinguished by PATH-WL. For a sufficient path length, PATH-WL is incomparable to a wide range of expressive graph neural networks, can count cycles, and achieves strong results on the notoriously difficult family of strongly regular graphs. Our theoretical results indicate that PATH-WL forms a new hierarchy of highly expressive graph neural networks.