Dwarf galaxies are small dark matter-dominated galaxies, some of which are embedded within the Milky Way; their lack of baryonic matter (stars and gas) makes them perfect testbeds for dark matter detection. Understanding the distribution of dark matter in these systems can be used to pin down microphysical dark matter interactions that influence the formation and evolution of structures in our Universe. We introduce a new approach for inferring the dark matter density profiles of dwarf galaxies from the observable kinematics of stars bound to these systems using graph-based machine learning. Our approach aims to address some of the limitations of established methods based on dynamical Jeans modeling such as the necessity of assuming equilibrium and reliance on second-order moments of the stellar velocity distribution. We show that by leveraging more information about the available phase space of bound stars, this method can place stronger constraints on dark matter profiles in dwarf galaxies and has the potential to resolve some of the ongoing puzzles associated with the small-scale structure of dark matter halos.