Research at the intersection of physics and machine learning has outsized potential to advance both fields: models and algorithms can be embedded with, or informed by, physics knowledge, and learned models as well as simulators of complex processes and systems can advance experimentation towards understanding. In practice, this physics-ML intersection is almost entirely ML surrogates for accelerating an existing numerical simulator. But can we look to ML & AI to discover physics knowledge we don't yet have governing equations for, to recover missing physics and fill gaps in human-expert understanding? It is this perspective we explore in this talk, in particular the use of agent-based modeling (ABM) as a new abstraction for computational fluid dynamics (CFD), pulling in advanced reinforcement learning (RL) methods from the AI field. We introduce multi-agent RL as an automated discovery tool of turbulence and other fluid dynamics models, leveraging the emergent phenomena of ABM to surface the unresolved subgrid-scale physics. These methods, although nascent, can significantly advance prediction and control of industrial aerodynamics and environmental flows in critical areas like nuclear fusion plasma and atmospheric transport of contaminants.