HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs

Informal mathematics has been central to modern large language model (LLM) reasoning, offering flexibility and efficient construction of arguments. However, purely informal reasoning is prone to logical gaps and subtle errors that are difficult to detect and correct. In contrast, formal theorem proving provides rigorous, verifiable mathematical reasoning, where each inference step is checked by a trusted compiler, but lacks the exploratory freedom of informal problem-solving. This mismatch leaves current LLM-based math agents without a principled way to combine the strengths of both paradigms. In this work, we introduce Hermes, the first tool-assisted agent that explicitly interleaves informal reasoning with formally verified proofs in Lean. The framework performs intermediate formal checking to prevent reasoning drift and a memory module for proof continuity across multi-step reasoning chains, enabling both exploration and verification. We evaluate Hermes on four challenging mathematical reasoning benchmarks using LLMs of varying parameter scales, from small models to state-of-the-art systems. Across all settings, Hermes reliably improves the reasoning accuracy of base models while substantially reducing reasoning token usage and computational cost compared to reward-based approaches. On difficult datasets such as AIME and HARDMath2, Hermes@1 achieves up to a 40% accuracy improvement while using 80% fewer total inference FLOPs. When scaled at test time, Hermes@5 boosts accuracy further by 20%.

Computers use two different approaches to solve hard math problems. The first is "informal reasoning," which is flexible and creative, similar to how a human sketches out ideas on a scratchpad, but it is prone to making logical mistakes. The second is "formal theorem proving", where a computer program checks every single step with absolute certainty, but this method is often too rigid and has a steep learning curve. To combine the best of both worlds, we created HERMES, a new AI agent that lets computers switch back and forth between these two methods. HERMES allows the AI to freely explore ideas while using a trusted coding system, called Lean, to double-check its work along the way, ensuring it stays on the right track. When tested on difficult math competitions, HERMES significantly improved accuracy while drastically cutting down on the computer resources. We have made this tool free and publicly available for anyone to use.