RetrOrchestrator: A Multi-Step Retrosynthesis Agent Dynamically Orchestrating Single-Step Transition Models

Multi-step retrosynthesis planning is a fundamental challenge in organic chemistry, defined by its enormous search space. Existing methods typically formulate it as a Markov Decision Process (MDP) with a fixed choice of transition model (i.e., a single-step retrosynthesis model), and focus on improving how to search through better policies and value functions. However, how the transition space itself is navigated remains largely unexplored. This limitation is particularly urgent given our observation of pronounced skill disparity among single-step prediction models: different models exhibit substantially different performance across molecule states. Motivated by this observation, we introduce RetrOrchestrator, an LLM-powered agent that explicitly accounts for model skill disparity by reframing retrosynthesis planning as a Partially Observable Markov Decision Process (POMDP). By regarding each single-step prediction model as a tool, we further propose a scaffold-aware reinforcement learning algorithm to optimize navigation policy within the transition space. As a result, RetrOrchestrator jointly searches which molecule to expand and which single-step model to apply for the molecule at the current step. Empirically, RetrOrchestrator significantly outperforms static baselines on the Retro*-190 benchmark, achieving a state-of-the-art 94.21% success rate (vs. 9.47% off-the-shelf LLM and 82.63\% non-LLM state-dependent router), with 92.49% of solved routes invoking two or more SSRs—evidence that the policy is not collapsing to a single specialist or a static router. The same gain persists on a larger out-of-distribution set (PDB-600), with RetrOrchestrator Pareto-optimal in both wall-clock time and model-query count. Code: https://github.com/ScottLiao920/verl-retro-agent.

Discovering new medicines and materials begins with a hard puzzle: given a target molecule, work backwards to find a sequence of chemical reactions that builds it from cheap, readily available ingredients. Researchers have developed many AI tools that each propose possible one-step reactions, but no single tool works best for every kind of molecule. Existing planners either pick one specialist and stick with it, or query every tool at every step — wasting time and computation while still missing hard targets. We built RetrOrchestrator, an AI agent powered by a large language model that learns when to call which chemistry tool at each step of the search, like a project manager deciding which expert to consult next. To train it, we developed a learning method that compares the agent's choices on chemically similar molecules, so it can tell which routing decisions actually helped. RetrOrchestrator solves 94% of the hardest benchmark targets — beating the strongest previous system, which uses a model nearly 450× larger — while issuing roughly half the chemical-tool queries of competing methods. It also generalizes to drug-like molecules it has never seen, suggesting language-model agents can speed up real-world discovery of new medicines and materials.