SynthonBench: Benchmarking Sample-Efficient Optimization in Combinatorial Chemical Spaces

Synthesis-on-demand libraries expose vast chemical spaces through reaction templates and compatible synthons. In this setting, the central question is not whether a generator can emit valid molecules, but which candidates to evaluate under a small oracle budget. We introduce SynthonBench, a benchmark for reaction-native, budgeted search in synthon-based libraries. Optimizers propose valid reaction/synthon tuples and receive oracle scores; after each run, stable product identifiers are joined to reference tables to compute top-$k$ recall, enrichment, regret, and two-objective Pareto metrics. SynthonBench includes exact-audit 1M tasks, 10M scale tasks, and single-seed 100M feasibility runs covering docking-surrogate, selectivity, and synthetic diagnostic objectives. Across nine baselines and five seeds on the 1M/10M suites, genetic algorithms and Thompson-style synthon sampling are strongest on docking-surrogate tasks, while factorization machines excel on pairwise synthetic objectives. On 1M docking, the best baseline reaches mean top-100 recall 0.645 versus 0.178 for random search; on 10M, 0.321 versus 0.039. The benchmark exposes where search methods exploit synthon structure and where larger reaction spaces change the leaderboard.