InfraRL: A Benchmark for Constrained Resource Allocation in Large-Scale Infrastructure Asset Management

Optimizing maintenance strategies for large-scale infrastructure is a critical sequential decision-making problem. While Reinforcement Learning (RL) offers a theoretical framework for such problems, practical deployment necessitates offline constrained RL—learning policies solely from static historical datasets under rigid budgetary limits without dangerous on-policy exploration. However, current research is hindered by benchmarks that fail to capture the confluence of distributional shift and hard constraints typical of real-world assets. We introduce InfraRL, a high-fidelity benchmark that uses bridge maintenance as a rigorous testbed for general infrastructure asset management challenges. Constructed from the U.S. National Bridge Inventory, InfraRL defines a rigorous offline task for optimizing maintenance strategies under hard budgetary constraints. We benchmark a diverse suite of baselines, ranging from industry-standard heuristics to single-agent, multi-agent, planning-based, and constrained offline RL methods. Through a comprehensive evaluation protocol, we analyze performance across structural utility, constraint adherence, and behavioral fidelity, revealing critical trade-offs between safety and long-term efficiency. Our code and data are available at https://github.com/BriSky-2021/InfraRL.

Infrastructure such as bridges requires regular maintenance, but agencies often face limited budgets and must decide which repairs to perform each year. Making these decisions is difficult because the benefits of maintenance may only become visible many years later, and poor decisions can lead to higher future costs and infrastructure deterioration. To help researchers develop and evaluate better decision-making methods, we introduce InfraRL, a benchmark built from more than 30 years of real bridge management records from the U.S. National Bridge Inventory. The benchmark provides a realistic test environment where computer algorithms must learn to allocate limited maintenance resources while balancing long-term infrastructure health and budget constraints. We evaluate a wide range of existing AI approaches, from methods that imitate historical decisions to methods that estimate long-term consequences of maintenance actions. Our results show that approaches that reason about future outcomes generally make better use of available budgets than simple imitation-based methods. We also find that combining data-driven learning with domain knowledge from infrastructure management leads to the most reliable long-term performance. InfraRL is intended as a shared research benchmark rather than a deployment-ready decision system. By providing realistic data, evaluation protocols, and baseline results, we hope it will help accelerate research on trustworthy AI tools for infrastructure planning and other public-resource allocation problems.