ProOPF: Benchmarking and Improving LLMs for Professional-Grade Power Systems Optimization Modeling

Growing renewable penetration introduces substantial uncertainty into power system operations, necessitating frequent adaptation of dispatch objectives and constraints and challenging expertise-intensive, near-real-time modeling workflows. Large Language Models (LLMs) provide a promising avenue for automating this process by translating natural-language (NL) operational requirements into executable optimization models via semantic reasoning and code synthesis. Yet existing LLM datasets and benchmarks for optimization modeling primarily target coarse-grained cross-domain generalization, offering limited, rigorous evaluation in power-system settings, particularly for Optimal Power Flow (OPF). We therefore introduce \textbf{ProOPF-D} and \textbf{ProOPF-B}, a dataset and benchmark for professional-grade OPF modeling: ProOPF-D contains 12K instances pairing NL requests with parameter adjustments and structural extensions to a canonical OPF, together with executable implementations; ProOPF-B provides 121 expert-annotated test cases with ground-truth code, enabling end-to-end evaluation under both concrete and abstract OPF modeling regimes. Our code, dataset, and benchmark are publicly available at \href{https://github.com/shenchao188/ProOPF-Benchamrk-Dataset}{this GitHub repository}.

Modern electric grids are becoming harder to operate because more power now comes from renewable sources such as wind and solar, whose output can change quickly with weather. When grid conditions change, engineers often need to update planning and optimization models that help decide how electricity should be generated and delivered safely. This process requires expert knowledge and can be too slow when decisions must be made in near real time. We study whether large language models can help by turning plain-language instructions from engineers into working code for these power-grid optimization models. To do so, we introduce ProOPF-D, a dataset of 12,000 examples that connect human requests with the corresponding model and code changes, and ProOPF-B, a set of 121 expert-designed test cases. Our results show that today’s language models can handle some simple, explicit changes, but often fail when they must infer missing details or make deeper changes to the model. Training on our dataset improves performance, suggesting a path toward more reliable AI assistants for power-system operation.