PGS: Effective LLM Code Refinement via Property-Oriented and Structurally Minimal Feedback

Large Language Models (LLMs) excel at code generation, yet ensuring the functional correctness of their outputs remains a persistent challenge. Recent studies have applied Test-Driven Development (TDD) to refine code, leveraging execution feedback to guide the model toward correct solutions. However, such feedback is often noisy and uninformative, stemming from the scarcity of high-quality test cases and the abundance of noisy, auto-generated ones. In this work, we shift the focus from test-case generation to feedback quality. We introduce the Property-Generated Solver (PGS), a novel feedback-centric framework designed to generate highly effective feedback via two principles: it must provide semantic guidance beyond simple I/O mismatches through property validation, and be structurally minimal, to reduce cognitive load and isolate root causes. PGS operates by checking high-level program properties (e.g., a sorting function must produce a non-decreasing sequence) then providing the simplest failing counterexample to the LLM. This property-driven, minimal feedback steers LLMs toward correct and generalizable solutions. Across diverse benchmarks, PGS demonstrates superior performance, achieving a bug fix rate 1.4x-1.6x higher than the strongest debugging-based approaches and establishing a new state-of-the-art in automated code refinement.

Large language models can write programs, but their code often contains hidden bugs. Current approaches try to find these bugs by running many tests, yet the feedback is frequently noisy or too complicated, leaving the model confused rather than enlightened. We propose a new strategy: instead of overwhelming the model with countless tests, we focus on making the feedback itself clearer and more meaningful. Our system checks whether the code follows basic logical rules, such as "a sorted list must be in ascending order", and then highlights the simplest example where the rule fails. This precise, easy-to-follow guidance helps the model grasp the real problem and fix it correctly, leading to much higher success in repairing faulty code.