Rapid Fixes, Gradual Failures: Exploring Iterative Self-Correction Dynamics in Large Language Models for Program Synthesis

Achieving one-shot success in code generation remains challenging for large language models (LLMs). Iterative self-repair, in which models refine their own programs in response to feedback, has been proposed as a potential remedy, but its effectiveness for programming tasks is not well understood. We study the dynamics of this loop across three frontier LLMs (GPT-4, GPT-5.1, Claude Sonnet 4.5), four execution-based benchmarks (HumanEval, MBPP+, APPS-Intro, APPS-Comp), two languages (Python and Java), and two history conditions (h = 0: critique only; h = 1: full refinement history). Three findings emerge. First, recovery is overwhelmingly a task-level property: whether at least one of the model's five initial samples already passes the task (its pass@1) predicts seed-level recovery far more strongly (ρ = 0.71) than the failing seed's own initial pass fraction (ρ = 0.31). Failing seeds on tasks where at least one other initial sample already passed are 36× more likely to recover than failing seeds on tasks where every initial sample failed. Second, refinement is heavily front-loaded: 53% of the total pass-fraction gain occurs at attempt 1, recovered and non-recovered seeds diverge immediately (a 46 percentage-point gap in pass fraction by attempt 1), and a substantial fraction of non-recovered seeds (~38% at h = 0, ~49% at h = 1) reproduce the same wrong program for ten attempts. Third, adding the full refinement history does not improve recovery rate (p > 0.35) and on harder benchmarks actively degrades it (-7.5 percentage points on APPS-Comp).