Improving Code Efficiency with Iterative Refinement using LLMs

Large Language Models (LLMs) generate functionally correct but often computationally inefficient code, yet current evaluation paradigms rarely assess efficiency. We propose ICER (Iterative Code Efficiency Refinement), an iterative self-improvement framework that optimizes LLM-generated code using runtime and memory feedback together with execution error signals. Unlike prior methods that enforce strict, monotonic acceptance criteria---which we find prone to system noise and overfitting to seen unit tests---ICER adopts a relaxed update policy that prioritizes correctness and exploration, enabling convergence toward better accuracy-efficiency trade-offs on unseen tests. Ablation studies reveal a counter-intuitive result: explicitly prompting for "Big-O" complexity predictions degrades optimization performance in later iterations. To investigate this disconnect, we introduce a tri-component evaluation framework spanning complexity prediction, constrained generation, and code refactoring. Our analysis confirms that frontier models struggle with explicit complexity labeling yet possess strong implicit optimization capabilities, refactoring inefficient code to near golden solution's runtime on competitive programming benchmarks.