Loop or Leap? Benchmarking Iterative vs Recursive Reasoning in Code LLMs with LRLBench
Abstract
Code-generation benchmarks report a single aggregate pass rate and provide no signal as to how a solution was obtained: a model scoring 70% on HumanEval could be reasoning about the underlying algorithm or emitting a memorised template, and the aggregate score does not distinguish the two. Most classical problems admit both iterative and recursive formulations, yet no existing benchmark evaluates whether a model can be constrained to either paradigm on demand. We introduce LRLBench, a paired benchmark of 3,000 LeetCode and Codeforces problems, each with a verified iterative solution, a verified recursive solution, synthetic test cases, and a full identifier rename via abstract syntax tree (AST) transformation. We evaluate four instruction-tuned large language models (LLMs) from 4B to 31B parameters on two tasks: paradigm-constrained code generation with 3 different prompting methods (iterative-forced prompt, recursive-forced prompt, free prompt) and program tracing on clean and renamed code. On code generation, forcing recursion lowers the share of fully correct solutions by 7.7-12.1 pp on Llama-3.1-8B, Mistral-Small-24B, and Gemma-4-31B, but only 0.4 pp on Qwen3-4B-Instruct. With no paradigm hint, models behave almost identically to the iterative-forced condition, identifying iteration as the default prior. On code tracing, post-training matters more than parameter count: Qwen3-4B reaches 61.4%, beating Llama-3.1-8B (20.5%) and Mistral-Small-24B (43.1%); Gemma-4-31B reaches 89.9%. The AST-based renaming degrades tracing accuracy for every model except Gemma-4-31B. We will open-source our code, dataset, and harness upon acceptance.