Can Large Language Models Generalize Procedures Across Representations?

Large language models (LLMs) are trained and tested extensively on symbolic representations such as code and graphs, yet real-world user tasks are often specified in natural language. To what extent can LLMs generalize across these representations? Here, we approach this question by studying isomorphic tasks involving procedures represented in code, graphs, and natural language (e.g., scheduling steps in planning). We find that training LLMs with popular post-training methods on graphs or code data alone does not reliably generalize to corresponding natural language tasks, while training solely on natural language can lead to inefficient performance gains. To address this gap, we propose a two-stage reinforcement learning curriculum that first trains on symbolic, then natural language data. The curriculum substantially improves model performance across model families and tasks. Remarkably, a 1.5B Qwen model trained by our method can closely match zero-shot GPT-4o in naturalistic planning. Finally, our analysis suggests that successful cross-representation generalization can be interpreted as a form of generative analogy, which our curriculum effectively encourages. The dataset and code used in this paper can be found \href{https://github.com/fangru-lin/proceduregeneralizationllm}{here}.

People ask AI assistants to do various tasks in natural language, such as 'plan a schedule for me', but many AI models are primarily optimized for symbolic tasks like code and graphs. This means that models can have a gap between the train and test representations. In this paper, we ask whether learning a problem-solving procedure in these tasks can directly help models solve similar natural language tasks. If not, how can we mitigate the gap? We use popular post-training methods, including supervised fine-tuning and reinforcement learning, to train models such as Qwen and LLaMA on one representation, then test on multiple representations. We find that when models are optimized on code or graph, they do not carry the capability for natural problem-solving. To mitigate this issue, we propose a two-stage reinforcement learning curriculum where we first train on code or graphs, then adapt to natural language. We find that this method helps elicit analogical reasoning in models, enabling them to apply learned symbolic procedures in natural-language domains. More broadly, our method suggests a new paradigm for post-training large language models: not only scaling data and models, but also deliberately scaling the variety of representations.