Language-based Trial and Error Falls Behind in the Era of Experience

While Large Language Models (LLMs) excel in language-based agentic tasks, their applicability to unseen, nonlinguistic environments (e.g., symbolic or spatial tasks) remains limited. Previous work attributes this performance gap to the mismatch between the pretraining distribution and the testing distribution. In this work, we demonstrate the primary bottleneck is the prohibitive cost of exploration: mastering these tasks requires extensive trial-and-error, which is computationally unsustainable for parameter-heavy LLMs operating in a high dimensional semantic space. To address this, we propose SCOUT (Sub-Scale Collaboration On Unseen Tasks), a novel framework that decouples exploration from exploitation. We employ lightweight "scouts" (e.g., small MLPs) to probe environmental dynamics at a speed and scale far exceeding LLMs. The collected trajectories are utilized to bootstrap the LLM via Supervised Fine-Tuning (SFT), followed by multi-turn Reinforcement Learning (RL) to activate its latent world knowledge. Empirically, SCOUT enables a Qwen2.5-3B-Instruct model to achieve an average score of 0.86, significantly outperforming proprietary models, including Gemini-2.5-Pro (0.60), while saving about 60% GPU hours consumption.

While Large Language Models excel in language-based applications, their performance on unseen symbolic or spatial tasks remains limited due to the prohibitive computational cost of trial-and-error exploration. For massive models, searching for optimal actions in a high-dimensional semantic space is highly inefficient and resource-heavy. To resolve this exploration bottleneck, we introduce SCOUT, a novel framework that decouples exploration from exploitation. We utilize lightweight, sub-scale networks to rapidly probe environmental dynamics and generate high-quality expert trajectories on commodity hardware. The language model then internalizes these task physics via supervised fine-tuning, followed by multi-turn reinforcement learning to activate its latent reasoning. Empirically, SCOUT enables a compact 3B parameter model to achieve an average score of 0.86, significantly outperforming proprietary systems like Gemini-2.5-Pro. Crucially, this collaboration framework reduces total GPU hours by approximately 60%, offering an efficient and environmentally sustainable pathway for training capable autonomous agents.