Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective

In this work, we reveal that Large Language Models (LLMs) possess intrinsic behavioral plasticity—akin to chameleons adapting their coloration to environmental cues—that can be exposed through token-conditional generation and stabilized via reinforcement learning. Specifically, by conditioning generation on carefully selected token prefixes sampled from responses exhibiting desired behaviors, LLMs seamlessly adapt their behavioral modes at inference time (e.g., switching from step-by-step reasoning to direct answering) without retraining. Based on this insight, we propose Token-Conditioned Reinforcement Learning (ToCoRL), a principled framework that leverages RL to internalize this chameleon-like plasticity, transforming transient inference-time adaptations into stable and learnable behavioral patterns. ToCoRL guides exploration with token-conditional generation and keep enhancing exploitation, enabling emergence of appropriate behaviors. Extensive experiments show that ToCoRL enables precise behavioral control without capability degradation. Notably, we show that large reasoning models, while performing strongly on complex mathematics, can be effectively adapted to excel at factual question answering, which was a capability previously hindered by their step-by-step reasoning patterns.

Language models possess chameleon-like behavior adaptability. This capability enables the development of specialized models that adopt behavior patterns well suited for certain types of problems. However, these specialized patterns often make the models less effective on other tasks. We expose such adaptability from a novel perspective. We find that by prefilling response generation with examples with a desired behavior pattern, language models can naturally follow and temporarily shift toward that behavior. To further internalize such transient behavior shifts, we guide reinforcement learning with these prefills, encouraging the model to explore and adopt the desired pattern more effectively while still improving exploitation and overall performance. Surprisingly, our method stabilizes this adaptation process, leading to the emergence of a new behavior pattern that is better suited to the trained model. This provides a lightweight and stable way to adapt model behavior, offering a promising direction toward more flexible and general-purpose language models.