ALSO: Adversarial Online Strategy Optimization for Social Agents

Social simulation provides a compelling testbed for studying social intelligence, where agents interact through multi-turn dialogues under evolving contexts and strategically adapting opponents. Such environments are inherently non-stationary, requiring agents to dynamically adjust their strategies over time. However, most Large Language Model (LLM) based social agents rely on static personas, while existing approaches for enhancing social intelligence, such as offline reinforcement learning or external planners, are ill-suited to these settings, typically assuming stationarity and incurring substantial training overhead. To bridge this gap, we propose ALSO (Adversarial onLine Strategy Optimization), the first framework for online strategy optimization in multi-agent social simulation. ALSO advances social adaptation through two key contributions. (1) ALSO formulates multi-turn interaction as an adversarial bandit problem, where combinations of static personas and dynamic strategy instructions are treated as arms, providing a principled solution to non-stationarity without relying on environmental stability assumptions. (2) To predict rewards and generalize sparse feedback in multi-turn dialogues, ALSO introduces a lightweight neural surrogate to predict rewards from interaction histories, enabling sample-efficient exploration and continuous online adaptation. Experiments on the Sotopia benchmark demonstrate that ALSO consistently outperforms static baselines and existing optimization methods in dynamic environments, validating the effectiveness of adversarial online strategy optimization for building robust social agents.

When AI chatbots are placed in social situations — negotiating a price, persuading a friend, planning together — the people they interact with constantly shift tactics, mood, and goals. But today's social AI agents stick to a fixed personality and strategy, so they fall flat once the conversation moves in unexpected directions. Existing fixes require expensive retraining and assume the surrounding environment stays predictable, which it rarely does. We developed ALSO, a framework that lets a social AI agent adjust its strategy mid-conversation, without any retraining. Think of it like a poker player picking a different playing style each hand: the agent tries a combination of personality and tactic, observes how the exchange unfolds, and updates its choices on the fly. To squeeze useful lessons from sparse feedback, we paired it with a small companion network that estimates how promising a strategy is from the conversation so far. On Sotopia, a popular social-intelligence benchmark, ALSO consistently outperforms fixed-personality agents and prior adaptation methods. It's a step toward AI assistants that hold up when the person across the table changes their mind, raises the stakes, or tries something new.