LLMs at the Bargaining Table

Bilateral negotiation is a particularly well suited scenario to test the strategic capability of large language models, since they are interactive, carried out in natural language, and involve imperfect information and belief formation. At the same time, the outcome is very structured: whether a deal is closed, and if so, the closing price. In this paper, we study the strategic capability of LLMs in the context of bilateral negotiation. While much of the recent literature have compared LLM behavior to human strategic play in behavioral experiments, we focus instead on measuring the economic efficiency and effectiveness of LLM behavior, and mapping LLM behavior to predictions by economic theory for fully rational agents. Our goal is not to study specific models, but to (1) demonstrate that LLMs naturally (i.e., with very light prompting) show high strategic capability that qualitatively matches theoretical predictions, and (2) more generally, propose a methodology for evaluating new models in terms of strategic capability.