When Plans Collide: Joint Planning in LLM Negotiation Dyads

Effective joint action requires LLM agents to do more than reason well individually; they must form and execute shared plans under partial information. Drawing on Bratman's theory of shared intentionality, we decompose this joint planning process into three stages: mutual responsiveness (agents acquire partner information needed for planning), joint plan formation (agents negotiate and commit to a shared plan rather than parallel individual ones), and subplan execution (each agent's actions cohere with the joint plan through to decision time). We introduce an iterated, multi-turn negotiation game in which two agents allocate shared resources toward private projects with verifiable Pareto-optimal joint plans. Individual agents find optimal solutions in isolation, yet dyads consistently fail to read them. We trace this gap to characteristic stage-specific breakdowns: low information disclosure starves plan formation at Stage 1; anchoring on uninformed proposals and absence of genuine joint planning at Stage 2; and verbally agreed plans abandoned at execution time at Stage 3. This decomposition reframes joint planning failure not as a single capability bottleneck but as a pipeline, enabling targeted interventions at each stage.