AVATAR-AGENT: A Multi-Agent LLM Planning System for 3D Avatar Generation

Large Language Models are increasingly used as planners for tasks with structured action spaces, but their reliability in real-world compositional domains with discrete asset constraints remains underexplored. We present AVATAR-AGENT, a hierarchical multi-agent LLM planning system for 3D avatar generation in which an LLM Strategist generates structured plans from natural language goals, an Executor decomposes each plan step into LLM-guided search and tool calls, and a vision-language Validator verifies plan outcomes against hard constraints, triggering targeted plan repair when failures are detected. Plans are expressed as ordered sequences of typed actions over a discrete asset corpus, enabling automated verification of constraint satisfaction. A short-term memory tracks failed search contexts and invalidated assets to prevent repeated planning errors during repair iterations. In evaluation on 100 prompts, AVATAR-AGENT achieves 94.0% constraint satisfaction with quality statistically comparable to human-created avatars, substantially outperforming a non-planning semantic retrieval baseline (72.6%). Ablations show that plan repair and short-term memory each contribute measurable improvements. We discuss what LLMs can contribute to compositional planning in this setting, the pitfalls we observed (VLM-based verification inconsistency, search reformulation drift, catalog gap blindness), and what guarantees the system does and does not provide.