LLM Persuasiveness Evaluation: A Structured Review of Automated Methods

As Large Language Models (LLMs) become increasingly persuasive, their ability to shape beliefs and behaviour at scale has raised concerns, prompting regulatory attention and calls for robust evaluation frameworks. Human-participant studies provide ecological validity but are costly, slow, and constrained by ethical challenges, making them impractical for systematic assessment of rapidly evolving systems. As an alternative, fully automated evaluation methods requiring no human involvement enable reproducible, fast, and ethically unconstrained large-scale testing. To organise this rapidly growing literature and inform future research and development, we provide the first systematic taxonomy of 30 automated methods across 27 papers, examining their designs, human validation results, limitations, and associated risks.