Evaluating and Rewarding LALMs for Expressive Role-Play TTS via Mean Continuation Log-Probability

Recent advances in Large Audio Language Models (LALMs) have extended Text-to-Speech (TTS) to interactive role-play scenarios, which demand high expressiveness and strict adherence to role-play instructions. However, existing models struggle to maintain stylistic consistency with character profiles and scene descriptions across multi-turn dialogues. A critical bottleneck is the lack of objective metrics for quantifying speaking style. To bridge this gap, we propose Mean Continuation Log-Probability (MCLP) as both an evaluation metric and a reward signal, validated on LALM-based Role-Play TTS (RP-TTS) tasks. MCLP leverages the in-context learning capability of pretrained LALMs to measure the likelihood of ground-truth speech tokens conditioned on a contextual history consisting of the transcript, generated speech, and repeated transcript, serving as a proxy for stylistic continuity. Furthermore, we employ MCLP as a reinforcement learning reward to enhance the style alignment between generated speech and role-play instructions. To support this task, we construct a large-scale RP-TTS dataset with rich scene and character annotations. Experiments demonstrate that MCLP is well aligned with human judgments of stylistic consistency and serves as an effective reward for improving RP-TTS, leading to consistent gains in both objective metrics and subjective evaluations. Our code is publicly available at https://github.com/y-ren16/MCLP.

Applications such as film dubbing, voice assistants, and virtual characters increasingly need speech that is expressive and consistent with a desired speaking style. This is challenging in role-play speech generation. For example, a character in a tense scene should speak differently from one in a joyful scene, and a cautious character should not sound the same as a bold and outgoing character. Beyond saying the correct words, the system should speak in a way that fits the intended character, scene, and dialogue context. This work studies how to evaluate and improve style consistency in role-play text-to-speech systems. We introduce a new scoring method that uses a large audio-language model to judge whether a generated speech sample is stylistically consistent with a reference recording. This score can be used not only to evaluate speech quality, but also as feedback for training the speech model. We also built a large role-play speech dataset from Chinese film and TV drama videos, with annotations describing scenes and characters. Experiments show that our style-consistency score agrees well with human judgments and helps train models to generate speech that both says the right words and better matches the target character, scene, and dialogue context.