Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech

Integrating large language models (LLMs) into text-to-speech (TTS) systems has improved speech expressiveness, yet interpretable emotional control remains challenging. Existing approaches primarily rely on external conditioning or global activation steering, offering limited insight into the internal representations underlying emotional control. In this work, we analyze emotion-related variation in the semantic hidden states of LLM-based TTS models using sparse autoencoders (SAEs) to identify sparse latent features. Our analysis shows that emotional variation is distributed across multiple sparse latent features, while intervening on a small subset enables interpretable emotion control. Building on this observation, we introduce a feature-level intervention framework for bidirectional emotion induction and suppression without modifying backbone parameters. We further show that distinct latent features are associated with specific acoustic attributes (e.g., pitch), suggesting that emotional expression arises from coordinated latent contributions rather than a single global shift. Empirically, steering these sparse latent features achieves comparable or superior emotion induction and suppression performance relative to global steering and existing TTS baselines.

Modern text-to-speech systems can read text aloud with natural emotion, but it is still hard to understand how they decide to sound happy, sad, or angry. This lack of understanding makes emotional control difficult: changing the voice often requires extra labels, extra models, or broad changes inside the system that are hard to interpret. We study the hidden patterns inside a large text-understanding AI model used for speech generation, and use a tool that breaks these patterns into a small number of more readable components. We find that emotion is not stored in one place. Instead, different components work together, and some are closely tied to audible properties such as pitch. By changing only selected components, we can make a generated voice express more of an emotion, or reduce that emotion, without changing the main speech model. Our results show that this fine-grained control can match or outperform broader steering methods and existing emotional speech systems. This provides a more transparent way to control expressive speech and helps explain how modern speech models represent emotion.