Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment

Chain-of-thought (CoT) traces often serve as a proxy for how Large Language Models (LLMs) arrive at their answers. However, growing evidence shows that models' CoT often fails to reflect their internal computations and can be manipulated to produce different CoTs without changing their outputs. In this work, we measure and improve the alignment between what LLMs say in their CoT and what they compute internally. To quantify such parametric faithfulness, we propose CoT-Interpretability Alignment (CIA), a metric that measures the agreement between a model's CoT traces and its internal reasoning strategies as detected by interpretability tools. We evaluate CIA on three tasks (two-hop question answering, hint intervention, and integer multiplication) across three LLMs, finding that current models exhibit low CIA scores across all tasks. We then experiment with improving CIA via post-training while setting both task accuracy and parametric faithfulness signals as a reward. Experiments show that we can substantially improve CoT parametric faithfulness while maintaining or improving task accuracy. We provide rich analysis, showing how such post-training changes model behaviors: in some tasks the model learns to change how it reasons, while in others it learns to change how it reports. Our work provides both a framework for auditing CoT parametric faithfulness and a pathway toward making models' explicit reasoning more trustworthy.