Can LLMs Reconstruct Why an Answer Is Correct? A Step-Level Evaluation of Answer-Conditioned Reasoning

Answer-conditioned rationale generation is widely used to synthesize reasoning traces for training, supervision, and evaluation. Given a question and the correct answer, a model is asked to explain why the answer is correct. However, it remains unclear whether such rationales are themselves factually accurate and logically valid. We systematically evaluate this question on two multi-hop question answering benchmarks with reference reasoning annotations, HotpotQA/R4C and MuSiQue. For each model, we compare rationales generated for questions it answers correctly versus incorrectly in a question-only setting, and evaluate both answer-only and answer-plus-evidence rationalization. Our results show that answer-conditioned rationales frequently fail to match reference reasoning, even when the model answers the question correctly. Providing supporting evidence substantially improves rationale correctness on HotpotQA/R4C, but has weaker and less consistent effects on MuSiQue. Error analysis further shows that incorrect rationales commonly omit key facts, follow defective reasoning paths, or introduce noisy extra reasoning. These findings suggest that LLM-generated rationales should not be treated as automatically reliable training signals.