Explaining What Machine Learning Learns through Explainable AI

Recent discussions of explainable artificial intelligence (XAI) often treat explanation as a homogeneous epistemic category. This paper argues that different XAI methods provide different forms of explanatory relevance and therefore reduce opacity in importantly different ways. Drawing on Boge’s (2022) distinction between h-opacity and w-opacity, I argue that many existing XAI methods primarily reduce computational visibility while leaving learned representational structure substantially unresolved. Consequently, strong forms of explanation-based trust require explanatorily relevant access to learned output-producing structures rather than predictive reliability alone.