Neuro-Fuzzy Concept Learning for Interpretable Large Multimodal Models

Large Multimodal Models (LMMs) integrate unimodal encoders with Large Language Models (LLMs) to execute complex multimodal tasks. Despite progress in the field, understanding the internal representations of these models through interpretable logic remains an open problem. To address this, we present a framework utilizing a Human-Inspired (Neuro-fuzzy) approach for learning token representations. In this method, we leverage fuzzy rules to compute activation firing strengths, which are subsequently defuzzified to extract distinct concepts. This mechanism allows for the interpretation of learned representations directly through explicit logic. Consequently, we derive "multimodal concepts" that are both semantically coherent and interpretable. We validate our approach through rigorous qualitative and quantitative experiments, demonstrating the utility of these concepts in interpreting test samples. Additionally, we evaluate the disentanglement of the learned concepts and the efficacy of their grounding in both visual and textual domains.

Artificial intelligence systems that can understand both images and text, often called large multimodal models, are becoming increasingly powerful and are now used in applications such as image captioning, healthcare, and scientific research. Despite their impressive performance, these models often operate as "black boxes," making it difficult to understand why they arrive at a particular decision or prediction. In this work, we introduce Neuro-FeX, a method inspired by the way humans reason using concepts and logical rules. It identifies meaningful concepts learned by a multimodal model and explains how these concepts contribute to its decisions. Instead of only highlighting which parts of an image or text are important, Neuro-FeX uncovers higher-level ideas and represents them through interpretable "IF–THEN" reasoning rules. For example, when a model recognizes an object such as a bus or a train, our method reveals the key concepts associated with it and shows how these concepts contribute to the model’s decision. Our experiments demonstrate that Neuro-FeX produces meaningful and interpretable concepts while maintaining strong alignment between visual and textual information. By improving transparency and enabling users to inspect how multimodal AI systems reason, this work contributes toward the development of more trustworthy, accountable, and human-centered artificial intelligence.