The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning

Real-world machine learning (ML) pipelines rarely produce a single model; instead, they produce a Rashomon set of many near-optimal ones. We show that this multiplicity reshapes key aspects of trustworthiness. At the individual-model level, sparse interpretable models tend to preserve privacy but are fragile to adversarial attacks. In contrast, the diversity within a large Rashomon set enables reactive robustness: even when an attack compromises one model, a practitioner can switch to a different near-optimal model that remains accurate, without retraining. However, the same diversity increases information leakage, as disclosing more near-optimal models provides an attacker with progressively richer views of the training data. This produces a robustness–privacy trade-off governed by diversity, which we analyze theoretically and empirically. Beyond this trade-off, Rashomon sets are stable under small distribution shifts, so a set computed once remains valid under such shifts without re-computation. Our results highlight the dual role of Rashomon sets as both a resource and a risk for trustworthy ML.

Machine learning systems are often treated as if there is one best model for a given task. In reality, there may be many different models that make predictions just as accurately, even though they behave slightly differently. This collection of equally good models is known as a Rashomon set. This paper studies why having many good models can be both helpful and risky. On the helpful side, if one model fails or is fooled by a carefully chosen input, we may be able to switch to another model that is just as accurate but makes decisions in a different way. This can make the overall system more resilient without requiring us to train a new model from scratch. On the risky side, sharing many good models can expose more about the data used to train them. Each model gives a slightly different view of the training data, and together these views can reveal more information than intended. Together, our results link the resilience of machine learning systems with their privacy risks: the same diversity of many models protects against malicious inputs also increases data exposure. Our work suggests that trustworthy machine learning should consider not only the model that is ultimately used, but also a larger collection of good models.