Measuring Diversity: Axioms and Challenges

This paper addresses the problem of quantifying diversity for a set of objects. First, we conduct a systematic review of existing diversity measures and explore their undesirable behavior in certain cases. Based on this review, we formulate three desirable properties (axioms) of a reliable diversity measure: monotonicity, uniqueness, and continuity. We show that none of the existing measures has all three properties and thus these measures are not suitable for quantifying diversity. Then, we construct two examples of measures that have all the desirable properties, thus proving that the list of axioms is not self-contradictory. Unfortunately, the constructed examples are too computationally expensive (NP-hard) for practical use. Thus, we pose an open problem of constructing a diversity measure that has all the listed properties and can be computed in practice or proving that all such measures are NP-hard to compute.

How can one measure diversity of a set of objects, like images, molecules, or recommendations? The need for reliable diversity measures arises because diverse outputs are critical in applications like AI-generated content and recommender systems, where variety enhances creativity and user satisfaction. We review existing diversity measures and find that they may behave unpredictably, failing to capture true diversity. To formally analyze this, we define three essential qualities for a good diversity measure: monotonicity, ensuring diversity increases as objects become more distinct; uniqueness, meaning duplicates reduce diversity compared to unique items; and continuity, requiring small changes in diversity with small changes in object differences. Our analysis shows that no current measure meets all three requirements. Finally, we design two new measures that satisfy these qualities, but they are too complex for practical use. This creates an open challenge: developing a diversity measure that has desirable qualities and is computationally practical for real-world applications.