Equivalence Testing of Parametric Models via the Effective Dimension

A natural way to test whether two parameterized models behave equivalently is to query both on $n$ i.i.d. inputs and check whether their predictions agree. What such a test certifies on the underlying input distribution is a statistical question, and classical answers via combinatorial complexity measures such as the VC dimension are loose for overparametrized families. We derive a theoretical guarantee for equivalence testing of parameterized probabilistic models using the notion of the Fisher-information--based effective dimension $d_{\mathrm{eff}}$ Abbas et al. (2021). The bound certifies expected predictive total-variation disagreement of $O(\varepsilon\sqrt{\ln n / n})$ on unseen data, where $\varepsilon$ is a tunable parameter controlling the tradeoff between the error bound and the confidence level. We instantiate the result on binary logistic regression and derive sample-complexity bounds for testing model equivalence, describing the dichotomy between a parameter-close regime, where $d_{\mathrm{eff}}$ governs the rate, and a parameter-distant regime, where it does not.