How does Bayesian Sampling help Membership Inference Attacks?

Membership Inference Attacks (MIAs) aim to estimate whether a specific data point was used in the training of a given model. Existing state-of-the-art attacks typically rely on training multiple reference models to approximate the conditional score distribution for individual data points, which leads to significant computational overhead and limits their practical applicability. In this work, we propose a novel approach -- Bayesian Membership Inference Attack (BMIA), which performs conditional attack through Bayesian sampling. Specifically, we apply Laplace approximation to a single reference model to obtain a posterior over model parameters, enabling direct estimation of the conditional score distribution. Theoretically, we demonstrate that Bayesian sampling reduces intra-model variance, thereby improving attack power. This insight naturally motivates the multi-reference variant that further enhances performance when additional reference models are available. Extensive experiments across image, text, and tabular datasets indicate that our method achieves state-of-the-art performance in both effectiveness and efficiency.

Machine learning models can sometimes reveal whether a data point was used during training, creating privacy risks. Membership inference attacks test this risk by asking whether a given example belonged to a model’s training set. Existing strong attacks often train many extra reference models, making them computationally expensive. We propose the Bayesian Membership Inference Attack (BMIA), which needs only one reference model. BMIA applies Bayesian sampling to estimate how that reference model’s predictions vary under uncertainty, allowing it to better model the behavior expected for non-training examples. This makes the membership decision more reliable for each data point. Across image, text, and tabular datasets, BMIA detects privacy leakage more accurately than prior methods while using much less computation. This makes privacy auditing more practical for modern machine learning models.