On the Difficulty of Feature Unlearning in Tabular Diffusion Models

Generative models have shown to be increasingly realistic data synthesizers. Across various domains, this has urged for better control of the data that influence their behavior. While unlearning methods show promise in image and language models, they remain understudied in the tabular context, generally, and in attribute-level (column/vertical) data removal, particularly. Here, we aim to kick-off this investigation. by proposing an evaluation framework to quantify the ability of post-training methods to remove features (columns). Our evaluation of multiple unlearners and a new variant, Vertical Negative Preference Optimization, allow us to present insights into the dynamics of feature removal. Therewith uncovering a consistent limitation: categorical features pose an additional challenge to unlearn successfully.