Adaptive Node Feature Selection for Graph Neural Networks

We propose an adaptive node feature selection approach for graph neural networks (GNNs) that identifies and removes unnecessary features during training. The ability to measure how features contribute to model output is key for interpreting decisions and reducing dimensionality by eliminating unhelpful variables. However, graph-structured data introduces complex dependencies that may be unsuited to classical feature importance metrics. Inspired by this, we present a data-, model-, and task-agnostic method that determines relevant features during training based on changes in validation performance upon permuting feature values. We theoretically motivate our approach by characterizing how the relationships between node data and graph structure influences GNN performance. Empirically, we show that (i) our highly general approach rivals the performance of tailored feature selection approaches that exploit prior assumptions; (ii) we return meaningful feature importance scores well before the GNN is fully trained; and (iii) our scores demonstrably extract relevant properties that inform feature importance for various graph learning settings.

Graph neural networks are powerful tools for predictions on complex, interconnected data, such as classifying academic papers by their citations counts or grouping products that are often purchased together. Each item in such data comes with a list of features describing their properties. For example, for a scientific paper, we may evaluate it based on the words it contains. These features are intended to be relevant to the prediction task, and each is typically treated as equally relevant. However, realistically, many are useless or even misleading. If we know which features actually matter, we can build simpler, faster, and more trustworthy models. Furthermore, we can better understand the systems we study, as we can directly interpret which measurements are actually important to our tasks. However, when items are inherently related, their connections can change which features are important, and existing ways to measure feature importance often ignore these connections or rely on assumptions that only work in specific situations. We develop a method that identifies the features that matter to our task while the model is still learning, rather than requiring a separate step to do so beforehand. Our method works by perturbing one feature at a time and observing how much the accuracy of the model drops, where a larger drop in performance implies that the feature was important. Our approach needs no assumptions about the type of graph data considered, and it is suitable for a variety of very different scenarios. This lets researchers automatically discard irrelevant features during training. Thus, our approach will help us develop more economic models and provide understandable insights into which factors truly drive predictions in real-world networked data.