VDW-GNNs: Vector diffusion wavelets for geometric graph neural networks

We introduce vector diffusion wavelets (VDWs), a novel family of wavelets inspired by the vector diffusion maps algorithm that was introduced to analyze data lying in the tangent bundle of a Riemannian manifold. We show that these wavelets may be effectively incorporated into a family of geometric graph neural networks, which we refer to as VDW-GNNs. We demonstrate that such networks are effective on synthetic point cloud data, as well as on real-world data derived from wind field and neural activity measurements. Theoretically, we prove that these new wavelets have desirable frame theoretic properties, similar to traditional diffusion wavelets. Additionally, we prove that these wavelets have useful symmetries with respect to rotations and translations.

Some types of data have inherent notions of direction. One example is wind data (think “10 km/hr northeast”). We call this vector-valued data. This kind of data is often arranged spatially, for example: a set of wind measurements recorded by a global network of sensors. But what if some sensors were to fail randomly? How might we accurately and efficiently model such vector-valued data, in order to estimate missing values? To tackle this type of problem, we built a general-purpose model that processes vector-valued data on networks (aka graphs), using a method we call vector diffusion wavelets (VDWs). VDWs are designed to efficiently capture both short- and long-range data relationships, without ‘blurring’ them together. We use these VDWs as part of a neural network to spot patterns in the data to solve different tasks, such as reconstructing missing data or characterizing brain activity. We tested our approach on both synthetic and real-world data, showing it is a highly general, efficient, and accurate method. It extends existing wavelet tools to vector data and demonstrates the efficiency of building model components from geometric principles, in contrast to more ‘brute force’ approaches of many contemporary deep learning models.