Semi-Supervised Learning for Molecular Graphs via Ensemble Consensus

Machine learning is transforming molecular sciences by accelerating property prediction, simulation, and the discovery of new molecules and materials. Acquiring labeled data in these domains is often costly and time-consuming, whereas large collections of unlabeled molecular data are readily available. Standard semi-supervised learning methods often rely on label-preserving augmentations, which are challenging to design in the molecular domain, where minor changes can drastically alter properties. In this work, we show that semi-supervised methods that rely on an ensemble consensus can boost predictive accuracy across a diverse range of molecular datasets, task types, and graph neural network architectures. We find that training with an ensemble consensus objective increases robustness in models and exhibits an effect similar to knowledge distillation; an individual member of an ensemble trained this way outperforms a full ensemble trained in a traditional supervised fashion in almost all cases. In addition, this type of semi-supervised training reduces calibration error.

Important scientific problems such as molecular medicine and biology naturally represent their data as graphs. Deep learning methods need large amounts of data where the correct label is known, but this is costly. For domains such as image-like data, existing methods can leverage a mix of data with and without labels for large accuracy improvements, but these approaches has so far not been effective for graph data. We propose a powerful method that enables the use of mixed data for graphs. Our method works by letting multiple deep learning models learn from each others predictions simultaneously.