Timezone: »

 
Evaluating Self-Supervised Learned Molecular Graphs
Hanchen Wang · Shengchao Liu · Jean Kaddour · Qi Liu · Jian Tang · Matt Kusner · Joan Lasenby

@
in AI for Science »
Event URL: https://openreview.net/forum?id=LeJC_Mf5rx- »

Because of data scarcity in real-world scenarios, obtaining pre-trained representations via self-supervised learning (SSL) has attracted increasing interest. Although various methods have been proposed, it is still under-explored what knowledge the networks learn from the pre-training tasks and how it relates to downstream properties. In this work, with an emphasis on chemical molecular graphs, we fill in this gap by devising a range of node-level, pair-level, and graph-level probe tasks to analyse the representations from pre-trained graph neural networks (GNNs). We empirically show that: 1. Pre-trained models have better downstream performance compared to randomly-initialised models due to their improved the capability of capturing global topology and recognising substructures. 2. However, randomly initialised models outperform pre-trained models in terms of retaining local topology. Such information gradually disappears from the early layers to the last layers for pre-trained models.

Author Information

Hanchen Wang (Cambridge; Caltech)

Joint PostDoc, ML for Genomics

Shengchao Liu (Mila, Université de Montréal)
Jean Kaddour (UCL)
Qi Liu (Department of Computer Science, University of Oxford)
Jian Tang (Mila)
Matt Kusner (University College London)
Joan Lasenby (University of Cambridge)

More from the Same Authors