Timezone: »

Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language
Philipp Seidl · Andreu Vall · Sepp Hochreiter · Günter Klambauer

Tue Jul 25 02:00 PM -- 04:30 PM (PDT) @ Exhibit Hall 1 #106

Activity and property prediction models are the central workhorses in drug discovery and materials sciences, but currently, they have to be trained or fine-tuned for new tasks. Without training or fine-tuning, scientific language models could be used for such low-data tasks through their announced zero- and few-shot capabilities. However, their predictive quality at activity prediction is lacking. In this work, we envision a novel type of activity prediction model that is able to adapt to new prediction tasks at inference time, via understanding textual information describing the task. To this end, we propose a new architecture with separate modules for chemical and natural language inputs, and a contrastive pretraining objective on data from large biochemical databases. In extensive experiments, we show that our method CLAMP yields improved predictive performance on few-shot learning benchmarks and zero-shot problems in drug discovery. We attribute the advances of our method to the modularized architecture and to our pre-training objective.

Author Information

Philipp Seidl (Johannes Kepler University Linz)
Andreu Vall (Johannes Kepler University Linz)
Sepp Hochreiter (ELLIS Unit Linz, LIT AI Lab, Institute for Machine Learning, Johannes Kepler University, Institute for Advanced Research in Artificial Intelligence (IARAI))
Sepp Hochreiter

Sepp Hochreiter is heading the Institute for Machine Learning, the ELLIS Unit Linz, the LIT AI Lab at the JKU Linz and is director of private research institute IARAI. He is a pioneer of Deep Learning as he discovered the famous problem of vanishing or exploding gradients and invented the long short-term memory (LSTM).

Günter Klambauer (Johannes Kepler University Linz Austria)

More from the Same Authors