Timezone: »

 
Poster
Few-Shot Conformal Prediction with Auxiliary Tasks
Adam Fisch · Tal Schuster · Tommi Jaakkola · Regina Barzilay

Thu Jul 22 09:00 AM -- 11:00 AM (PDT) @ Virtual
in Poster Session 5 »

We develop a novel approach to conformal prediction when the target task has limited data available for training. Conformal prediction identifies a small set of promising output candidates in place of a single prediction, with guarantees that the set contains the correct answer with high probability. When training data is limited, however, the predicted set can easily become unusably large. In this work, we obtain substantially tighter prediction sets while maintaining desirable marginal guarantees by casting conformal prediction as a meta-learning paradigm over exchangeable collections of auxiliary tasks. Our conformalization algorithm is simple, fast, and agnostic to the choice of underlying model, learning algorithm, or dataset. We demonstrate the effectiveness of this approach across a number of few-shot classification and regression tasks in natural language processing, computer vision, and computational chemistry for drug discovery.

Keywords: Algorithms · Time Series Analysis · Probabilistic Methods · Multitask, Transfer, and Meta Learning
[ Slides] 

Author Information

Adam Fisch (MIT)
Tal Schuster (MIT CSAIL)
Tommi Jaakkola (MIT)
Regina Barzilay (MIT CSAIL)
Regina Barzilay

Regina Barzilay is an Israeli-American computer scientist. She is a professor at the Massachusetts Institute of Technology and a faculty lead for artificial intelligence at the MIT Jameel Clinic. Her research interests are in natural language processing and applications of deep learning to chemistry and oncology.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors