Timezone: »

 
Poster
UNIREX: A Unified Learning Framework for Language Model Rationale Extraction
Aaron Chan · Maziar Sanjabi · Lambert Mathias · Liang Tan · Shaoliang Nie · Xiaochang Peng · Xiang Ren · Hamed Firooz

Tue Jul 19 03:30 PM -- 05:30 PM (PDT) @ Hall E #118
in Poster Session 1 »

An extractive rationale explains a language model's (LM's) prediction on a given task instance by highlighting the text inputs that most influenced the prediction. Ideally, rationale extraction should be faithful (reflective of LM's actual behavior) and plausible (convincing to humans), without compromising the LM's (i.e., task model's) task performance. Although attribution algorithms and select-predict pipelines are commonly used in rationale extraction, they both rely on certain heuristics that hinder them from satisfying all three desiderata. In light of this, we propose UNIREX, a flexible learning framework which generalizes rationale extractor optimization as follows: (1) specify architecture for a learned rationale extractor; (2) select explainability objectives (\ie faithfulness and plausibility criteria); and (3) jointly train the task model and rationale extractor on the task using selected objectives. UNIREX enables replacing prior works' heuristic design choices with a generic learned rationale extractor in (1) and optimizing it for all three desiderata in (2)-(3). To facilitate comparison between methods w.r.t. multiple desiderata, we introduce the Normalized Relative Gain (NRG) metric. On five English text classification datasets, our best UNIREX configuration outperforms baselines by an average of 32.9% NRG.Plus, UNIREX rationale extractors' faithfulness can even generalize to unseen datasets and tasks.

Keywords: APP: Language, Speech and Dialog · SA: Trustworthy Machine Learning · SA: Accountability, Transparency and Interpretability

Author Information

Aaron Chan (University of Southern California)
Maziar Sanjabi (Meta AI)
Lambert Mathias (Facebook)
Liang Tan (Facebook)
Shaoliang Nie (Facebook)
Xiaochang Peng
Xiang Ren (University of Southern California)

Xiang Ren joined the Department of Computer Science at USC as Assistant Professor in 2018. Previously, he was a visiting researcher at Stanford University. Xiang received his PhD in Computer Science at University of Illinois at Urbana-Champaign (2017), where he was a Google PhD Fellow and a Richard T. Cheng Fellow working with Prof. Jiawei Han. Xiang's research develops data-driven and machine learning methods for turning unstructured text data into machine-actionable structures. Xiang's research has been recognized with several prestigious awards including a Yahoo!-DAIS Research Excellence Award, a Yelp Dataset Challenge award, a C. W. Gear Outstanding Graduate Student Award and a David J. Kuck Outstanding M.S. Thesis Award. Technologies he developed has been transferred to US Army Research Lab, National Institute of Health, Microsoft, Yelp and TripAdvisor.

Hamed Firooz (Facebook)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors