Skip to yearly menu bar Skip to main content


Doubly robust off-policy evaluation with shrinkage

Yi Su · Maria Dimakopoulou · Akshay Krishnamurthy · Miroslav Dudik

Keywords: [ Recommender Systems ] [ Online Learning / Bandits ] [ Online Learning, Active Learning, and Bandits ]


We propose a new framework for designing estimators for off-policy evaluation in contextual bandits. Our approach is based on the asymptotically optimal doubly robust estimator, but we shrink the importance weights to minimize a bound on the mean squared error, which results in a better bias-variance tradeoff in finite samples. We use this optimization-based framework to obtain three estimators: (a) a weight-clipping estimator, (b) a new weight-shrinkage estimator, and (c) the first shrinkage-based estimator for combinatorial action sets. Extensive experiments in both standard and combinatorial bandit benchmark problems show that our estimators are highly adaptive and typically outperform state-of-the-art methods.

Chat is not available.