Propensity scores are commonly used to balance observed confounders while estimating treatment effects. When the confounders are high-dimensional or unstructured, the learned propensity scores can be miscalibrated and ineffective in the correction of confounding. We argue that the probabilistic output of a learned propensity score model should be calibrated, i.e. predictive treatment probability of 90% should correspond to 90% individuals being assigned the treatment group. We investigate the theoretical properties of a calibrated propensity score model and its role in unbiased treatment effect estimation. We demonstrate improved causal effect estimation with calibrated propensity scores in several tasks including high-dimensional genome-wide association studies.