While recent automatic data augmentation works lead to state-of-the-art results, their design spaces and the derived data augmentation strategies still incorporate strong human priors. In this work, instead of selecting a set of hand-picked default augmentations alongside the searched data augmentations, we propose a fully automated approach for data augmentation search called Deep AutoAugment (DAA). We propose a search strategy that matches the directions of the validation gradients and the training gradients averaged over all possible augmentations. Our experiments show that DAA achieves strong performance on CIFAR-10/100 and SVHN with much less search cost compared to state-of-the-art data augmentation search methods.