Detecting out-of-distribution (OOD) input samples at the inference time is a key element in the trustworthy deployment of intelligent models. While there has been a tremendous improvement in various flavors of OOD detection in recent years, the detection performance under adversarial settings lags far behind the performance in the standard setting. In order to bridge this gap, we introduce RODEO in this paper, a data-centric approach that generates effective outliers for robust OOD detection. More specifically, we first show that targeting the classification of adversarially perturbed in- and out-of-distribution samples through outlier exposure (OE) could be an effective strategy for the mentioned purpose as long as the training outliers meet certain quality standards. We hypothesize that the outliers in the OE should possess several characteristics simultaneously to be effective in the adversarial training: diversity, and both conceptual differentiability and analogy to the inliers. These aspects seem to play a more critical role in the adversarial setup compared to the standard training. Next, we propose to take advantage of existing text-to-image generative models, conditioned on the inlier or normal samples, and text prompts that minimally edit the normal samples, and turn them into near-distribution outliers. This process helps to satisfy the three mentioned criteria for the generated outliers, and significantly boosts the performance of OE specially in the adversarial setting. We demonstrate the general effectiveness of this approach in various related problems including novelty/anomaly detection, Open-Set Recognition (OSR), and OOD detection. We also make a comprehensive comparison of our method against other adaptive OE techniques under the adversarial setting to showcase its effectiveness.