In medical applications, misclassifications can result in undetected diseases or incorrect diagnoses. Hence, being cautious when the model is uncertain is important. One way to be more cautious is to include a reject option in a classifier to allow it to abstain from making a prediction if its confidence in its prediction is low. This paper proposes a model-agnostic rejector based on the Local Outlier Factor anomaly score in the context of an important medical application: sleep stage scoring. This rejector improves the model's trustworthiness by detecting observations which substantially deviate from the training set. Moreover, the method can help identify populations which are missing in the training set.