Loading...

Towards Spurious Correlation Robustness of Out-of-Distribution Detection Methods

Zohrabi, Reihaneh | 2024

0 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 57245 (19)
  4. University: Sharif University of Technology
  5. Department: Computer Engineering
  6. Advisor(s): Rohban, Mohammad Hossein
  7. Abstract:
  8. Many machine learning models make confident decisions when encountering out-of-distribution (OOD) data which differ from their training data distribution. However, these models should not make predictions on unfamiliar samples they have not seen before, and rejecting such unknown samples is crucial for deploying trustworthy models in real-world applications. Consequently, OOD detection has garnered significant attention over the past decade. Despite the development of highly accurate methods to address this issue, there has been little focus on their robustness against various factors. One common factor threatening the robustness of these methods is the presence of spurious correlations in data. In other words, a model's reliance on statistical but spurious features in the training data, such as backgrounds or textures in images, and the loss or weakening of these correlations in the test data can reduce the robustness and performance of the detection method. This research aims to comprehensively examine the existing datasets in this field for spurious correlations and evaluate the performance of a broad range of OOD detection methods against this vulnerability. Ultimately, the goal is to propose a method robust to spurious correlations. In the first part, we evaluate and analyze the performance of a wide spectrum of post-hoc OOD detection methods against this type of vulnerability. In order to have a valid evaluation, we have made a spurious OOD dataset, which is more challenging and suitable for evaluation than the previous datasets. The results obtained from the evaluations show the weakness of the previous works in detecting spurious OOD samples, and with the increase of the correlation, this performance drop increases. Finally, we present an innovative approach based on generative probabilistic modeling designed to more accurately identify OOD data in the presence of such correlations. This method does not require hyperparameter tuning and is remarkably simple yet highly effective at detecting any type of OOD data, whether spurious or not, achieving superior performance on benchmarks in this field. More precisely, the proposed approach has achieved AUROC of 99.7, which is state of the art compared to previous methods
  9. Keywords:
  10. Robustness ; Spurious Correlation ; Generative Models ; Post-hoc Methods ; Trustworthy Machine Learning ; Out of Distribuiton Detection

 Digital Object List

 Bookmark

...see more