Loading...

Unsupervised Labeling for Supervised Anomaly Detection

Abazari, Maryam | 2020

383 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 53612 (01)
  4. University: Sharif University of Technology
  5. Department: Industrial Engineering
  6. Advisor(s): Akhavan Niaki, Taghi
  7. Abstract:
  8. Identifying anomalous events is one of the vital topics in research as it often leads to the detection of actionable and critical information such as intrusions, faults, and system failures. With its importance, there has been a substantial body of work for network anomaly detection using supervised and unsupervised machine learning techniques with their own strengths and weaknesses. In this work, we take advantage of both worlds of unsupervised and supervised learning methods. The basic process model we present in this paper includes (i) clustering the training data set to create referential labels, (ii) building a supervised learning model with the automatically produced labels, and (iii) testing individual data points in question using the established learning model. To attain this process, a model composed of a fully-connected autoencoder and a custom SOM layer is used, where the SOM code vectors are learned jointly with the autoencoder weights. After training the aforementioned model, we set up a new feature space, based on SOM code vectors and the autoencoder weights, by which the referential labels can be obtained from SOM output. Then, we use the new feature space and the referential labels as input for KNN, PNN, SVM, Random Forest, and Logistic Regression. Through our extensive experiments with a public data set (MNIST), we will show that the presented method performs very well, yielding fairly comparable performance to the traditional method running with the original labels provided in the data set, with respect to the accuracy for anomaly detection
  9. Keywords:
  10. Autoencoder ; Self-Organizing Map (SOM) ; Anomaly Detection ; Supervised Learning ; Machine Learning ; Unsupervised Learning

 Digital Object List

 Bookmark

No TOC