Loading...

Unsupervised anomaly detection with LSTM autoencoders using statistical data-filtering

Maleki, S ; Sharif University of Technology | 2021

332 Viewed
  1. Type of Document: Article
  2. DOI: 10.1016/j.asoc.2021.107443
  3. Publisher: Elsevier Ltd , 2021
  4. Abstract:
  5. To address one of the most challenging industry problems, we develop an enhanced training algorithm for anomaly detection in unlabelled sequential data such as time-series. We propose the outputs of a well-designed system are drawn from an unknown probability distribution, U, in normal conditions. We introduce a probability criterion based on the classical central limit theorem that allows evaluation of the likelihood that a data-point is drawn from U. This enables the labelling of the data on the fly. Non-anomalous data is passed to train a deep Long Short-Term Memory (LSTM) autoencoder that distinguishes anomalies when the reconstruction error exceeds a threshold. To illustrate our algorithm's efficacy, we consider two real industrial case studies where gradually-developing and abrupt anomalies occur. Moreover, we compare our algorithm's performance with four of the recent and widely used algorithms in the domain. We show that our algorithm achieves considerably better results in that it timely detects anomalies while others either miss or lag in doing so. © 2021 Elsevier B.V
  6. Keywords:
  7. Learning systems ; Long short-term memory ; Probability distributions ; Algorithm's performance ; Central Limit Theorem ; Industrial case study ; Industry problems ; Probability criterion ; Reconstruction error ; Statistical datas ; Unsupervised anomaly detection ; Anomaly detection
  8. Source: Applied Soft Computing ; Volume 108 , 2021 ; 15684946 (ISSN)
  9. URL: https://www.sciencedirect.com/science/article/abs/pii/S1568494621003665