Deep unsupervised anomaly detection

Deep unsupervised anomaly detection

This paper proposes a novel method to detect anomalies in large datasets under a fully unsupervised setting. The key idea behind our algorithm is to learn the representation underlying normal data. To this end, we leverage the latest clustering technique suitable for handling high dimensional data....

Full description

Saved in:
Bibliographic Details
Main Authors: LI, Tangqing, WANG, Zheng, LIU, Siying, LIN, Wen-yan
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2021
Subjects:
Computer vision
clustering algorithms
benchmark testing
reliability
anomaly detection
Computer Sciences
Databases and Information Systems
Theory and Algorithms
Online Access:https://ink.library.smu.edu.sg/sis_research/6111
https://ink.library.smu.edu.sg/context/sis_research/article/7114/viewcontent/0184.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
Description
Summary:This paper proposes a novel method to detect anomalies in large datasets under a fully unsupervised setting. The key idea behind our algorithm is to learn the representation underlying normal data. To this end, we leverage the latest clustering technique suitable for handling high dimensional data. This hypothesis provides a reliable starting point for normal data selection. We train an autoencoder from the normal data subset, and iterate between hypothesizing normal candidate subset based on clustering and representation learning. The reconstruction error from the learned autoencoder serves as a scoring function to assess the normality of the data. Experimental results on several public benchmark datasets show that the proposed method outperforms state-of-the-art unsupervised techniques and is comparable to semi-supervised techniques in most cases.

Similar Items