Deep anomaly detection with deviation networks

Although deep learning has been applied to successfully address many data mining problems, relatively limited work has been done on deep learning for anomaly detection. Existing deep anomaly detection methods, which focus on learning new feature representations to enable downstream anomaly detection...

Full description

Saved in:
Bibliographic Details
Main Authors: PANG, Guansong, SHEN, Chunhua, HENGEL, Anton van den
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2019
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7138
https://ink.library.smu.edu.sg/context/sis_research/article/8141/viewcontent/3292500.3330871.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8141
record_format dspace
spelling sg-smu-ink.sis_research-81412022-04-22T04:22:56Z Deep anomaly detection with deviation networks PANG, Guansong SHEN, Chunhua HENGEL, Anton van den Although deep learning has been applied to successfully address many data mining problems, relatively limited work has been done on deep learning for anomaly detection. Existing deep anomaly detection methods, which focus on learning new feature representations to enable downstream anomaly detection methods, perform indirect optimization of anomaly scores, leading to data-inefficient learning and suboptimal anomaly scoring. Also, they are typically designed as unsupervised learning due to the lack of large-scale labeled anomaly data. As a result, they are difficult to leverage prior knowledge (e.g., a few labeled anomalies) when such information is available as in many real-world anomaly detection applications. This paper introduces a novel anomaly detection framework and its instantiation to address these problems. Instead of representation learning, our method fulfills an end-to-end learning of anomaly scores by a neural deviation learning, in which we leverage a few (e.g., multiple to dozens) labeled anomalies and a prior probability to enforce statistically significant deviations of the anomaly scores of anomalies from that of normal data objects in the upper tail. Extensive results show that our method can be trained substantially more data-efficiently and achieves significantly better anomaly scoring than state-of-the-art competing methods. 2019-08-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7138 info:doi/10.1145/3292500.3330871 https://ink.library.smu.edu.sg/context/sis_research/article/8141/viewcontent/3292500.3330871.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Anomaly Detection Deep Learning Representation Learning Neural Networks Outlier Detection Databases and Information Systems OS and Networks
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Anomaly Detection
Deep Learning
Representation Learning
Neural Networks
Outlier Detection
Databases and Information Systems
OS and Networks
spellingShingle Anomaly Detection
Deep Learning
Representation Learning
Neural Networks
Outlier Detection
Databases and Information Systems
OS and Networks
PANG, Guansong
SHEN, Chunhua
HENGEL, Anton van den
Deep anomaly detection with deviation networks
description Although deep learning has been applied to successfully address many data mining problems, relatively limited work has been done on deep learning for anomaly detection. Existing deep anomaly detection methods, which focus on learning new feature representations to enable downstream anomaly detection methods, perform indirect optimization of anomaly scores, leading to data-inefficient learning and suboptimal anomaly scoring. Also, they are typically designed as unsupervised learning due to the lack of large-scale labeled anomaly data. As a result, they are difficult to leverage prior knowledge (e.g., a few labeled anomalies) when such information is available as in many real-world anomaly detection applications. This paper introduces a novel anomaly detection framework and its instantiation to address these problems. Instead of representation learning, our method fulfills an end-to-end learning of anomaly scores by a neural deviation learning, in which we leverage a few (e.g., multiple to dozens) labeled anomalies and a prior probability to enforce statistically significant deviations of the anomaly scores of anomalies from that of normal data objects in the upper tail. Extensive results show that our method can be trained substantially more data-efficiently and achieves significantly better anomaly scoring than state-of-the-art competing methods.
format text
author PANG, Guansong
SHEN, Chunhua
HENGEL, Anton van den
author_facet PANG, Guansong
SHEN, Chunhua
HENGEL, Anton van den
author_sort PANG, Guansong
title Deep anomaly detection with deviation networks
title_short Deep anomaly detection with deviation networks
title_full Deep anomaly detection with deviation networks
title_fullStr Deep anomaly detection with deviation networks
title_full_unstemmed Deep anomaly detection with deviation networks
title_sort deep anomaly detection with deviation networks
publisher Institutional Knowledge at Singapore Management University
publishDate 2019
url https://ink.library.smu.edu.sg/sis_research/7138
https://ink.library.smu.edu.sg/context/sis_research/article/8141/viewcontent/3292500.3330871.pdf
_version_ 1770576230191464448