Weakly-supervised deep anomaly detection with pairwise relation learning

This paper studies a rarely explored but critical anomaly detection problem: weakly-supervised anomaly detection with limited labeled anomalies and a large unlabeled data set. This problem is very important because it (i) enables anomalyinformed modeling which helps identify anomalies of interests a...

Full description

Saved in:
Bibliographic Details
Main Authors: PANG, Guansong, HENGEL, Anton Van Den, SHEN, Chuanhua
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2019
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7025
https://ink.library.smu.edu.sg/context/sis_research/article/8028/viewcontent/1910.13601v1.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8028
record_format dspace
spelling sg-smu-ink.sis_research-80282022-03-17T15:03:00Z Weakly-supervised deep anomaly detection with pairwise relation learning PANG, Guansong HENGEL, Anton Van Den SHEN, Chuanhua This paper studies a rarely explored but critical anomaly detection problem: weakly-supervised anomaly detection with limited labeled anomalies and a large unlabeled data set. This problem is very important because it (i) enables anomalyinformed modeling which helps identify anomalies of interests and address the notorious high false positives in unsupervised anomaly detection, and (ii) eliminates the reliance on large-scale and complete labeled anomaly data in fullysupervised settings. However, the problem is especially challenging since we have only limited labeled data for a single class, and moreover, the seen anomalies often cannot cover all types of anomalies (i.e., unseen anomalies). We address this problem by formulating the problem as a pairwise relation learning task. Particularly, our approach defines a two-stream ordinal regression network to learn the relation of randomly selected instance pairs, i.e., whether the instance pair contains labeled anomalies or just unlabeled data instances. The resulting model leverages both the labeled and unlabeled data to effectively augment the data and learn generalized representations of both normality and abnormality. Extensive empirical results show that our approach (i) significantly outperforms state-of-the-art competing methods in detecting both seen and unseen anomalies and (ii) is substantially more data-efficient 2019-10-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7025 https://ink.library.smu.edu.sg/context/sis_research/article/8028/viewcontent/1910.13601v1.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Artificial Intelligence and Robotics Databases and Information Systems
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Artificial Intelligence and Robotics
Databases and Information Systems
spellingShingle Artificial Intelligence and Robotics
Databases and Information Systems
PANG, Guansong
HENGEL, Anton Van Den
SHEN, Chuanhua
Weakly-supervised deep anomaly detection with pairwise relation learning
description This paper studies a rarely explored but critical anomaly detection problem: weakly-supervised anomaly detection with limited labeled anomalies and a large unlabeled data set. This problem is very important because it (i) enables anomalyinformed modeling which helps identify anomalies of interests and address the notorious high false positives in unsupervised anomaly detection, and (ii) eliminates the reliance on large-scale and complete labeled anomaly data in fullysupervised settings. However, the problem is especially challenging since we have only limited labeled data for a single class, and moreover, the seen anomalies often cannot cover all types of anomalies (i.e., unseen anomalies). We address this problem by formulating the problem as a pairwise relation learning task. Particularly, our approach defines a two-stream ordinal regression network to learn the relation of randomly selected instance pairs, i.e., whether the instance pair contains labeled anomalies or just unlabeled data instances. The resulting model leverages both the labeled and unlabeled data to effectively augment the data and learn generalized representations of both normality and abnormality. Extensive empirical results show that our approach (i) significantly outperforms state-of-the-art competing methods in detecting both seen and unseen anomalies and (ii) is substantially more data-efficient
format text
author PANG, Guansong
HENGEL, Anton Van Den
SHEN, Chuanhua
author_facet PANG, Guansong
HENGEL, Anton Van Den
SHEN, Chuanhua
author_sort PANG, Guansong
title Weakly-supervised deep anomaly detection with pairwise relation learning
title_short Weakly-supervised deep anomaly detection with pairwise relation learning
title_full Weakly-supervised deep anomaly detection with pairwise relation learning
title_fullStr Weakly-supervised deep anomaly detection with pairwise relation learning
title_full_unstemmed Weakly-supervised deep anomaly detection with pairwise relation learning
title_sort weakly-supervised deep anomaly detection with pairwise relation learning
publisher Institutional Knowledge at Singapore Management University
publishDate 2019
url https://ink.library.smu.edu.sg/sis_research/7025
https://ink.library.smu.edu.sg/context/sis_research/article/8028/viewcontent/1910.13601v1.pdf
_version_ 1770576190125375488