Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder

Unsupervised anomaly detection (UAD) aims to find anomalous images by optimising a detector using a training set that contains only normal images. UAD approaches can be based on reconstruction methods, self-supervised approaches, and Imagenet pre-trained models. Reconstruction methods, which detect...

Full description

Saved in:
Bibliographic Details
Main Authors: TIAN, Yu, PANG, Guansong, LIU, Yuyuan, WANG, Chong, CHEN, Yuanhong, LIU, Fengbei, SINGH, Rajvinder, VERJANS, Johan W., WANG, Mengyu, CARNEIRO, Gustavo
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2023
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/8329
https://ink.library.smu.edu.sg/context/sis_research/article/9332/viewcontent/UnsupervisedAnomaly_av.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-9332
record_format dspace
spelling sg-smu-ink.sis_research-93322023-12-05T03:00:26Z Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder TIAN, Yu PANG, Guansong LIU, Yuyuan WANG, Chong CHEN, Yuanhong LIU, Fengbei SINGH, Rajvinder VERJANS, Johan W. WANG, Mengyu CARNEIRO, Gustavo Unsupervised anomaly detection (UAD) aims to find anomalous images by optimising a detector using a training set that contains only normal images. UAD approaches can be based on reconstruction methods, self-supervised approaches, and Imagenet pre-trained models. Reconstruction methods, which detect anomalies from image reconstruction errors, are advantageous because they do not rely on the design of problem-specific pretext tasks needed by self-supervised approaches, and on the unreliable translation of models pre-trained from non-medical datasets. However, reconstruction methods may fail because they can have low reconstruction errors even for anomalous images. In this paper, we introduce a new reconstruction-based UAD approach that addresses this low-reconstruction error issue for anomalous images. Our UAD approach, the memory-augmented multi-level cross-attentional masked autoencoder (MemMC-MAE), is a transformer-based approach, consisting of a novel memory-augmented self-attention operator for the encoder and a new multi-level cross-attention operator for the decoder. MemMC-MAE masks large parts of the input image during its reconstruction, reducing the risk that it will produce low reconstruction errors because anomalies are likely to be masked and cannot be reconstructed. However, when the anomaly is not masked, then the normal patterns stored in the encoder's memory combined with the decoder's multi-level cross-attention will constrain the accurate reconstruction of the anomaly. We show that our method achieves SOTA anomaly detection and localisation on colonoscopy, pneumonia, and covid-19 chest x-ray datasets. 2023-10-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/8329 info:doi/10.1007/978-3-031-45676-3_2 https://ink.library.smu.edu.sg/context/sis_research/article/9332/viewcontent/UnsupervisedAnomaly_av.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Pneumonia Covid-19 Colonoscopy Unsupervised Learning Anomaly Detection' Anomaly Segmentation Vision Transformer Artificial Intelligence and Robotics Graphics and Human Computer Interfaces
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Pneumonia
Covid-19
Colonoscopy
Unsupervised Learning
Anomaly Detection' Anomaly Segmentation
Vision Transformer
Artificial Intelligence and Robotics
Graphics and Human Computer Interfaces
spellingShingle Pneumonia
Covid-19
Colonoscopy
Unsupervised Learning
Anomaly Detection' Anomaly Segmentation
Vision Transformer
Artificial Intelligence and Robotics
Graphics and Human Computer Interfaces
TIAN, Yu
PANG, Guansong
LIU, Yuyuan
WANG, Chong
CHEN, Yuanhong
LIU, Fengbei
SINGH, Rajvinder
VERJANS, Johan W.
WANG, Mengyu
CARNEIRO, Gustavo
Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder
description Unsupervised anomaly detection (UAD) aims to find anomalous images by optimising a detector using a training set that contains only normal images. UAD approaches can be based on reconstruction methods, self-supervised approaches, and Imagenet pre-trained models. Reconstruction methods, which detect anomalies from image reconstruction errors, are advantageous because they do not rely on the design of problem-specific pretext tasks needed by self-supervised approaches, and on the unreliable translation of models pre-trained from non-medical datasets. However, reconstruction methods may fail because they can have low reconstruction errors even for anomalous images. In this paper, we introduce a new reconstruction-based UAD approach that addresses this low-reconstruction error issue for anomalous images. Our UAD approach, the memory-augmented multi-level cross-attentional masked autoencoder (MemMC-MAE), is a transformer-based approach, consisting of a novel memory-augmented self-attention operator for the encoder and a new multi-level cross-attention operator for the decoder. MemMC-MAE masks large parts of the input image during its reconstruction, reducing the risk that it will produce low reconstruction errors because anomalies are likely to be masked and cannot be reconstructed. However, when the anomaly is not masked, then the normal patterns stored in the encoder's memory combined with the decoder's multi-level cross-attention will constrain the accurate reconstruction of the anomaly. We show that our method achieves SOTA anomaly detection and localisation on colonoscopy, pneumonia, and covid-19 chest x-ray datasets.
format text
author TIAN, Yu
PANG, Guansong
LIU, Yuyuan
WANG, Chong
CHEN, Yuanhong
LIU, Fengbei
SINGH, Rajvinder
VERJANS, Johan W.
WANG, Mengyu
CARNEIRO, Gustavo
author_facet TIAN, Yu
PANG, Guansong
LIU, Yuyuan
WANG, Chong
CHEN, Yuanhong
LIU, Fengbei
SINGH, Rajvinder
VERJANS, Johan W.
WANG, Mengyu
CARNEIRO, Gustavo
author_sort TIAN, Yu
title Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder
title_short Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder
title_full Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder
title_fullStr Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder
title_full_unstemmed Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder
title_sort unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder
publisher Institutional Knowledge at Singapore Management University
publishDate 2023
url https://ink.library.smu.edu.sg/sis_research/8329
https://ink.library.smu.edu.sg/context/sis_research/article/9332/viewcontent/UnsupervisedAnomaly_av.pdf
_version_ 1784855636031307776