Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection

Current polyp detection methods from colonoscopy videos use exclusively normal (i.e., healthy) training images, which i) ignore the importance of temporal information in consecutive video frames, and ii) lack knowledge about the polyps. Consequently, they often have high detection errors, especially...

Full description

Saved in:
Bibliographic Details
Main Authors: YU, Tian, PANG, Guansong, LIU, Fengbei, LIU, Yuyuan, WANG, Chong, CHEN, Yuanhong, VERJANS, Johan, CARNEIRO, Gustavo
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2022
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7549
https://ink.library.smu.edu.sg/context/sis_research/article/8552/viewcontent/2203.12121.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8552
record_format dspace
spelling sg-smu-ink.sis_research-85522022-11-29T07:07:01Z Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection YU, Tian PANG, Guansong LIU, Fengbei LIU, Yuyuan WANG, Chong CHEN, Yuanhong VERJANS, Johan CARNEIRO, Gustavo Current polyp detection methods from colonoscopy videos use exclusively normal (i.e., healthy) training images, which i) ignore the importance of temporal information in consecutive video frames, and ii) lack knowledge about the polyps. Consequently, they often have high detection errors, especially on challenging polyp cases (e.g., small, flat, or partially visible polyps). In this work, we formulate polyp detection as a weakly-supervised anomaly detection task that uses video-level labelled training data to detect frame-level polyps. In particular, we propose a novel convolutional transformer-based multiple instance learning method designed to identify abnormal frames (i.e., frames with polyps) from anomalous videos (i.e., videos containing at least one frame with polyp). In our method, local and global temporal dependencies are seamlessly captured while we simultaneously optimise video and snippet-level anomaly scores. A contrastive snippet mining method is also proposed to enable an effective modelling of the challenging polyp cases. The resulting method achieves a detection accuracy that is substantially better than current state-of-the-art approaches on a new large-scale colonoscopy video dataset introduced in this work. Our code and dataset are available at https://github.com/tianyu0207/weakly-polyp. 2022-09-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7549 info:doi/10.1007/978-3-031-16437-8_9 https://ink.library.smu.edu.sg/context/sis_research/article/8552/viewcontent/2203.12121.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Polyp detection Colonoscopy Weakly-supervised learning Video anomaly detection Vision transformer Artificial Intelligence and Robotics Medical Sciences
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Polyp detection
Colonoscopy
Weakly-supervised learning
Video anomaly detection
Vision transformer
Artificial Intelligence and Robotics
Medical Sciences
spellingShingle Polyp detection
Colonoscopy
Weakly-supervised learning
Video anomaly detection
Vision transformer
Artificial Intelligence and Robotics
Medical Sciences
YU, Tian
PANG, Guansong
LIU, Fengbei
LIU, Yuyuan
WANG, Chong
CHEN, Yuanhong
VERJANS, Johan
CARNEIRO, Gustavo
Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection
description Current polyp detection methods from colonoscopy videos use exclusively normal (i.e., healthy) training images, which i) ignore the importance of temporal information in consecutive video frames, and ii) lack knowledge about the polyps. Consequently, they often have high detection errors, especially on challenging polyp cases (e.g., small, flat, or partially visible polyps). In this work, we formulate polyp detection as a weakly-supervised anomaly detection task that uses video-level labelled training data to detect frame-level polyps. In particular, we propose a novel convolutional transformer-based multiple instance learning method designed to identify abnormal frames (i.e., frames with polyps) from anomalous videos (i.e., videos containing at least one frame with polyp). In our method, local and global temporal dependencies are seamlessly captured while we simultaneously optimise video and snippet-level anomaly scores. A contrastive snippet mining method is also proposed to enable an effective modelling of the challenging polyp cases. The resulting method achieves a detection accuracy that is substantially better than current state-of-the-art approaches on a new large-scale colonoscopy video dataset introduced in this work. Our code and dataset are available at https://github.com/tianyu0207/weakly-polyp.
format text
author YU, Tian
PANG, Guansong
LIU, Fengbei
LIU, Yuyuan
WANG, Chong
CHEN, Yuanhong
VERJANS, Johan
CARNEIRO, Gustavo
author_facet YU, Tian
PANG, Guansong
LIU, Fengbei
LIU, Yuyuan
WANG, Chong
CHEN, Yuanhong
VERJANS, Johan
CARNEIRO, Gustavo
author_sort YU, Tian
title Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection
title_short Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection
title_full Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection
title_fullStr Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection
title_full_unstemmed Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection
title_sort contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection
publisher Institutional Knowledge at Singapore Management University
publishDate 2022
url https://ink.library.smu.edu.sg/sis_research/7549
https://ink.library.smu.edu.sg/context/sis_research/article/8552/viewcontent/2203.12121.pdf
_version_ 1770576370695405568