Exploring structural knowledge for automated visual inspection of moving trains

Deep learning methods are becoming the de-facto standard for generic visual recognition in the literature. However, their adaptations to industrial scenarios, such as visual recognition for machines, product streamlines, etc., which consist of countless components, have not been investigated well ye...

Full description

Saved in:
Bibliographic Details
Main Authors: CHEN, Cen, ZOU, Xiaofeng, ZENG, Zeng, CHENG, Zhongyao, ZHANG, Le, HOI, Steven C. H.
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2022
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7242
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8245
record_format dspace
spelling sg-smu-ink.sis_research-82452022-09-02T06:15:50Z Exploring structural knowledge for automated visual inspection of moving trains CHEN, Cen ZOU, Xiaofeng ZENG, Zeng CHENG, Zhongyao ZHANG, Le HOI, Steven C. H. Deep learning methods are becoming the de-facto standard for generic visual recognition in the literature. However, their adaptations to industrial scenarios, such as visual recognition for machines, product streamlines, etc., which consist of countless components, have not been investigated well yet. Compared with the generic object detection, there is some strong structural knowledge in these scenarios (e.g., fixed relative positions of components, component relationships, etc.). A case worth exploring could be automated visual inspection for trains, where there are various correlated components. However, the dominant object detection paradigm is limited by treating the visual features of each object region separately without considering common sense knowledge among objects. In this article, we propose a novel automated visual inspection framework for trains exploring structural knowledge for train component detection, which is called SKTCD. SKTCD is an end-to-end trainable framework, in which the visual features of train components and structural knowledge (including hierarchical scene contexts and spatial-aware component relationships) are jointly exploited for train component detection. We propose novel residual multiple gated recurrent units (Res-MGRUs) that can optimally fuse the visual features of train components and messages from the structural knowledge in a weighted-recurrent way. In order to verify the feasibility of SKTCD, a dataset that contains high-resolution images captured from moving trains has been collected, in which 18 590 critical train components are manually annotated. Extensive experiments on this dataset and on the PASCAL VOC dataset have demonstrated that SKTCD outperforms the existing challenging baselines significantly. The dataset as well as the source code can be downloaded online (https://github.com/smartprobe/SKCD). 2022-02-01T08:00:00Z text https://ink.library.smu.edu.sg/sis_research/7242 info:doi/10.1109/TCYB.2020.2998126 Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Object detection Visualization Fasteners Feature extraction Inspection Proposals Wheels Automated visual inspection deep convolutional neural networks (DCNNs) object detection train component detection Artificial Intelligence and Robotics Databases and Information Systems
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Object detection
Visualization
Fasteners
Feature extraction
Inspection
Proposals
Wheels
Automated visual inspection
deep convolutional neural networks (DCNNs)
object detection
train component detection
Artificial Intelligence and Robotics
Databases and Information Systems
spellingShingle Object detection
Visualization
Fasteners
Feature extraction
Inspection
Proposals
Wheels
Automated visual inspection
deep convolutional neural networks (DCNNs)
object detection
train component detection
Artificial Intelligence and Robotics
Databases and Information Systems
CHEN, Cen
ZOU, Xiaofeng
ZENG, Zeng
CHENG, Zhongyao
ZHANG, Le
HOI, Steven C. H.
Exploring structural knowledge for automated visual inspection of moving trains
description Deep learning methods are becoming the de-facto standard for generic visual recognition in the literature. However, their adaptations to industrial scenarios, such as visual recognition for machines, product streamlines, etc., which consist of countless components, have not been investigated well yet. Compared with the generic object detection, there is some strong structural knowledge in these scenarios (e.g., fixed relative positions of components, component relationships, etc.). A case worth exploring could be automated visual inspection for trains, where there are various correlated components. However, the dominant object detection paradigm is limited by treating the visual features of each object region separately without considering common sense knowledge among objects. In this article, we propose a novel automated visual inspection framework for trains exploring structural knowledge for train component detection, which is called SKTCD. SKTCD is an end-to-end trainable framework, in which the visual features of train components and structural knowledge (including hierarchical scene contexts and spatial-aware component relationships) are jointly exploited for train component detection. We propose novel residual multiple gated recurrent units (Res-MGRUs) that can optimally fuse the visual features of train components and messages from the structural knowledge in a weighted-recurrent way. In order to verify the feasibility of SKTCD, a dataset that contains high-resolution images captured from moving trains has been collected, in which 18 590 critical train components are manually annotated. Extensive experiments on this dataset and on the PASCAL VOC dataset have demonstrated that SKTCD outperforms the existing challenging baselines significantly. The dataset as well as the source code can be downloaded online (https://github.com/smartprobe/SKCD).
format text
author CHEN, Cen
ZOU, Xiaofeng
ZENG, Zeng
CHENG, Zhongyao
ZHANG, Le
HOI, Steven C. H.
author_facet CHEN, Cen
ZOU, Xiaofeng
ZENG, Zeng
CHENG, Zhongyao
ZHANG, Le
HOI, Steven C. H.
author_sort CHEN, Cen
title Exploring structural knowledge for automated visual inspection of moving trains
title_short Exploring structural knowledge for automated visual inspection of moving trains
title_full Exploring structural knowledge for automated visual inspection of moving trains
title_fullStr Exploring structural knowledge for automated visual inspection of moving trains
title_full_unstemmed Exploring structural knowledge for automated visual inspection of moving trains
title_sort exploring structural knowledge for automated visual inspection of moving trains
publisher Institutional Knowledge at Singapore Management University
publishDate 2022
url https://ink.library.smu.edu.sg/sis_research/7242
_version_ 1770576289451737088