Deep pixel-level matching via attention for video co-segmentation

• Main Authors: LI, Junliang, WONG, Hon-Cheng, HE, Shengfeng, LO, Sio-Long, ZHANG, Guifang, WANG, Wenxiao
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2020
• Subjects: video co-segmentation; pixel-level matching; attention; Information Security
• Online Access: https://ink.library.smu.edu.sg/sis_research/7852
• Institution: Singapore Management University

In video object co-segmentation, methods based on patch-level matching are widely leveraged to extract the similarity between video frames. However, these methods can easily lead to pixel misclassification because they reduce the precision of pixel localization; thus, the accuracies of the segmentation results of these methods are deducted. To address this problem, we propose a framework based on deep neural networks and equipped with a new attention module, which is designed for pixel-level matching to segment the object across video frames in this paper. In this attention module, the pixel-level matching step is able to compare the feature value of each pixel from one input frame with that of each pixel from another input frame for computing the similarity between two frames. Then a features fusion step is applied to efficiently fuse the feature maps of each frame with the similarity information for generating dense attention features. Finally, an up-sampling step refines the feature maps for obtaining high quality segmentation results by using these dense attention features. The ObMiC and DAVIS 2016 datasets were utilized to train and test our framework. Experimental results show that our framework achieves higher accuracy than those of other video segmentation methods that perform well in common information extraction.