Local fusion networks with chained residual pooling for video action recognition

Action recognition is an important yet challenging problem. We here present a novel method, multistage local fusion networks with residual connections, to boost the performance of video action recognition. In realistic videos, an action instance may have a long time span and some frames may suffer f...

Full description

Saved in:
Bibliographic Details
Main Authors: He, Feixiang, Liu, Fayao, Yao, Rui, Lin, Guosheng
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2020
Subjects:
Online Access:https://hdl.handle.net/10356/143069
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-143069
record_format dspace
spelling sg-ntu-dr.10356-1430692020-07-28T01:31:10Z Local fusion networks with chained residual pooling for video action recognition He, Feixiang Liu, Fayao Yao, Rui Lin, Guosheng School of Computer Science and Engineering Engineering::Computer science and engineering Action Recognition Residual Connection Action recognition is an important yet challenging problem. We here present a novel method, multistage local fusion networks with residual connections, to boost the performance of video action recognition. In realistic videos, an action instance may have a long time span and some frames may suffer from deteriorated object appearance due to motion blur or video defocus. Our method enhances the per-frame representation by capturing information from neighboring frames. We propose a local fusion block which considers neighboring frames to capture appearance and local motion information for generating per-frame representation. Our local fusion is performed in a multistage manner allowing feature fusion from varying neighborhood sizes in the temporal dimension. We employ residual connections in the fusion blocks to enable effective gradient propagation through the whole network allowing effective end-to-end training. We achieve competitive results on two challenging and public available datasets, namely HMDB51 and UCF101, which shows the effectiveness of the proposed method. Accepted version 2020-07-28T01:31:10Z 2020-07-28T01:31:10Z 2018 Journal Article He, F., Liu, F., Yao, R., & Lin, G. (2019). Local fusion networks with chained residual pooling for video action recognition. Image and Vision Computing, 81, 34-41. doi:10.1016/j.imavis.2018.12.002 0262-8856 https://hdl.handle.net/10356/143069 10.1016/j.imavis.2018.12.002 2-s2.0-85059864452 81 34 41 en Image and Vision Computing © 2019 Elsevier B.V. All rights reserved. This paper was published in Image and Vision Computing and is made available with permission of Elsevier B.V. application/pdf
institution Nanyang Technological University
building NTU Library
country Singapore
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Action Recognition
Residual Connection
spellingShingle Engineering::Computer science and engineering
Action Recognition
Residual Connection
He, Feixiang
Liu, Fayao
Yao, Rui
Lin, Guosheng
Local fusion networks with chained residual pooling for video action recognition
description Action recognition is an important yet challenging problem. We here present a novel method, multistage local fusion networks with residual connections, to boost the performance of video action recognition. In realistic videos, an action instance may have a long time span and some frames may suffer from deteriorated object appearance due to motion blur or video defocus. Our method enhances the per-frame representation by capturing information from neighboring frames. We propose a local fusion block which considers neighboring frames to capture appearance and local motion information for generating per-frame representation. Our local fusion is performed in a multistage manner allowing feature fusion from varying neighborhood sizes in the temporal dimension. We employ residual connections in the fusion blocks to enable effective gradient propagation through the whole network allowing effective end-to-end training. We achieve competitive results on two challenging and public available datasets, namely HMDB51 and UCF101, which shows the effectiveness of the proposed method.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
He, Feixiang
Liu, Fayao
Yao, Rui
Lin, Guosheng
format Article
author He, Feixiang
Liu, Fayao
Yao, Rui
Lin, Guosheng
author_sort He, Feixiang
title Local fusion networks with chained residual pooling for video action recognition
title_short Local fusion networks with chained residual pooling for video action recognition
title_full Local fusion networks with chained residual pooling for video action recognition
title_fullStr Local fusion networks with chained residual pooling for video action recognition
title_full_unstemmed Local fusion networks with chained residual pooling for video action recognition
title_sort local fusion networks with chained residual pooling for video action recognition
publishDate 2020
url https://hdl.handle.net/10356/143069
_version_ 1681058723882074112