Person-level action recognition in complex events via TSD-TSM networks

The task of person-level action recognition in complex events aims to densely detect pedestrians and individually predict their actions from surveillance videos. In this paper, we present a simple yet efficient pipeline for this task, referred to as TSD-TSM networks. Firstly, we adopt the TSD detect...

Full description

Saved in:
Bibliographic Details
Main Authors: HAO, Yanbin, LIU, Zi-Niu, ZHANG, Hao, ZHU, Bin, CHEN, Jingjing, JIANG, Yu-Gang, NGO, Chong-wah
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2020
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/6503
https://ink.library.smu.edu.sg/context/sis_research/article/7506/viewcontent/3394171.3416276.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-7506
record_format dspace
spelling sg-smu-ink.sis_research-75062022-01-10T04:52:46Z Person-level action recognition in complex events via TSD-TSM networks HAO, Yanbin LIU, Zi-Niu ZHANG, Hao ZHU, Bin CHEN, Jingjing JIANG, Yu-Gang NGO, Chong-wah The task of person-level action recognition in complex events aims to densely detect pedestrians and individually predict their actions from surveillance videos. In this paper, we present a simple yet efficient pipeline for this task, referred to as TSD-TSM networks. Firstly, we adopt the TSD detector for the pedestrian localization on each single keyframe. Secondly, we generate the sequential ROIs for a person proposal by replicating the adjusted bounding box coordinates around the keyframe. Particularly, we propose to conduct straddling expansion and region squaring on the original bounding box of a person proposal to widen the potential space of motion and interaction and lead to a square box for ROI detection. Finally, we adapt the TSM classifier on the generated ROI sequences to perform action classification and further adopt late fusion to promote the prediction. Our proposed pipeline achieved the 3rd place in the ACM-MM 2020 grand challenge, i.e., Large-scale Human-centric Video Analysis in Complex Events (Track-4), obtaining final 15.31% wf-mAP@avg and 20.63% f-mAP@avg on the testing set. 2020-10-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6503 info:doi/10.1145/3394171.3416276 https://ink.library.smu.edu.sg/context/sis_research/article/7506/viewcontent/3394171.3416276.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University complex events human action recognition pedestrian detection Databases and Information Systems Graphics and Human Computer Interfaces
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic complex events
human action recognition
pedestrian detection
Databases and Information Systems
Graphics and Human Computer Interfaces
spellingShingle complex events
human action recognition
pedestrian detection
Databases and Information Systems
Graphics and Human Computer Interfaces
HAO, Yanbin
LIU, Zi-Niu
ZHANG, Hao
ZHU, Bin
CHEN, Jingjing
JIANG, Yu-Gang
NGO, Chong-wah
Person-level action recognition in complex events via TSD-TSM networks
description The task of person-level action recognition in complex events aims to densely detect pedestrians and individually predict their actions from surveillance videos. In this paper, we present a simple yet efficient pipeline for this task, referred to as TSD-TSM networks. Firstly, we adopt the TSD detector for the pedestrian localization on each single keyframe. Secondly, we generate the sequential ROIs for a person proposal by replicating the adjusted bounding box coordinates around the keyframe. Particularly, we propose to conduct straddling expansion and region squaring on the original bounding box of a person proposal to widen the potential space of motion and interaction and lead to a square box for ROI detection. Finally, we adapt the TSM classifier on the generated ROI sequences to perform action classification and further adopt late fusion to promote the prediction. Our proposed pipeline achieved the 3rd place in the ACM-MM 2020 grand challenge, i.e., Large-scale Human-centric Video Analysis in Complex Events (Track-4), obtaining final 15.31% wf-mAP@avg and 20.63% f-mAP@avg on the testing set.
format text
author HAO, Yanbin
LIU, Zi-Niu
ZHANG, Hao
ZHU, Bin
CHEN, Jingjing
JIANG, Yu-Gang
NGO, Chong-wah
author_facet HAO, Yanbin
LIU, Zi-Niu
ZHANG, Hao
ZHU, Bin
CHEN, Jingjing
JIANG, Yu-Gang
NGO, Chong-wah
author_sort HAO, Yanbin
title Person-level action recognition in complex events via TSD-TSM networks
title_short Person-level action recognition in complex events via TSD-TSM networks
title_full Person-level action recognition in complex events via TSD-TSM networks
title_fullStr Person-level action recognition in complex events via TSD-TSM networks
title_full_unstemmed Person-level action recognition in complex events via TSD-TSM networks
title_sort person-level action recognition in complex events via tsd-tsm networks
publisher Institutional Knowledge at Singapore Management University
publishDate 2020
url https://ink.library.smu.edu.sg/sis_research/6503
https://ink.library.smu.edu.sg/context/sis_research/article/7506/viewcontent/3394171.3416276.pdf
_version_ 1770575977326313472