Temporally enhanced image object proposals for online video object and action detections

Despite the recent advances of image object proposals (IOPs) and video object proposals (VOPs), it still remains a challenge to apply them to online video object/action detection. To address this problem, we propose a novel form of image object proposals, Temporally Enhanced Image Object Proposals (...

Full description

Saved in:
Bibliographic Details
Main Authors: Yang, Jiong, Yuan, Junsong
Other Authors: Interdisciplinary Graduate School (IGS)
Format: Article
Language:English
Published: 2020
Subjects:
Online Access:https://hdl.handle.net/10356/142119
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-142119
record_format dspace
spelling sg-ntu-dr.10356-1421192020-06-16T03:20:48Z Temporally enhanced image object proposals for online video object and action detections Yang, Jiong Yuan, Junsong Interdisciplinary Graduate School (IGS) Engineering::Computer science and engineering Video Proposal Despite the recent advances of image object proposals (IOPs) and video object proposals (VOPs), it still remains a challenge to apply them to online video object/action detection. To address this problem, we propose a novel form of image object proposals, Temporally Enhanced Image Object Proposals (TE-IOPs), for online video object/action detection. The proposed TE-IOPs augment the existing IOPs at every frame by their temporal dynamics in the past few frames. We develop a dynamic programming scheme to efficiently search for such TE-IOPs in an online manner. Compared with existing VOPs that cannot run online, our TE-IOPs can be used for online detection. Compared with IOPs, our TE-IOPs bring rich temporal dynamics with minor computational cost. Experiments on benchmark datasets validate the superior performance of the proposed TE-IOPs over existing IOPs and VOPs, in terms of both the proposal re-ranking and the application of online action detection. NRF (Natl Research Foundation, S’pore) MOE (Min. of Education, S’pore) 2020-06-16T03:20:48Z 2020-06-16T03:20:48Z 2018 Journal Article Yang, J., & Yuan, J. (2018). Temporally enhanced image object proposals for online video object and action detections. Journal of Visual Communication and Image Representation, 53, 245-256. doi:10.1016/j.jvcir.2018.03.018 1047-3203 https://hdl.handle.net/10356/142119 10.1016/j.jvcir.2018.03.018 2-s2.0-85045451673 53 245 256 en Journal of Visual Communication and Image Representation © 2018 Elsevier Inc. All rights reserved.
institution Nanyang Technological University
building NTU Library
country Singapore
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Video
Proposal
spellingShingle Engineering::Computer science and engineering
Video
Proposal
Yang, Jiong
Yuan, Junsong
Temporally enhanced image object proposals for online video object and action detections
description Despite the recent advances of image object proposals (IOPs) and video object proposals (VOPs), it still remains a challenge to apply them to online video object/action detection. To address this problem, we propose a novel form of image object proposals, Temporally Enhanced Image Object Proposals (TE-IOPs), for online video object/action detection. The proposed TE-IOPs augment the existing IOPs at every frame by their temporal dynamics in the past few frames. We develop a dynamic programming scheme to efficiently search for such TE-IOPs in an online manner. Compared with existing VOPs that cannot run online, our TE-IOPs can be used for online detection. Compared with IOPs, our TE-IOPs bring rich temporal dynamics with minor computational cost. Experiments on benchmark datasets validate the superior performance of the proposed TE-IOPs over existing IOPs and VOPs, in terms of both the proposal re-ranking and the application of online action detection.
author2 Interdisciplinary Graduate School (IGS)
author_facet Interdisciplinary Graduate School (IGS)
Yang, Jiong
Yuan, Junsong
format Article
author Yang, Jiong
Yuan, Junsong
author_sort Yang, Jiong
title Temporally enhanced image object proposals for online video object and action detections
title_short Temporally enhanced image object proposals for online video object and action detections
title_full Temporally enhanced image object proposals for online video object and action detections
title_fullStr Temporally enhanced image object proposals for online video object and action detections
title_full_unstemmed Temporally enhanced image object proposals for online video object and action detections
title_sort temporally enhanced image object proposals for online video object and action detections
publishDate 2020
url https://hdl.handle.net/10356/142119
_version_ 1681058931737100288