Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features

This paper deals with the problem of recognizing the unique role in dynamic environments. Different from social roles, the unique role refers to those who are unusual in their carrying items or movements in the scene. In this paper, we propose a hierarchical probabilistic reasoning method that relat...

Full description

Saved in:
Bibliographic Details
Main Authors: Yang, Chule, Yue, Yufeng, Zhang, Jun, Wen, Mingxing, Wang, Danwei
Other Authors: School of Electrical and Electronic Engineering
Format: Article
Language:English
Published: 2019
Subjects:
Online Access:https://hdl.handle.net/10356/106302
http://hdl.handle.net/10220/48869
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-106302
record_format dspace
spelling sg-ntu-dr.10356-1063022020-06-08T09:33:34Z Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features Yang, Chule Yue, Yufeng Zhang, Jun Wen, Mingxing Wang, Danwei School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics Probabilistic Logic Semantics This paper deals with the problem of recognizing the unique role in dynamic environments. Different from social roles, the unique role refers to those who are unusual in their carrying items or movements in the scene. In this paper, we propose a hierarchical probabilistic reasoning method that relates spatial relationships between interested objects and humans with their temporal changes to recognize the unique individual. Two observation models, Object Existence Model (OEM) and Human Action Model (HAM), are established to support role inference by analyzing the corresponding semantic-interaction features and spatio-temporal features. Then, OEM and HAM results of each person are compared with the overall distribution in the scene, respectively. Finally, we can determine the role through the fusion of two observation models. Experiments are conducted in both indoor and outdoor environments concerning different settings, degrees of clutter, and occlusions. The results show that the proposed method can adapt to a variety of scenarios and outperforms other methods on accuracy and robustness, moreover, exhibiting stable performance even in complex scenes. NRF (Natl Research Foundation, S’pore) Accepted version 2019-06-20T05:16:48Z 2019-12-06T22:08:34Z 2019-06-20T05:16:48Z 2019-12-06T22:08:34Z 2019 2019 Journal Article Yang, C., Yue, Y., Zhang, J., Wen, M., & Wang, D. (2019). Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features. IEEE Transactions on Multimedia, 21(5), 1195-1208. doi:10.1109/TMM.2018.2875513 1520-9210 https://hdl.handle.net/10356/106302 http://hdl.handle.net/10220/48869 10.1109/TMM.2018.2875513 210580 5 21 1195 1208 210580 en IEEE Transactions on Multimedia © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: https://doi.org/10.1109/TMM.2018.2875513 14 p. application/pdf
institution Nanyang Technological University
building NTU Library
country Singapore
collection DR-NTU
language English
topic Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics
Probabilistic Logic
Semantics
spellingShingle Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics
Probabilistic Logic
Semantics
Yang, Chule
Yue, Yufeng
Zhang, Jun
Wen, Mingxing
Wang, Danwei
Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features
description This paper deals with the problem of recognizing the unique role in dynamic environments. Different from social roles, the unique role refers to those who are unusual in their carrying items or movements in the scene. In this paper, we propose a hierarchical probabilistic reasoning method that relates spatial relationships between interested objects and humans with their temporal changes to recognize the unique individual. Two observation models, Object Existence Model (OEM) and Human Action Model (HAM), are established to support role inference by analyzing the corresponding semantic-interaction features and spatio-temporal features. Then, OEM and HAM results of each person are compared with the overall distribution in the scene, respectively. Finally, we can determine the role through the fusion of two observation models. Experiments are conducted in both indoor and outdoor environments concerning different settings, degrees of clutter, and occlusions. The results show that the proposed method can adapt to a variety of scenarios and outperforms other methods on accuracy and robustness, moreover, exhibiting stable performance even in complex scenes.
author2 School of Electrical and Electronic Engineering
author_facet School of Electrical and Electronic Engineering
Yang, Chule
Yue, Yufeng
Zhang, Jun
Wen, Mingxing
Wang, Danwei
format Article
author Yang, Chule
Yue, Yufeng
Zhang, Jun
Wen, Mingxing
Wang, Danwei
author_sort Yang, Chule
title Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features
title_short Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features
title_full Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features
title_fullStr Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features
title_full_unstemmed Probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features
title_sort probabilistic reasoning for unique role recognition based on the fusion of semantic-interaction and spatio-temporal features
publishDate 2019
url https://hdl.handle.net/10356/106302
http://hdl.handle.net/10220/48869
_version_ 1681058960297164800