Combining pose-invariant kinematic features and object context features for RGB-D action recognition
Action recognition using RGB-D cameras is a popular research topic. Recognising actions in a pose-invariant manner is very challenging due to view changes, posture changes and huge intra-class variations. This study aims to propose a novel pose-invariant action recognition framework based on kinemat...
Saved in:
Main Authors: | , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2019
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/102468 http://hdl.handle.net/10220/49519 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-102468 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1024682020-09-26T21:54:08Z Combining pose-invariant kinematic features and object context features for RGB-D action recognition Ramanathan, Manoj Kochanowicz, Jaroslaw Thalmann, Nadia Magnenat Institute for Media Innovation (IMI) Poseinvariant Kinematic Features Real-time Action/Activity Recognition Visual arts and music::Media Action recognition using RGB-D cameras is a popular research topic. Recognising actions in a pose-invariant manner is very challenging due to view changes, posture changes and huge intra-class variations. This study aims to propose a novel pose-invariant action recognition framework based on kinematic features and object context features. Using RGB, depth and skeletal joints, the proposed framework extracts a novel set of pose-invariant motion kinematic features based on 3D scene flow and captures the motion of body parts with respect to the body. The obtained features are converted to a human body centric space that allows partial viewinvariant recognition of actions. The proposed pose-invariant kinematic features are extracted for both foreground (RGB and depth) and skeleton joints and separate classifiers are trained. Bordacount based classifier decision fusion is employed to obtain an action recognition result. For capturing object context features, a convolutional neural network (CNN) classifier is proposed to identify the involved objects. The proposed context features also include temporal information on object interaction and help in obtaining a final action recognition. The proposed framework works even with non-upright human postures and allows simultaneous action recognition for multiple people, which are topics that remain comparatively unresearched. The performance and robustness of the proposed pose-invariant action recognition framework are tested on several benchmark datasets. We also show that the proposed method works in real-time. NRF (Natl Research Foundation, S’pore) Published version 2019-08-05T01:46:27Z 2019-12-06T20:55:28Z 2019-08-05T01:46:27Z 2019-12-06T20:55:28Z 2019 Journal Article Ramanathan, M., Kochanowicz, J., & Thalmann, N. M. (2019). Combining pose-invariant kinematic features and object context features for RGB-D action recognition. International Journal of Machine Learning and Computing, 9(1), 44-50. doi: 10.18178/ijmlc.2019.9.1.763 https://hdl.handle.net/10356/102468 http://hdl.handle.net/10220/49519 10.18178/ijmlc.2019.9.1.763 en International Journal of Machine Learning and Computing © 2019 The Author(s) (published by International Journal of Machine Learning and Computing). This is an open-access article distributed under the terms of the Creative Commons Attribution License. 7 p. application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
country |
Singapore |
collection |
DR-NTU |
language |
English |
topic |
Poseinvariant Kinematic Features Real-time Action/Activity Recognition Visual arts and music::Media |
spellingShingle |
Poseinvariant Kinematic Features Real-time Action/Activity Recognition Visual arts and music::Media Ramanathan, Manoj Kochanowicz, Jaroslaw Thalmann, Nadia Magnenat Combining pose-invariant kinematic features and object context features for RGB-D action recognition |
description |
Action recognition using RGB-D cameras is a popular research topic. Recognising actions in a pose-invariant manner is very challenging due to view changes, posture changes and huge intra-class variations. This study aims to propose a novel pose-invariant action recognition framework based on kinematic features and object context features. Using RGB, depth and skeletal joints, the proposed framework extracts a novel set of pose-invariant motion kinematic features based on 3D scene flow and captures the motion of body parts with respect to the body. The obtained features are converted to a human body centric space that allows partial viewinvariant recognition of actions. The proposed pose-invariant kinematic features are extracted for both foreground (RGB and depth) and skeleton joints and separate classifiers are trained. Bordacount based classifier decision fusion is employed to obtain an action recognition result. For capturing object context features, a convolutional neural network (CNN) classifier is proposed to identify the involved objects. The proposed context features also include temporal information on object interaction and help in obtaining a final action recognition. The proposed framework works even with non-upright human postures and allows simultaneous action recognition for multiple people, which are topics that remain comparatively unresearched. The performance and robustness of the proposed pose-invariant action recognition framework are tested on several benchmark datasets. We also show that the proposed method works in real-time. |
author2 |
Institute for Media Innovation (IMI) |
author_facet |
Institute for Media Innovation (IMI) Ramanathan, Manoj Kochanowicz, Jaroslaw Thalmann, Nadia Magnenat |
format |
Article |
author |
Ramanathan, Manoj Kochanowicz, Jaroslaw Thalmann, Nadia Magnenat |
author_sort |
Ramanathan, Manoj |
title |
Combining pose-invariant kinematic features and object context features for RGB-D action recognition |
title_short |
Combining pose-invariant kinematic features and object context features for RGB-D action recognition |
title_full |
Combining pose-invariant kinematic features and object context features for RGB-D action recognition |
title_fullStr |
Combining pose-invariant kinematic features and object context features for RGB-D action recognition |
title_full_unstemmed |
Combining pose-invariant kinematic features and object context features for RGB-D action recognition |
title_sort |
combining pose-invariant kinematic features and object context features for rgb-d action recognition |
publishDate |
2019 |
url |
https://hdl.handle.net/10356/102468 http://hdl.handle.net/10220/49519 |
_version_ |
1681057724766355456 |