Uncertainty-guided appearance-motion association network for out-of-distribution action detection
Out-of-distribution (OOD) detection targets to detect and reject test samples with semantic shifts, to prevent models trained on in-distribution (ID) dataset from producing unreliable predictions. Existing works only extract the appearance features on image datasets, and cannot handle dynamic real-w...
Saved in:
Main Authors: | , , |
---|---|
Other Authors: | |
Format: | Conference or Workshop Item |
Language: | English |
Published: |
2024
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/178516 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-178516 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1785162024-10-20T15:37:18Z Uncertainty-guided appearance-motion association network for out-of-distribution action detection Fang, Xiang Arvind Easwaran Genest, Blaise Interdisciplinary Graduate School (IGS) College of Computing and Data Science 2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR) CNRS@CREATE Energy Research Institute @ NTU (ERI@N) Computer and Information Science Uncertainty-guided appearance-motion association Out-of-distribution action detection Out-of-distribution (OOD) detection targets to detect and reject test samples with semantic shifts, to prevent models trained on in-distribution (ID) dataset from producing unreliable predictions. Existing works only extract the appearance features on image datasets, and cannot handle dynamic real-world scenarios with much motion information. Therefore, we target a more realistic and challenging OOD detection task: OOD action detection (ODAD). Given an untrimmed video, ODAD first classifies the ID actions and recognizes the OOD actions, and then localizes ID and OOD actions. To this end, in this paper, we propose a novel Uncertainty-Guided Appearance-Motion Association Network (UAAN), which explores both appearance features and motion contexts to reason spatial-temporal inter-object interaction for ODAD. Firstly, we design separate appearance and motion branches to extract corresponding appearance-oriented and motion-aspect object representations. In each branch, we construct a spatial-temporal graph to reason appearance-guided and motion-driven inter-object interaction. Then, we design an appearance-motion attention module to fuse the appearance and motion features for final action detection. Experimental results on two challenging datasets show that our proposed UAAN beats state-of-the-art methods by a significant margin, which illustrates its effectiveness. Submitted/Accepted version 2024-10-17T01:08:00Z 2024-10-17T01:08:00Z 2024 Conference Paper Fang, X., Arvind Easwaran & Genest, B. (2024). Uncertainty-guided appearance-motion association network for out-of-distribution action detection. 2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR), 176-182. https://dx.doi.org/10.1109/MIPR62202.2024.00034 979-8-3503-5142-2 2770-4319 https://hdl.handle.net/10356/178516 10.1109/MIPR62202.2024.00034 176 182 en © 2024 IEEE. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online at http://doi.org/10.1109/MIPR62202.2024.00034. application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Computer and Information Science Uncertainty-guided appearance-motion association Out-of-distribution action detection |
spellingShingle |
Computer and Information Science Uncertainty-guided appearance-motion association Out-of-distribution action detection Fang, Xiang Arvind Easwaran Genest, Blaise Uncertainty-guided appearance-motion association network for out-of-distribution action detection |
description |
Out-of-distribution (OOD) detection targets to detect and reject test samples with semantic shifts, to prevent models trained on in-distribution (ID) dataset from producing unreliable predictions. Existing works only extract the appearance features on image datasets, and cannot handle dynamic real-world scenarios with much motion information. Therefore, we target a more realistic and challenging OOD detection task: OOD action detection (ODAD). Given an untrimmed video, ODAD first classifies the ID actions and recognizes the OOD actions, and then localizes ID and OOD actions. To this end, in this paper, we propose a novel Uncertainty-Guided Appearance-Motion Association Network (UAAN), which explores both appearance features and motion contexts to reason spatial-temporal inter-object interaction for ODAD. Firstly, we design separate appearance and motion branches to extract corresponding appearance-oriented and motion-aspect object representations. In each branch, we construct a spatial-temporal graph to reason appearance-guided and motion-driven inter-object interaction. Then, we design an appearance-motion attention module to fuse the appearance and motion features for final action detection. Experimental results on two challenging datasets show that our proposed UAAN beats state-of-the-art methods by a significant margin, which illustrates its effectiveness. |
author2 |
Interdisciplinary Graduate School (IGS) |
author_facet |
Interdisciplinary Graduate School (IGS) Fang, Xiang Arvind Easwaran Genest, Blaise |
format |
Conference or Workshop Item |
author |
Fang, Xiang Arvind Easwaran Genest, Blaise |
author_sort |
Fang, Xiang |
title |
Uncertainty-guided appearance-motion association network for out-of-distribution action detection |
title_short |
Uncertainty-guided appearance-motion association network for out-of-distribution action detection |
title_full |
Uncertainty-guided appearance-motion association network for out-of-distribution action detection |
title_fullStr |
Uncertainty-guided appearance-motion association network for out-of-distribution action detection |
title_full_unstemmed |
Uncertainty-guided appearance-motion association network for out-of-distribution action detection |
title_sort |
uncertainty-guided appearance-motion association network for out-of-distribution action detection |
publishDate |
2024 |
url |
https://hdl.handle.net/10356/178516 |
_version_ |
1814777768989163520 |