Visual relationship detection with contextual information

Understanding an image goes beyond recognizing and locating the objects in it, the relationships between objects also very important in image understanding. Most previous methods have focused on recognizing local predictions of the relationships. But real-world image relationships often determined b...

Full description

Saved in:
Bibliographic Details
Main Authors: Li, Yugang, Wang, Yongbin, Chen, Zhe, Zhu, Yuting
Other Authors: School of Electrical and Electronic Engineering
Format: Article
Language:English
Published: 2021
Subjects:
Online Access:https://hdl.handle.net/10356/146883
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-146883
record_format dspace
spelling sg-ntu-dr.10356-1468832021-03-12T06:34:55Z Visual relationship detection with contextual information Li, Yugang Wang, Yongbin Chen, Zhe Zhu, Yuting School of Electrical and Electronic Engineering Engineering::Computer science and engineering Visual Relationship Deep Learning Understanding an image goes beyond recognizing and locating the objects in it, the relationships between objects also very important in image understanding. Most previous methods have focused on recognizing local predictions of the relationships. But real-world image relationships often determined by the surrounding objects and other contextual information. In this work, we employ this insight to propose a novel framework to deal with the problem of visual relationship detection. The core of the framework is a relationship inference network, which is a recurrent structure designed for combining the global contextual information of the object to infer the relationship of the image. Experimental results on Stanford VRD and Visual Genome demonstrate that the proposed method achieves a good performance both in efficiency and accuracy. Finally, we demonstrate the value of visual relationship on two computer vision tasks: image retrieval and scene graph generation. Published version 2021-03-12T06:34:54Z 2021-03-12T06:34:54Z 2020 Journal Article Li, Y., Wang, Y., Chen, Z. & Zhu, Y. (2020). Visual relationship detection with contextual information. Computers, Materials and Continua, 63(3), 1575-1589. https://dx.doi.org/10.32604/CMC.2020.07451 1546-2218 https://hdl.handle.net/10356/146883 10.32604/CMC.2020.07451 2-s2.0-85091087063 3 63 1575 1589 en Computers, Materials and Continua © 2020 The Author(s). This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Visual Relationship
Deep Learning
spellingShingle Engineering::Computer science and engineering
Visual Relationship
Deep Learning
Li, Yugang
Wang, Yongbin
Chen, Zhe
Zhu, Yuting
Visual relationship detection with contextual information
description Understanding an image goes beyond recognizing and locating the objects in it, the relationships between objects also very important in image understanding. Most previous methods have focused on recognizing local predictions of the relationships. But real-world image relationships often determined by the surrounding objects and other contextual information. In this work, we employ this insight to propose a novel framework to deal with the problem of visual relationship detection. The core of the framework is a relationship inference network, which is a recurrent structure designed for combining the global contextual information of the object to infer the relationship of the image. Experimental results on Stanford VRD and Visual Genome demonstrate that the proposed method achieves a good performance both in efficiency and accuracy. Finally, we demonstrate the value of visual relationship on two computer vision tasks: image retrieval and scene graph generation.
author2 School of Electrical and Electronic Engineering
author_facet School of Electrical and Electronic Engineering
Li, Yugang
Wang, Yongbin
Chen, Zhe
Zhu, Yuting
format Article
author Li, Yugang
Wang, Yongbin
Chen, Zhe
Zhu, Yuting
author_sort Li, Yugang
title Visual relationship detection with contextual information
title_short Visual relationship detection with contextual information
title_full Visual relationship detection with contextual information
title_fullStr Visual relationship detection with contextual information
title_full_unstemmed Visual relationship detection with contextual information
title_sort visual relationship detection with contextual information
publishDate 2021
url https://hdl.handle.net/10356/146883
_version_ 1695636081027842048