Deep visual saliency on stereoscopic images

Visual saliency on stereoscopic 3D (S3D) images has been shown to be heavily influenced by image quality. Hence, this dependency is an important factor in image quality prediction, image restoration and discomfort reduction, but it is still very difficult to predict such a nonlinear relation in imag...

Full description

Saved in:
Bibliographic Details
Main Authors: Nguyen, Anh-Duc, Kim, Jongyoo, Oh, Heeseok, Kim, Haksub, Lin, Weisi, Lee, Sanghoon
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2020
Subjects:
Online Access:https://hdl.handle.net/10356/142326
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-142326
record_format dspace
spelling sg-ntu-dr.10356-1423262020-06-19T04:20:52Z Deep visual saliency on stereoscopic images Nguyen, Anh-Duc Kim, Jongyoo Oh, Heeseok Kim, Haksub Lin, Weisi Lee, Sanghoon School of Computer Science and Engineering Engineering::Computer science and engineering Saliency Prediction Stereoscopic Image Visual saliency on stereoscopic 3D (S3D) images has been shown to be heavily influenced by image quality. Hence, this dependency is an important factor in image quality prediction, image restoration and discomfort reduction, but it is still very difficult to predict such a nonlinear relation in images. In addition, most algorithms specialized in detecting visual saliency on pristine images may unsurprisingly fail when facing distorted images. In this paper, we investigate a deep learning scheme named Deep Visual Saliency (DeepVS) to achieve a more accurate and reliable saliency predictor even in the presence of distortions. Since visual saliency is influenced by low-level features (contrast, luminance, and depth information) from a psychophysical point of view, we propose seven low-level features derived from S3D image pairs and utilize them in the context of deep learning to detect visual attention adaptively to human perception. During analysis, it turns out that the low-level features play a role to extract distortion and saliency information. To construct saliency predictors, we weight and model the human visual saliency through two different network architectures, a regression and a fully convolutional neural networks. Our results from thorough experiments confirm that the predicted saliency maps are up to 70% correlated with human gaze patterns, which emphasize the need for the hand-crafted features as input to deep neural networks in S3D saliency detection. 2020-06-19T04:20:52Z 2020-06-19T04:20:52Z 2018 Journal Article Nguyen, A.-D., Kim, J., Oh, H., Kim, H., Lin, W., & Lee, S. (2019). Deep visual saliency on stereoscopic images. IEEE Transactions on Image Processing, 28(4), 1939-1953. doi:10.1109/TIP.2018.2879408 1057-7149 https://hdl.handle.net/10356/142326 10.1109/TIP.2018.2879408 2-s2.0-85056147202 4 28 1939 1953 en IEEE Transactions on Image Processing © 2018 IEEE. All rights reserved.
institution Nanyang Technological University
building NTU Library
country Singapore
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Saliency Prediction
Stereoscopic Image
spellingShingle Engineering::Computer science and engineering
Saliency Prediction
Stereoscopic Image
Nguyen, Anh-Duc
Kim, Jongyoo
Oh, Heeseok
Kim, Haksub
Lin, Weisi
Lee, Sanghoon
Deep visual saliency on stereoscopic images
description Visual saliency on stereoscopic 3D (S3D) images has been shown to be heavily influenced by image quality. Hence, this dependency is an important factor in image quality prediction, image restoration and discomfort reduction, but it is still very difficult to predict such a nonlinear relation in images. In addition, most algorithms specialized in detecting visual saliency on pristine images may unsurprisingly fail when facing distorted images. In this paper, we investigate a deep learning scheme named Deep Visual Saliency (DeepVS) to achieve a more accurate and reliable saliency predictor even in the presence of distortions. Since visual saliency is influenced by low-level features (contrast, luminance, and depth information) from a psychophysical point of view, we propose seven low-level features derived from S3D image pairs and utilize them in the context of deep learning to detect visual attention adaptively to human perception. During analysis, it turns out that the low-level features play a role to extract distortion and saliency information. To construct saliency predictors, we weight and model the human visual saliency through two different network architectures, a regression and a fully convolutional neural networks. Our results from thorough experiments confirm that the predicted saliency maps are up to 70% correlated with human gaze patterns, which emphasize the need for the hand-crafted features as input to deep neural networks in S3D saliency detection.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Nguyen, Anh-Duc
Kim, Jongyoo
Oh, Heeseok
Kim, Haksub
Lin, Weisi
Lee, Sanghoon
format Article
author Nguyen, Anh-Duc
Kim, Jongyoo
Oh, Heeseok
Kim, Haksub
Lin, Weisi
Lee, Sanghoon
author_sort Nguyen, Anh-Duc
title Deep visual saliency on stereoscopic images
title_short Deep visual saliency on stereoscopic images
title_full Deep visual saliency on stereoscopic images
title_fullStr Deep visual saliency on stereoscopic images
title_full_unstemmed Deep visual saliency on stereoscopic images
title_sort deep visual saliency on stereoscopic images
publishDate 2020
url https://hdl.handle.net/10356/142326
_version_ 1681059779368189952