Optimal region selection for stereoscopic video subtitle insertion

Stereoscopic subtitle insertion is a fundamental and essential element in stereoscopic film and TV industry. However, little work has been dedicated to the optimal region selection for stereoscopic subtitle insertion. In addition, there is no public database reported for the performance evaluation o...

Full description

Saved in:
Bibliographic Details
Main Authors: Yue, Guanghui, Hou, Chunping, Lei, Jianjun, Fang, Yuming, Lin, Weisi
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2020
Subjects:
Online Access:https://hdl.handle.net/10356/140153
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-140153
record_format dspace
spelling sg-ntu-dr.10356-1401532020-05-27T02:46:46Z Optimal region selection for stereoscopic video subtitle insertion Yue, Guanghui Hou, Chunping Lei, Jianjun Fang, Yuming Lin, Weisi School of Computer Science and Engineering Engineering::Computer science and engineering Stereoscopic Subtitle Visual Discomfort Stereoscopic subtitle insertion is a fundamental and essential element in stereoscopic film and TV industry. However, little work has been dedicated to the optimal region selection for stereoscopic subtitle insertion. In addition, there is no public database reported for the performance evaluation of it. In this paper, we build the first large-scale video database (TJU3D) for stereoscopic video subtitle insertion, which includes 50 video sequences with rich screen scenes. Compared with 2D subtitle region selection, there are several problems we have to consider in stereoscopic subtitle region selection: 1) the subtitle should avoid depth cue collision and occlusion from objects in stereoscopic video sequences; 2) the disparity value of the subtitle must be minimized to reduce visual discomfort; and 3) the temporal coherence constraint must be considered during region selection for subtitles in video sequences. By considering these constraints, we propose an optimal region selection algorithm for stereoscopic subtitle insertion. First, we compute the disparity map of each video frame in video sequences. For each frame, the optimal position and disparity value of the subtitle are determined by a subtitle region selection algorithm, which contains two parts (i.e., the coarse selection and fine selection). After that, by considering the temporal consistency between adjacent frames, the position and disparity value of each frame are further classified and processed in order to avoid the subtitle jitter. We evaluate the proposed method on TJU3D video database through two visual discomfort prediction metrics and one subjective experiment. To further verify the effectiveness of the proposed method, we also validate the performance of the proposed method on video comfort assessment database, i.e., IEEE-SA Stereo Database. Experimental results demonstrate that the visual discomfort is greatly reduced when using the proposed method compared with the basic method. 2020-05-27T02:46:45Z 2020-05-27T02:46:45Z 2017 Journal Article Yue, G., Hou, C., Lei, J., Fang, Y., & Lin, W. (2018). Optimal region selection for stereoscopic video subtitle insertion. IEEE Transactions on Circuits and Systems for Video Technology, 28(11), 3141-3153. doi:10.1109/TCSVT.2017.2739756 1051-8215 https://hdl.handle.net/10356/140153 10.1109/TCSVT.2017.2739756 2-s2.0-85028457246 11 28 3141 3153 en IEEE Transactions on Circuits and Systems for Video Technology © 2017 IEEE. All rights reserved.
institution Nanyang Technological University
building NTU Library
country Singapore
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Stereoscopic Subtitle
Visual Discomfort
spellingShingle Engineering::Computer science and engineering
Stereoscopic Subtitle
Visual Discomfort
Yue, Guanghui
Hou, Chunping
Lei, Jianjun
Fang, Yuming
Lin, Weisi
Optimal region selection for stereoscopic video subtitle insertion
description Stereoscopic subtitle insertion is a fundamental and essential element in stereoscopic film and TV industry. However, little work has been dedicated to the optimal region selection for stereoscopic subtitle insertion. In addition, there is no public database reported for the performance evaluation of it. In this paper, we build the first large-scale video database (TJU3D) for stereoscopic video subtitle insertion, which includes 50 video sequences with rich screen scenes. Compared with 2D subtitle region selection, there are several problems we have to consider in stereoscopic subtitle region selection: 1) the subtitle should avoid depth cue collision and occlusion from objects in stereoscopic video sequences; 2) the disparity value of the subtitle must be minimized to reduce visual discomfort; and 3) the temporal coherence constraint must be considered during region selection for subtitles in video sequences. By considering these constraints, we propose an optimal region selection algorithm for stereoscopic subtitle insertion. First, we compute the disparity map of each video frame in video sequences. For each frame, the optimal position and disparity value of the subtitle are determined by a subtitle region selection algorithm, which contains two parts (i.e., the coarse selection and fine selection). After that, by considering the temporal consistency between adjacent frames, the position and disparity value of each frame are further classified and processed in order to avoid the subtitle jitter. We evaluate the proposed method on TJU3D video database through two visual discomfort prediction metrics and one subjective experiment. To further verify the effectiveness of the proposed method, we also validate the performance of the proposed method on video comfort assessment database, i.e., IEEE-SA Stereo Database. Experimental results demonstrate that the visual discomfort is greatly reduced when using the proposed method compared with the basic method.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Yue, Guanghui
Hou, Chunping
Lei, Jianjun
Fang, Yuming
Lin, Weisi
format Article
author Yue, Guanghui
Hou, Chunping
Lei, Jianjun
Fang, Yuming
Lin, Weisi
author_sort Yue, Guanghui
title Optimal region selection for stereoscopic video subtitle insertion
title_short Optimal region selection for stereoscopic video subtitle insertion
title_full Optimal region selection for stereoscopic video subtitle insertion
title_fullStr Optimal region selection for stereoscopic video subtitle insertion
title_full_unstemmed Optimal region selection for stereoscopic video subtitle insertion
title_sort optimal region selection for stereoscopic video subtitle insertion
publishDate 2020
url https://hdl.handle.net/10356/140153
_version_ 1681058632605630464