Model-based referenceless quality metric of 3D synthesized images using local image description

• Main Authors: Gu, Ke, Jakhetiya, Vinit, Qiao, Jun-Fei, Li, Xiaoli, Lin, Weisi, Thalmann, Daniel
• Other Authors: School of Computer Science and Engineering
• Format: Article
• Language: English
• Published: 2020
• Subjects: Engineering::Computer science and engineering; Quality Assessment; No-reference
• Online Access: https://hdl.handle.net/10356/142320
• Institution: Nanyang Technological University

New challenges have been brought out along with the emerging of 3D-related technologies, such as virtual reality, augmented reality (AR), and mixed reality. Free viewpoint video (FVV), due to its applications in remote surveillance, remote education, and so on, based on the flexible selection of direction and viewpoint, has been perceived as the development direction of next-generation video technologies and has drawn a wide range of researchers' attention. Since FVV images are synthesized via a depth image-based rendering (DIBR) procedure in the "blind" environment (without reference images), a reliable real-time blind quality evaluation and monitoring system is urgently required. But existing assessment metrics do not render human judgments faithfully mainly because geometric distortions are generated by DIBR. To this end, this paper proposes a novel referenceless quality metric of DIBR-synthesized images using the autoregression (AR)-based local image description. It was found that, after the AR prediction, the reconstructed error between a DIBR-synthesized image and its AR-predicted image can accurately capture the geometry distortion. The visual saliency is then leveraged to modify the proposed blind quality metric to a sizable margin. Experiments validate the superiority of our no-reference quality method as compared with prevailing full-, reduced-, and no-reference models.