Perceptual graphic rendering and quality evaluation

• Main Author: Dong, Lu
• Other Authors: Lin Weisi
• Format: Theses and Dissertations
• Language: English
• Published: 2016
• Subjects: DRNTU::Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/65917
• Institution: Nanyang Technological University

Computer graphics are widely used in various areas nowadays, such as entertainment, industry design, education, architecture and scientific visualization. Graphic rendering is the process of displaying 3D models visually as 2D images. An important issue in graphic rendering is to improve the rendering efficiency. Since the ultimate evaluator of the realistically rendered images is the human visual system (HVS), it is meaningful to substantially decrease computational, storage and transmission cost by exploiting perceptual properties of the HVS without degradation of perceived rendering quality. To design a perceptual rendering system, we need to consider two processes: one is how to build computational models of the characteristics of the HVS; the other is how to incorporate these models into the rendering process. In this work, we explore both the two processes for three schemes in graphic rendering. Firstly, we propose a new visual saliency based perceptual rendering scheme. In general, a scene has one or more salient objects, and the visual attention of observers is attracted by the salient objects. This property of the HVS allows us to decrease computational resources in non-salient objects without degrading the perceived rendering quality. Different from existing work in saliency-based perceptual rendering which use visual saliency as the only guide for computation allocation, we integrate rendering complexity with visual saliency to drive rendering. The rendering complexity of each pixel is estimated with a sample variance metric, and the computational resources are iteratively distributed among pixels according to visual saliency and rendering complexity. Secondly, we build an enhanced visual saliency detection method for rendered graphics by making use of 3D model information. The proposed method computes the object-level contrast between each object and the remaining objects in a rendered image, and detects salient objects based on the graphic contrast. The proposed method is able to find the accurate boundaries of salient objects and improve the accuracy of saliency detection. We also incorporate this enhanced saliency detection method in our saliency-based perceptual rendering scheme, to generate consistent rendering quality in salient objects. Thirdly, we explore a property of the HVS that has not been exploited by existing rendering applications, i.e. low sensitivity to irregular structure. Some 3D scenes include a large amount of irregular textural information, and the noise masking ability of irregular structure allows us to decrease the computational resources in irregular structure, without introducing visible artifacts into the rendered image. We propose an irregularity measure which relies on the similarity between each pixel and the neighboring pixels. The irregularity measure is applied in a perceptual rendering scheme to improve the rendering efficiency of scenes with a large amount of irregular texture. In addition, we also explore perceptual quality evaluation of 3D models. Perceptual quality evaluation of 3D models is critical for design and optimization of related graphic algorithms, such as model simplification, compression and perceptual rendering. 3D models are widely represented by 3D triangle meshes, due to their simplicity. We design a novel objective quality evaluation method to assess the quality of distorted 3D meshes with respect to a reference one. Curvature difference between each pair of corresponding vertices is calculated, and then modulated by two new components, namely the visual masking module and the saturation effect module. Structural distortion between 3D meshes is also calculated and pooled with the said curvature difference into a quality score. The proposed quality evaluation method yields consistent results in three publicly available databases.