Dense correspondence estimation, image set compression and image retargeting quality assessment

• Main Author: Zhang, Yabin
• Other Authors: Lin Weisi
• Format: Theses and Dissertations
• Language: English
• Published: 2018
• Subjects: DRNTU::Engineering::Computer science and engineering
• Online Access: http://hdl.handle.net/10356/75890
• Institution: Nanyang Technological University

Establishing dense correspondence among images is one fundamental topic in image set compression, image retargeting, computer vision and other image manipulation tasks. As image set compression for cloud-based storage begins to attract more attentions and the increasing variety of display devices has imposed the demand for image adaption to different sizes and corresponding objective quality assessment metrics, existing dense corresponding techniques such as image alignment and optical flow show limited ability to clarify the complicated image relationships. In this thesis, we first study and design specialized dense correspondence estimation techniques, and employ them to reduce inter-image redundancy for image set compression and model the retargeting modifications for image retargeting quality assessment. To reduce inter-image redundancy in image set for cloud-based storage, we develop a novel dense correspondence based prediction approach to model the consistent pixel-to-pixel relationship between images via the fast-convergent random search. While dense correspondence provides effective local similarity estimation, it may suffer from large side information expense when it cannot be parameterized effectively. By introducing consistency check constraints, we obtain the smooth and consistent dense correspondence as the inter-image prediction results, so we can efficiently reconstruct a new reference image by geometric transformations and luminance adjustments for the HEVC inter-prediction based video-like encoding. Because of the reconstruction in local units, our approach is more robust to the complex local variations compared with the global estimation based methods. To better understand the image retargeting process and perform the evaluation for different image retargeting operators, we provide a unified interpretation of image retargeting with the resampling grid generation and forward resampling. We demonstrate that the geometric change estimation is an efficient way to clarify the dense correspondence relationship among images. We formulate the geometric change estimation as a backward registration problem via Markov random field and provide an effective and practical solution. For image retargeting quality assessment applications, backward registration can provide reliable dense geometric change estimation, which links the properties between the original and retargeted images and makes the measurement of the corresponding quality degradation more efficiently. Under the guidance of the estimated dense correspondence, we develop an effective aspect ratio similarity metric by exploiting the local block changes to evaluate the visual quality of retargeted images. Furthermore, we point out that the existing low-level feature based methods are limited due to the visual importance dependency and the lack of high-level visual distortion modelling for the high-level image retargeting quality assessment task. To address these problems, we combine the dense correspondence estimation with different level feature detection methods and develop three effective general-purpose features to model various kinds of quality degradation factors in the retargeted image. Finally, we propose a multiple-level feature based framework to predict the perceptual retargeted image quality.