Faithful extreme rescaling via generative prior reciprocated invertible representations

This paper presents a Generative prior ReciprocAted Invertible rescaling Network (GRAIN) for generating faithful high-resolution (HR) images from low-resolution (LR) invertible images with an extreme upscaling factor (64×). Previous researches have leveraged the prior knowledge of a pretrained GAN m...

Full description

Saved in:
Bibliographic Details
Main Authors: ZHONG, Zhixuan, CHAI, Liangyu, ZHOU, Yang, DENG, Bailin, PAN, Jia, HE, Shengfeng
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2022
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/8444
https://ink.library.smu.edu.sg/context/sis_research/article/9447/viewcontent/Zhong_Faithful_Extreme_Rescaling_via_Generative_Prior_Reciprocated_Invertible_Representations_CVPR_2022_paper.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-9447
record_format dspace
spelling sg-smu-ink.sis_research-94472024-01-04T09:54:34Z Faithful extreme rescaling via generative prior reciprocated invertible representations ZHONG, Zhixuan CHAI, Liangyu ZHOU, Yang DENG, Bailin PAN, Jia HE, Shengfeng This paper presents a Generative prior ReciprocAted Invertible rescaling Network (GRAIN) for generating faithful high-resolution (HR) images from low-resolution (LR) invertible images with an extreme upscaling factor (64×). Previous researches have leveraged the prior knowledge of a pretrained GAN model to generate high-quality upscaling results. However, they fail to produce pixel-accurate results due to the highly ambiguous extreme mapping process. We remedy this problem by introducing a reciprocated invertible image rescaling process, in which high-resolution information can be delicately embedded into an invertible low-resolution image and generative prior for a faithful HR reconstruction. In particular, the invertible LR features not only carry significant HR semantics, but also are trained to predict scale-specific latent codes, yielding a preferable utilization of generative features. On the other hand, the enhanced generative prior is re-injected to the rescaling process, compensating the lost details of the invertible rescaling. Our reciprocal mechanism perfectly integrates the advantages of invertible encoding and generative prior, leading to the first feasible extreme rescaling solution. Extensive experiments demonstrate superior performance against state-of-the-art upscaling methods. Code is available at https://github.com/cszzx/GRAIN. 2022-06-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/8444 info:doi/10.1109/CVPR52688.2022.00562 https://ink.library.smu.edu.sg/context/sis_research/article/9447/viewcontent/Zhong_Faithful_Extreme_Rescaling_via_Generative_Prior_Reciprocated_Invertible_Representations_CVPR_2022_paper.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Face and gestures Image and video synthesis and generation Low-level vision Databases and Information Systems Graphics and Human Computer Interfaces
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Face and gestures
Image and video synthesis and generation
Low-level vision
Databases and Information Systems
Graphics and Human Computer Interfaces
spellingShingle Face and gestures
Image and video synthesis and generation
Low-level vision
Databases and Information Systems
Graphics and Human Computer Interfaces
ZHONG, Zhixuan
CHAI, Liangyu
ZHOU, Yang
DENG, Bailin
PAN, Jia
HE, Shengfeng
Faithful extreme rescaling via generative prior reciprocated invertible representations
description This paper presents a Generative prior ReciprocAted Invertible rescaling Network (GRAIN) for generating faithful high-resolution (HR) images from low-resolution (LR) invertible images with an extreme upscaling factor (64×). Previous researches have leveraged the prior knowledge of a pretrained GAN model to generate high-quality upscaling results. However, they fail to produce pixel-accurate results due to the highly ambiguous extreme mapping process. We remedy this problem by introducing a reciprocated invertible image rescaling process, in which high-resolution information can be delicately embedded into an invertible low-resolution image and generative prior for a faithful HR reconstruction. In particular, the invertible LR features not only carry significant HR semantics, but also are trained to predict scale-specific latent codes, yielding a preferable utilization of generative features. On the other hand, the enhanced generative prior is re-injected to the rescaling process, compensating the lost details of the invertible rescaling. Our reciprocal mechanism perfectly integrates the advantages of invertible encoding and generative prior, leading to the first feasible extreme rescaling solution. Extensive experiments demonstrate superior performance against state-of-the-art upscaling methods. Code is available at https://github.com/cszzx/GRAIN.
format text
author ZHONG, Zhixuan
CHAI, Liangyu
ZHOU, Yang
DENG, Bailin
PAN, Jia
HE, Shengfeng
author_facet ZHONG, Zhixuan
CHAI, Liangyu
ZHOU, Yang
DENG, Bailin
PAN, Jia
HE, Shengfeng
author_sort ZHONG, Zhixuan
title Faithful extreme rescaling via generative prior reciprocated invertible representations
title_short Faithful extreme rescaling via generative prior reciprocated invertible representations
title_full Faithful extreme rescaling via generative prior reciprocated invertible representations
title_fullStr Faithful extreme rescaling via generative prior reciprocated invertible representations
title_full_unstemmed Faithful extreme rescaling via generative prior reciprocated invertible representations
title_sort faithful extreme rescaling via generative prior reciprocated invertible representations
publisher Institutional Knowledge at Singapore Management University
publishDate 2022
url https://ink.library.smu.edu.sg/sis_research/8444
https://ink.library.smu.edu.sg/context/sis_research/article/9447/viewcontent/Zhong_Faithful_Extreme_Rescaling_via_Generative_Prior_Reciprocated_Invertible_Representations_CVPR_2022_paper.pdf
_version_ 1787590750579458048