Delving into important samples of semi-supervised old photo restoration: A new dataset and method

The degradation of printed photographs due to inadequate preservation is a major problem that can be addressed through deep learning-based restoration methods. However, these methods are often limited by their reliance on annotated data, making them less effective for new domains with limited traini...

Full description

Saved in:
Bibliographic Details
Main Authors: CAI, Werwei, ZHANG, Huaidong, XU, Xuemiao, XU, Chenshu, ZHANG, Kun, HE, Shengfeng
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2024
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/8820
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-9823
record_format dspace
spelling sg-smu-ink.sis_research-98232024-05-30T07:06:03Z Delving into important samples of semi-supervised old photo restoration: A new dataset and method CAI, Werwei ZHANG, Huaidong XU, Xuemiao XU, Chenshu ZHANG, Kun HE, Shengfeng The degradation of printed photographs due to inadequate preservation is a major problem that can be addressed through deep learning-based restoration methods. However, these methods are often limited by their reliance on annotated data, making them less effective for new domains with limited training samples. In this paper, we propose a semi-supervised old photo restoration network that employs a continuous important sample mining strategy. Specifically, we explore the learning potential of limited data from three aspects: correcting imbalanced data distribution, assigning significant pseudo labels, and learning from unlabeled data. First, we coordinate a random mask augmented strategy with the Double-consistency Alignment method to address the unbalanced damaged category (scratched damage is more prevalent than other artifact types). Second, we develop a novel Perceptual-aware Pseudo-label Propagation method that selects initial recovered results as reliable pseudo-labels to continuously expand the sample pool. Lastly, we propose a Damage-augmented Contrastive Learning method that constructs positive, anchor, and negative samples within a semi-supervised framework to mine correlations of unlabeled data more effectively. To evaluate our approach, we introduce the Old Photo Detection Dataset () and the Old Photo Restoration Dataset (), both of which consist of 563 (6,179 augmented) photo pairs recovered by professional artists. Our extensive experiments show that our approach significantly outperforms existing methods. Furthermore, we demonstrate the effectiveness of our approach by training an external old photographic plate restoration network using the deuterogenic old photographic film dataset and obtaining promising results. 2024-01-01T08:00:00Z text https://ink.library.smu.edu.sg/sis_research/8820 info:doi/10.1109/TMM.2024.3400695 Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University contrastive learning Correlation Image restoration Old photo restoration Reliability semi-supervised learning Semisupervised learning Task analysis Training data Graphics and Human Computer Interfaces Software Engineering
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic contrastive learning
Correlation
Image restoration
Old photo restoration
Reliability
semi-supervised learning
Semisupervised learning
Task analysis
Training data
Graphics and Human Computer Interfaces
Software Engineering
spellingShingle contrastive learning
Correlation
Image restoration
Old photo restoration
Reliability
semi-supervised learning
Semisupervised learning
Task analysis
Training data
Graphics and Human Computer Interfaces
Software Engineering
CAI, Werwei
ZHANG, Huaidong
XU, Xuemiao
XU, Chenshu
ZHANG, Kun
HE, Shengfeng
Delving into important samples of semi-supervised old photo restoration: A new dataset and method
description The degradation of printed photographs due to inadequate preservation is a major problem that can be addressed through deep learning-based restoration methods. However, these methods are often limited by their reliance on annotated data, making them less effective for new domains with limited training samples. In this paper, we propose a semi-supervised old photo restoration network that employs a continuous important sample mining strategy. Specifically, we explore the learning potential of limited data from three aspects: correcting imbalanced data distribution, assigning significant pseudo labels, and learning from unlabeled data. First, we coordinate a random mask augmented strategy with the Double-consistency Alignment method to address the unbalanced damaged category (scratched damage is more prevalent than other artifact types). Second, we develop a novel Perceptual-aware Pseudo-label Propagation method that selects initial recovered results as reliable pseudo-labels to continuously expand the sample pool. Lastly, we propose a Damage-augmented Contrastive Learning method that constructs positive, anchor, and negative samples within a semi-supervised framework to mine correlations of unlabeled data more effectively. To evaluate our approach, we introduce the Old Photo Detection Dataset () and the Old Photo Restoration Dataset (), both of which consist of 563 (6,179 augmented) photo pairs recovered by professional artists. Our extensive experiments show that our approach significantly outperforms existing methods. Furthermore, we demonstrate the effectiveness of our approach by training an external old photographic plate restoration network using the deuterogenic old photographic film dataset and obtaining promising results.
format text
author CAI, Werwei
ZHANG, Huaidong
XU, Xuemiao
XU, Chenshu
ZHANG, Kun
HE, Shengfeng
author_facet CAI, Werwei
ZHANG, Huaidong
XU, Xuemiao
XU, Chenshu
ZHANG, Kun
HE, Shengfeng
author_sort CAI, Werwei
title Delving into important samples of semi-supervised old photo restoration: A new dataset and method
title_short Delving into important samples of semi-supervised old photo restoration: A new dataset and method
title_full Delving into important samples of semi-supervised old photo restoration: A new dataset and method
title_fullStr Delving into important samples of semi-supervised old photo restoration: A new dataset and method
title_full_unstemmed Delving into important samples of semi-supervised old photo restoration: A new dataset and method
title_sort delving into important samples of semi-supervised old photo restoration: a new dataset and method
publisher Institutional Knowledge at Singapore Management University
publishDate 2024
url https://ink.library.smu.edu.sg/sis_research/8820
_version_ 1814047566141063168