Delving deep into pixelized face recovery and defense

• Main Authors: ZHONG, Zhixuan, DU, Yong, ZHOU, Yang, CAO, Jiangzhong, HE, Shengfeng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Contradictory problems; Data driven; Deterministics; Face depixelization; Face pixelization; Feature representation; Image recovery; Inpainting; Pixelization; Privacy preservation; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/8372; https://ink.library.smu.edu.sg/context/sis_research/article/9375/viewcontent/Delving_deep_into_pixelized_face_recovery_and_defense.pdf
• Institution: Singapore Management University

Pixelization is arguably one of the most well-adopted deterministic obfuscation techniques for privacy preservation purposes. Although the recovery of pixelized faces is underexplored, the powerful deep neural networks might combat this problem in a data-driven manner. As a consequence, an unbreakable pixelization approach is desired. To achieve this goal, in this paper, we delve into two contradictory problems of unrecoverable pixelization and its counterpart, depixelization, by leveraging the best recovery to strengthen the robustness of the unrecoverable pixelized patterns. In particular, on the offensive end of recovery, we combat the large and continuous nature of pixelized regions by proposing two strategies, 1) an iterative depixelization network that progressively decomposes and predicts the pixelized regions and thus outer results are used to support inner inferences; 2) a dynamic dilated convolution operation is proposed to stride over the redundant identical pixels from the same pixelized region, enabling the network to adaptively extract valid feature representations. We show that our tailored depixelization method significantly outperforms several baselines or inpainting approaches by over 1.0 FID and 2% ID-SIM improvements on CelebA dataset which includes 182,732 human face images, and therefore we study how to defend this advanced recovery and produce unrecoverable pixelized patterns. To balance the visual perception and robustness of pixelization, we propose to generate two types of adversarial examples, pixel-wise and block-wise perturbations, which make different trade-offs between quality and robustness. By deploying our depixelization network in a semi-whitebox setting, our pixelization method can generate imperceptible perturbations while being robust to depixelization.