Cycle-object consistency for image-to-image domain adaptation

• Main Authors: Lin, Che-Tsung, Kew, Jie-Long, Chan, Chee Seng, Lai, Shang -Hong, Zach, Christopher
• Format: Article
• Published: Elsevier 2023
• Subjects: QA75 Electronic computers. Computer science; TK Electrical engineering. Electronics Nuclear engineering
• Online Access: http://eprints.um.edu.my/38613/
• Institution: Universiti Malaya

Recent advances in generative adversarial networks (GANs) have been proven effective in performing do-main adaptation for object detectors through data augmentation. While GANs are exceptionally success-ful, those methods that can preserve objects well in the image-to-image translation task usually require an auxiliary task, such as semantic segmentation to prevent the image content from being too distorted. However, pixel-level annotations are difficult to obtain in practice. Alternatively, instance-aware image -translation model treats object instances and background separately. Yet, it requires object detectors at test time, assuming that off-the-shelf detectors work well in both domains. In this work, we present AugGAN-Det, which introduces Cycle-object Consistency (CoCo) loss to generate instance-aware trans-lated images across complex domains. The object detector of the target domain is directly leveraged in generator training and guides the preserved objects in the translated images to carry target-domain ap-pearances. Compared to previous models, which e.g., require pixel-level semantic segmentation to force the latent distribution to be object-preserving, this work only needs bounding box annotations which are significantly easier to acquire. Next, as to the instance-aware GAN models, our model, AugGAN-Det, inter-nalizes global and object style-transfer without explicitly aligning the instance features. Most importantly, a detector is not required at test time. Experimental results demonstrate that our model outperforms re-cent object-preserving and instance-level models and achieves state-of-the-art detection accuracy and visual perceptual quality.(c) 2023 Elsevier Ltd. All rights reserved.