Cycle-object consistency for image-to-image domain adaptation
Recent advances in generative adversarial networks (GANs) have been proven effective in performing do-main adaptation for object detectors through data augmentation. While GANs are exceptionally success-ful, those methods that can preserve objects well in the image-to-image translation task usually...
Saved in:
Main Authors: | , , , , |
---|---|
Format: | Article |
Published: |
Elsevier
2023
|
Subjects: | |
Online Access: | http://eprints.um.edu.my/38613/ |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Universiti Malaya |
id |
my.um.eprints.38613 |
---|---|
record_format |
eprints |
spelling |
my.um.eprints.386132023-11-22T23:30:15Z http://eprints.um.edu.my/38613/ Cycle-object consistency for image-to-image domain adaptation Lin, Che-Tsung Kew, Jie-Long Chan, Chee Seng Lai, Shang -Hong Zach, Christopher QA75 Electronic computers. Computer science TK Electrical engineering. Electronics Nuclear engineering Recent advances in generative adversarial networks (GANs) have been proven effective in performing do-main adaptation for object detectors through data augmentation. While GANs are exceptionally success-ful, those methods that can preserve objects well in the image-to-image translation task usually require an auxiliary task, such as semantic segmentation to prevent the image content from being too distorted. However, pixel-level annotations are difficult to obtain in practice. Alternatively, instance-aware image -translation model treats object instances and background separately. Yet, it requires object detectors at test time, assuming that off-the-shelf detectors work well in both domains. In this work, we present AugGAN-Det, which introduces Cycle-object Consistency (CoCo) loss to generate instance-aware trans-lated images across complex domains. The object detector of the target domain is directly leveraged in generator training and guides the preserved objects in the translated images to carry target-domain ap-pearances. Compared to previous models, which e.g., require pixel-level semantic segmentation to force the latent distribution to be object-preserving, this work only needs bounding box annotations which are significantly easier to acquire. Next, as to the instance-aware GAN models, our model, AugGAN-Det, inter-nalizes global and object style-transfer without explicitly aligning the instance features. Most importantly, a detector is not required at test time. Experimental results demonstrate that our model outperforms re-cent object-preserving and instance-level models and achieves state-of-the-art detection accuracy and visual perceptual quality.(c) 2023 Elsevier Ltd. All rights reserved. Elsevier 2023-06 Article PeerReviewed Lin, Che-Tsung and Kew, Jie-Long and Chan, Chee Seng and Lai, Shang -Hong and Zach, Christopher (2023) Cycle-object consistency for image-to-image domain adaptation. Pattern Recognition, 138. ISSN 0031-3203, DOI https://doi.org/10.1016/j.patcog.2023.10941 <https://doi.org/10.1016/j.patcog.2023.10941>. 10.1016/j.patcog.2023.10941 |
institution |
Universiti Malaya |
building |
UM Library |
collection |
Institutional Repository |
continent |
Asia |
country |
Malaysia |
content_provider |
Universiti Malaya |
content_source |
UM Research Repository |
url_provider |
http://eprints.um.edu.my/ |
topic |
QA75 Electronic computers. Computer science TK Electrical engineering. Electronics Nuclear engineering |
spellingShingle |
QA75 Electronic computers. Computer science TK Electrical engineering. Electronics Nuclear engineering Lin, Che-Tsung Kew, Jie-Long Chan, Chee Seng Lai, Shang -Hong Zach, Christopher Cycle-object consistency for image-to-image domain adaptation |
description |
Recent advances in generative adversarial networks (GANs) have been proven effective in performing do-main adaptation for object detectors through data augmentation. While GANs are exceptionally success-ful, those methods that can preserve objects well in the image-to-image translation task usually require an auxiliary task, such as semantic segmentation to prevent the image content from being too distorted. However, pixel-level annotations are difficult to obtain in practice. Alternatively, instance-aware image -translation model treats object instances and background separately. Yet, it requires object detectors at test time, assuming that off-the-shelf detectors work well in both domains. In this work, we present AugGAN-Det, which introduces Cycle-object Consistency (CoCo) loss to generate instance-aware trans-lated images across complex domains. The object detector of the target domain is directly leveraged in generator training and guides the preserved objects in the translated images to carry target-domain ap-pearances. Compared to previous models, which e.g., require pixel-level semantic segmentation to force the latent distribution to be object-preserving, this work only needs bounding box annotations which are significantly easier to acquire. Next, as to the instance-aware GAN models, our model, AugGAN-Det, inter-nalizes global and object style-transfer without explicitly aligning the instance features. Most importantly, a detector is not required at test time. Experimental results demonstrate that our model outperforms re-cent object-preserving and instance-level models and achieves state-of-the-art detection accuracy and visual perceptual quality.(c) 2023 Elsevier Ltd. All rights reserved. |
format |
Article |
author |
Lin, Che-Tsung Kew, Jie-Long Chan, Chee Seng Lai, Shang -Hong Zach, Christopher |
author_facet |
Lin, Che-Tsung Kew, Jie-Long Chan, Chee Seng Lai, Shang -Hong Zach, Christopher |
author_sort |
Lin, Che-Tsung |
title |
Cycle-object consistency for image-to-image domain adaptation |
title_short |
Cycle-object consistency for image-to-image domain adaptation |
title_full |
Cycle-object consistency for image-to-image domain adaptation |
title_fullStr |
Cycle-object consistency for image-to-image domain adaptation |
title_full_unstemmed |
Cycle-object consistency for image-to-image domain adaptation |
title_sort |
cycle-object consistency for image-to-image domain adaptation |
publisher |
Elsevier |
publishDate |
2023 |
url |
http://eprints.um.edu.my/38613/ |
_version_ |
1783876668929081344 |