KNOWLEDGE DISTILLATION AND SIAMESE NETWORK ADOPTION FOR SEMANTIC SEGMENTATION USING SEMI- SUPERVISED LEARNING
The demand for large amounts of labeled data and large computations is a common problem in semantic segmentation. Semi-supervised answers the problem by utilizing data without labels in the training process, but choosing the right method in the unsupervised learning process is a challenge in itself....
Saved in:
Main Author: | |
---|---|
Format: | Theses |
Language: | Indonesia |
Online Access: | https://digilib.itb.ac.id/gdl/view/69104 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Institut Teknologi Bandung |
Language: | Indonesia |
id |
id-itb.:69104 |
---|---|
spelling |
id-itb.:691042022-09-20T11:47:14ZKNOWLEDGE DISTILLATION AND SIAMESE NETWORK ADOPTION FOR SEMANTIC SEGMENTATION USING SEMI- SUPERVISED LEARNING Abdurrohman, Harits Indonesia Theses semi-supervised learning, self-supervised, semantic segmentation, siamese network, knowledge distillation INSTITUT TEKNOLOGI BANDUNG https://digilib.itb.ac.id/gdl/view/69104 The demand for large amounts of labeled data and large computations is a common problem in semantic segmentation. Semi-supervised answers the problem by utilizing data without labels in the training process, but choosing the right method in the unsupervised learning process is a challenge in itself. Self-supervised has a variety of useful pretexts for building an encoder, but only a few of which can assist in semantic segmentation. A research conducted by (X. Chen dan He, 2020) shows that siamese network as contrastive learning can be achieved without using negative pairs. In this study, we propose SimFCN, a semi-supervised learning model based on a siamese network for the semantic segmentation domain. By combining FCN as the main decoder with a siamese network projection layer, this method is able to construct parameters for models with limited labeled data in the semantic segmentation domain using low computational models such as ResNet-18d. SimFCN in the PASCAL VOC dataset obtained mIoU values of 30.9% by only using ~0.5% of total dataset (60 labeled images) and 51.3% for ~10% of total dataset (1000 labeled images). SimFCN is superior to the Cross-Consistency Training model with fewer parameters and lighter computations. From the evaluation conducted, SimFCN is better in building an encoder compared to Cross-Consistency-Training. In this study, we also propose KD-Siamese, which is adopt knowledge distillation in the supervised section and use a teacher encoder to build triplet-loss on a siamese network. KD-Siamese achieved 10.4% mIoU for validation with 100 labeled data (~1%). From this study we found that knowledge distillation requires long iterations to reach desired performance and a complex teacher model does not necessarily build a good student model. text |
institution |
Institut Teknologi Bandung |
building |
Institut Teknologi Bandung Library |
continent |
Asia |
country |
Indonesia Indonesia |
content_provider |
Institut Teknologi Bandung |
collection |
Digital ITB |
language |
Indonesia |
description |
The demand for large amounts of labeled data and large computations is a common problem in semantic segmentation. Semi-supervised answers the problem by utilizing data without labels in the training process, but choosing the right method in the unsupervised learning process is a challenge in itself. Self-supervised has a variety of useful pretexts for building an encoder, but only a few of which can assist in semantic segmentation. A research conducted by (X. Chen dan He, 2020) shows that siamese network as contrastive learning can be achieved without using negative pairs. In this study, we propose SimFCN, a semi-supervised learning model based on a siamese network for the semantic segmentation domain. By combining FCN as the main decoder with a siamese network projection layer, this method is able to construct parameters for models with limited labeled data in the semantic segmentation domain using low computational models such as ResNet-18d. SimFCN in the PASCAL VOC dataset obtained mIoU values of 30.9% by only using ~0.5% of total dataset (60 labeled images) and 51.3% for ~10% of total dataset (1000 labeled images). SimFCN is superior to the Cross-Consistency Training model with fewer parameters and lighter computations. From the evaluation conducted, SimFCN is better in building an encoder compared to Cross-Consistency-Training. In this study, we also propose KD-Siamese, which is adopt knowledge distillation in the supervised section and use a teacher encoder to build triplet-loss on a siamese network. KD-Siamese achieved 10.4% mIoU for validation with 100 labeled data (~1%). From this study we found that knowledge distillation requires long iterations to reach desired performance and a complex teacher model does not necessarily build a good student model. |
format |
Theses |
author |
Abdurrohman, Harits |
spellingShingle |
Abdurrohman, Harits KNOWLEDGE DISTILLATION AND SIAMESE NETWORK ADOPTION FOR SEMANTIC SEGMENTATION USING SEMI- SUPERVISED LEARNING |
author_facet |
Abdurrohman, Harits |
author_sort |
Abdurrohman, Harits |
title |
KNOWLEDGE DISTILLATION AND SIAMESE NETWORK ADOPTION FOR SEMANTIC SEGMENTATION USING SEMI- SUPERVISED LEARNING |
title_short |
KNOWLEDGE DISTILLATION AND SIAMESE NETWORK ADOPTION FOR SEMANTIC SEGMENTATION USING SEMI- SUPERVISED LEARNING |
title_full |
KNOWLEDGE DISTILLATION AND SIAMESE NETWORK ADOPTION FOR SEMANTIC SEGMENTATION USING SEMI- SUPERVISED LEARNING |
title_fullStr |
KNOWLEDGE DISTILLATION AND SIAMESE NETWORK ADOPTION FOR SEMANTIC SEGMENTATION USING SEMI- SUPERVISED LEARNING |
title_full_unstemmed |
KNOWLEDGE DISTILLATION AND SIAMESE NETWORK ADOPTION FOR SEMANTIC SEGMENTATION USING SEMI- SUPERVISED LEARNING |
title_sort |
knowledge distillation and siamese network adoption for semantic segmentation using semi- supervised learning |
url |
https://digilib.itb.ac.id/gdl/view/69104 |
_version_ |
1822005943087398912 |