KNOWLEDGE DISTILLATION AND SIAMESE NETWORK ADOPTION FOR SEMANTIC SEGMENTATION USING SEMI- SUPERVISED LEARNING
The demand for large amounts of labeled data and large computations is a common problem in semantic segmentation. Semi-supervised answers the problem by utilizing data without labels in the training process, but choosing the right method in the unsupervised learning process is a challenge in itself....
Saved in:
| Main Author: | |
|---|---|
| Format: | Theses |
| Language: | Indonesia |
| Online Access: | https://digilib.itb.ac.id/gdl/view/69104 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Institution: | Institut Teknologi Bandung |
| Language: | Indonesia |
| Summary: | The demand for large amounts of labeled data and large computations is a common problem in semantic segmentation. Semi-supervised answers the problem by utilizing data without labels in the training process, but choosing the right method in the unsupervised learning process is a challenge in itself. Self-supervised has a variety of useful pretexts for building an encoder, but only a few of which can assist in semantic segmentation. A research conducted by (X. Chen dan He, 2020) shows that siamese network as contrastive learning can be achieved without using negative pairs. In this study, we propose SimFCN, a semi-supervised learning model based on a siamese network for the semantic segmentation domain. By combining FCN as the main decoder with a siamese network projection layer, this method is able to construct parameters for models with limited labeled data in the semantic segmentation domain using low computational models such as ResNet-18d. SimFCN in the PASCAL VOC dataset obtained mIoU values of 30.9% by only using ~0.5% of total dataset (60 labeled images) and 51.3% for ~10% of total dataset (1000 labeled images). SimFCN is superior to the Cross-Consistency Training model with fewer parameters and lighter computations. From the evaluation conducted, SimFCN is better in building an encoder compared to Cross-Consistency-Training. In this study, we also propose KD-Siamese, which is adopt knowledge distillation in the supervised section and use a teacher encoder to build triplet-loss on a siamese network. KD-Siamese achieved 10.4% mIoU for validation with 100 labeled data (~1%). From this study we found that knowledge distillation requires long iterations to reach desired performance and a complex teacher model does not necessarily build a good student model. |
|---|