AUTOMATIC SPATIAL CONTROL BASED ON SEMANTIC IMAGE SEGMENTATION RESULT IN NEURAL STYLE TRANSFER

• Main Author: William
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/39602
• Institution: Institut Teknologi Bandung

Painting an object with style from another painting isn’t an easy task. One of artificial intelligence research topic in art field is neural style transfer (Gatys et al, 2016). Neural style transfer is a process to make a new image with style from another image. Results in Gatys et al. (2016) still have weakness. There are regions in content image that stylized from semantically different area of style image. Gatys et al. (2017) published a method to overcome this problem with spatial control. The goal is to control which region of the style images is used to stylize each region in the content image. This method uses masks to separate the object regions in the image. However, the mask is generated manually by user input. This thesis propose a method to do spatial control in neural style transfer using automatically generated masks. The masks is generated with DeepLab (Chen et al., 2016), state-of-the-art semantic image segmentation model. We also compare neural style transfer result with three types of pre-condition: (1) without segmentation mask, (2) using hard segmentation mask, and (3) using soft segmentation mask. There are three main process in the proposed system, i.e. generating segmentation mask, segmentation mask label adjustment, and style transfer process. As evaluation, we compare the results of neural style transfer with three different pre-condition. Based on the evaluation result, image stylized with segmentation mask successfully overcome the previous problem (stylization of region in content image using semantically incompatible region on style image). This weakness occur in result images without using segmentation mask. However, there are side effects in usage of segmentation mask. In many results, the usage of segmentation mask reduces the aesthectic aspect of image. Results with soft segmentation mask don’t have a significant diiference compared to result with hard segmentation mask. Some generated image with soft segmentation mask give poor results, there are styles from different region on style image applied on the same object on content image.