Learning to recognize objects by adaptive knowledge transfer

• Main Author: Tao, Qingyi
• Other Authors: Chen Change Loy
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision; Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/151507
• Institution: Nanyang Technological University

When humans learn new knowledge and skills, we can naturally transfer them to other domains. Along with the learning procedures, we learn knowledge and skills for certain tasks and transfer them to similar tasks; we also can use the old knowledge to facilitate the learning of new knowledge. While effective knowledge transfer is a congenital and important learning ability of humans, it is not easy for machine learning mechanisms to adopt the ability of knowledge transfer. In recent years, there are plenty of works studying transfer learning in deep learning. There are still some practical challenges that remain undiscovered, especially under different problem settings encountered in real situations. In this thesis, we explore how to adopt knowledge transfer mechanisms in deep learning approaches in several practical scenarios. Four different works are proposed to study knowledge transfer across different domains and tasks via domain adaptation and model transfer. In particular, we study the web knowledge transfer for object detection task by adapting the web data to the real target dataset, which aims at reducing the human annotation effort for training object detector. In the incremental learning scenario, we study the cross-utilization of the old and new knowledge to overcome the catastrophic forgetting during the incremental and progressive learning process. Lastly, we explore transfer learning in the medical imaging domain by transferring the model pre-trained on normal images. Overall, the major contributions are summarized as follows: - A web knowledge transfer method to enhance the learning of weakly supervised object detection. The proposed method includes an effective web data collection pipeline and a curriculum learning scheme to achieve more effective model optimization during multi-instance learning. - An annotation-effective object detection method by adapting web data to the target data for object detection. This work attempts to learn an object detector from web supervision by adversarial domain adaptation. - An incremental learning scheme that adapts an old model to a new model without forgetting the old knowledge. A systematic study is performed to explore different class incremental methods. Furthermore, we propose a graph-based method to mine the old sample forgettability along with the training of the new tasks, and dynamically select samples that are more forgettable to overcome the catastrophic forgetting. - A lesion detection method for 3D CT images by utilizing model weights that are pre-trained on normal 2D RGB images. Furthermore, an attention-based feature aggregation method is proposed to adaptively transfer the information from neighboring slices to the key slices for more discriminative representation. Through this thesis, we demonstrate three different paradigms of knowledge transfer, including (1) the cross-domain knowledge transfer for adapting web data to an application with real unconstrained data, (2) the continual knowledge transfer from old tasks to new tasks without forgetting the old knowledge, and (3) the model transfer from the normal image domain to the medical domain. Under several practical tasks, the experiments are conducted to demonstrate the effectiveness of the proposed knowledge transfer approaches.