Easy-but-effective domain sub-similarity learning for transfer regression

• Main Authors: Wei, Pengfei, Sagarna, Ramon, Ke, Yiping, Ong, Yew-Soon
• Other Authors: School of Computer Science and Engineering
• Format: Article
• Language: English
• Published: 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence; Transfer Regression; Transfer Covariance Function; Gaussian Process
• Online Access: https://hdl.handle.net/10356/147569
• Institution: Nanyang Technological University

Transfer covariance function, which can model domain similarity and adaptively control the knowledge transfer across domains, is widely used in transfer learning. In this paper, we concentrate on Gaussian process (GP) models using a transfer covariance function for regression problems in a black-box learning scenario. Precisely, we investigate a family of rather general transfer covariance functions, T*, that can model the heterogeneous sub-similarities of domains through multiple kernel learning. A necessary and sufficient condition to obtain valid GPs using T* (GPT* ) for any data is given. This condition becomes specially handy for practical applications as (i) it enables semantic interpretations of the sub-similarities and (ii) it can readily be used for model learning. In particular, we propose a computationally inexpensive model learning rule that can explicitly capture different sub-similarities of domains. We propose two instantiations of GPT* , one with a set of predefined constant base kernels and one with a set of learnable parametric base kernels. Extensive experiments on 36 synthetic transfer tasks and 12 real-world transfer tasks demonstrate the effectiveness of GPT* on the sub-similarity capture and the transfer performance.