S-prompts learning with pre-trained transformers: An Occam's razor for domain incremental learning

• Main Authors: WANG, Yabin, HUANG, Zhiwu, HONG, Xiaopeng.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Prompts Learning; Pre-trained Transformers; Occam's Razor; Domain Incremental Learning; Artificial Intelligence and Robotics; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/7614; https://ink.library.smu.edu.sg/context/sis_research/article/8617/viewcontent/03_Sprompts_NeurIPS2022.pdf
• Institution: Singapore Management University

State-of-the-art deep neural networks are still struggling to address the catastrophic forgetting problem in continual learning. In this paper, we propose one simple paradigm (named as S-Prompting) and two concrete approaches to highly reduce the forgetting degree in one of the most typical continua...