Inception transformer

• Main Authors: SI, Chenyang, YU, Weihao, ZHOU, Pan, ZHOU, Yichen, WANG, Xinchao, YAN, Shuicheng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/9026; https://ink.library.smu.edu.sg/context/sis_research/article/10029/viewcontent/2022_NeurIPS_inception.pdf
• Institution: Singapore Management University

Recent studies show that Transformer has strong capability of building long-range dependencies, yet is incompetent in capturing high frequencies that predominantly convey local information. To tackle this issue, we present a novel and general-purpose Inception Transformer, or iFormer for short, that...