Inception transformer

• Main Authors: SI, Chenyang, YU, Weihao, ZHOU, Pan, ZHOU, Yichen, WANG, Xinchao, YAN, Shuicheng
• 格式: text
• 語言: English
• 出版: Institutional Knowledge at Singapore Management University 2022
• 主題: Theory and Algorithms
• 在線閱讀: https://ink.library.smu.edu.sg/sis_research/9026; https://ink.library.smu.edu.sg/context/sis_research/article/10029/viewcontent/2022_NeurIPS_inception.pdf
• 機構: Singapore Management University

Recent studies show that Transformer has strong capability of building long-range dependencies, yet is incompetent in capturing high frequencies that predominantly convey local information. To tackle this issue, we present a novel and general-purpose Inception Transformer, or iFormer for short, that...