MetaFormer baselines for vision

• Main Authors: YU, Weihao, SI, Chenyang, ZHOU, Pan, LUO, Mi, ZHOU, Yichen, FENG, Jiashi, YAN, Shuicheng, WANG, Xinchao
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: MetaFormer; Transformer; Neural Networks; Image Classification; Deep Learning; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/9054; https://ink.library.smu.edu.sg/context/sis_research/article/10057/viewcontent/2021_TPAMI_MetaFormer.pdf
• Institution: Singapore Management University

Abstract—MetaFormer, the abstracted architecture of Transformer, has been found to play a significant role in achieving competitive performance. In this paper, we further explore the capacity of MetaFormer, again, by migrating our focus away from the token mixer design: we introduce several baseline...