Wave-ViT: Unifying wavelet and transformers for visual representation learning

• Main Authors: YAO, Ting, PAN, Yingwei, LI, Yehao, NGO, Chong-wah, MEI, Tao
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Vision transformer; Wavelet transform; Self-attention; learning; Image recognition; Artificial Intelligence and Robotics; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/7508; https://ink.library.smu.edu.sg/context/sis_research/article/8511/viewcontent/2207.04978.pdf
• Institution: Singapore Management University

Multi-scale Vision Transformer (ViT) has emerged as a powerful backbone for computer vision tasks, while the self-attention computation in Transformer scales quadratically w.r.t. the input patch number. Thus, existing solutions commonly employ down-sampling operations (e.g., average pooling) over ke...