Low-rank and global-representation-key-based attention for graph transformer

• Main Authors: Kong, Lingping, Ojha, Varun, Gao, Ruobin, Suganthan, Ponnuthurai Nagaratnam, Snášel, Václav
• 其他作者: School of Civil and Environmental Engineering
• 格式: Article
• 語言: English
• 出版: 2023
• 主題: Engineering::Civil engineering; Engineering::Computer science and engineering; Graph Transformer; Graph Representation
• 在線閱讀: https://hdl.handle.net/10356/170863
• 機構: Nanyang Technological University

Transformer architectures have been applied to graph-specific data such as protein structure and shopper lists, and they perform accurately on graph/node classification and prediction tasks. Researchers have proved that the attention matrix in Transformers has low-rank properties, and the self-atten...