Low-rank and global-representation-key-based attention for graph transformer

• Main Authors: Kong, Lingping, Ojha, Varun, Gao, Ruobin, Suganthan, Ponnuthurai Nagaratnam, Snášel, Václav
• Other Authors: School of Civil and Environmental Engineering
• Format: Article
• Language: English
• Published: 2023
• Subjects: Engineering::Civil engineering; Engineering::Computer science and engineering; Graph Transformer; Graph Representation
• Online Access: https://hdl.handle.net/10356/170863
• Institution: Nanyang Technological University

Transformer architectures have been applied to graph-specific data such as protein structure and shopper lists, and they perform accurately on graph/node classification and prediction tasks. Researchers have proved that the attention matrix in Transformers has low-rank properties, and the self-atten...