Low-rank and global-representation-key-based attention for graph transformer

• المؤلفون الرئيسيون: Kong, Lingping, Ojha, Varun, Gao, Ruobin, Suganthan, Ponnuthurai Nagaratnam, Snášel, Václav
• مؤلفون آخرون: School of Civil and Environmental Engineering
• التنسيق: مقال
• اللغة: English
• منشور في: 2023
• الموضوعات: Engineering::Civil engineering; Engineering::Computer science and engineering; Graph Transformer; Graph Representation
• الوصول للمادة أونلاين: https://hdl.handle.net/10356/170863
• المؤسسة: Nanyang Technological University

Transformer architectures have been applied to graph-specific data such as protein structure and shopper lists, and they perform accurately on graph/node classification and prediction tasks. Researchers have proved that the attention matrix in Transformers has low-rank properties, and the self-atten...