Graph contrastive learning

• Main Author: Tran, Nguyen Manh Thien
• Other Authors: Lihui Chen
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Electrical and electronic engineering::Computer hardware, software and systems
• Online Access: https://hdl.handle.net/10356/167024
• Institution: Nanyang Technological University

Heterogeneous graph is a natural way to model complex relationships and interactions among entities in the real world, such as social networks or user--product relations. Learning good representations for heterogeneous graphs is a crucial step in deploying large-scale graph-based systems in an efficient and effective manner. Despite many great breakthroughs in self-supervised learning for Computer Vision and Natural Language Processing applications, similar efforts for graph data often pale in comparison, especially for heterogeneous graphs. Graph augmentation methods are limited and are weaker those for image data, limiting the potential of contrastive learning on graphs. Node dropping or edge perturbations are typically not suitable for heterogeneous graphs as they may result in invalid structures. Motivated by this, we propose to improve HeCo, the current state-of-the-art method for heterogeneous graph representation learning, with intra-view contrastive learning to obtain extra supervision signal. To maintain a graph's structural integrity, only Dropout is used to generate augmented views. To ensure the contrastive learning objective remain challenging, we further apply modified ArcFace loss to encourage more discriminative embeddings. We call our method HeCo-drop. HeCo-drop enhances HeCo consistently on various datasets, with up to +1% improvements in AUC scores. In addition, we analyse the key differences between graph and image/text data, thus outlining the challenges in adapting existing self-supervised methods to graphs.