Ranking-aware contrastive learning with large language models

• Main Author: Hu, Yuqi
• Other Authors: Lihui Chen
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Ranking consistency; Contrastive learning; Ranking distillation
• Online Access: https://hdl.handle.net/10356/177983
• Institution: Nanyang Technological University

Generating high-quality word and sentence representations is a foundational task in natural language processing (NLP). In recent years, various embedding methodologies have been proposed, notably those leveraging the capabilities of large language models for in-context learning. Research has shown that language model performance can be enhanced by integrating a query with multiple examples. Inspired by this research, this project explores the use of a contrastive learning framework combined with ranking knowledge to enhance the generation and retrieval of sentence embeddings, aiming to more accurately identify the most similar sentences in in-context learning scenarios. Subsequent experiments tested various ranking strategies within the contrastive learning framework, yielding novel insights and conclusions.