On-the-fly knowledge distillation model for sentence embedding

• 主要作者: Zhu, Xuchun
• 其他作者: Lihui Chen
• 格式: Thesis-Master by Coursework
• 語言: English
• 出版: Nanyang Technological University 2024
• 主題: Computer and Information Science; On-the-fly model; Knowledge distillation; Sentence embeddings; SimCSE; DistillCSE
• 在線閱讀: https://hdl.handle.net/10356/174236
• 機構: Nanyang Technological University

In this dissertation, we run experimental study to investigate the performance of sentence embedding using an on-the-fly knowledge distillation model based on DistillCSE framework. This model utilizes SimCSE as the initial teacher model. After a certain number of training steps, it caches an interm...