On-the-fly knowledge distillation model for sentence embedding

• المؤلف الرئيسي: Zhu, Xuchun
• مؤلفون آخرون: Lihui Chen
• التنسيق: Thesis-Master by Coursework
• اللغة: English
• منشور في: Nanyang Technological University 2024
• الموضوعات: Computer and Information Science; On-the-fly model; Knowledge distillation; Sentence embeddings; SimCSE; DistillCSE
• الوصول للمادة أونلاين: https://hdl.handle.net/10356/174236
• المؤسسة: Nanyang Technological University

In this dissertation, we run experimental study to investigate the performance of sentence embedding using an on-the-fly knowledge distillation model based on DistillCSE framework. This model utilizes SimCSE as the initial teacher model. After a certain number of training steps, it caches an interm...