Inference acceleration of large language models

• المؤلف الرئيسي: Zhang, Boyu
• مؤلفون آخرون: Mao Kezhi
• التنسيق: Thesis-Master by Coursework
• اللغة: English
• منشور في: Nanyang Technological University 2024
• الموضوعات: Computer and Information Science; Large language model; Quantization; Approximate computation; Self-attention; Transformer
• الوصول للمادة أونلاين: https://hdl.handle.net/10356/181660

This dissertation delves into the challenges and bottlenecks faced by current large language models during inference from three core perspectives: data, model, and system. Through meticulous research, key factors impacting inference speed are identified, encompassing data processing efficiency, model structure complexity, and system resource allocation and utilization. Building on this foundation, I review and interpret previous research in this field, systematically summarizing their core ideas, implementation pathways, and achievements. By deeply analyzing these studies, it not only highlight their respective strengths and weaknesses but also propose targeted improvement suggestions in line with current technological trends.