Performance analysis of Llama 2 among other LLMs

• Main Authors: HUANG, Donghao, HU, Zhenda, WANG, Zhaoxia
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: large language model; in-context learning; generative pre-trained transformer; model evaluation; Artificial Intelligence and Robotics; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/9162; https://ink.library.smu.edu.sg/context/sis_research/article/10165/viewcontent/6._IEEE_CAI2024_paper_ID_371_Performance_Analysis_of_Llama_2_Among__Other_LLMs.pdf
• Institution: Singapore Management University

Llama 2, an open-source large language model developed by Meta, offers a versatile and high-performance solution for natural language processing, boasting a broad scale, competitive dialogue capabilities, and open accessibility for research and development, thus driving innovation in AI applications. Despite these advancements, there remains a limited understanding of the underlying principles and performance of Llama 2 compared with other LLMs. To address this gap, this paper presents a comprehensive evaluation of Llama 2, focusing on its application in in-context learning — an AI design pattern that harnesses pre-trained LLMs for processing confidential and sensitive data. Through a rigorous comparative analysis with other open-source LLMs and OpenAI models, this study sheds light on Llama 2’s performance, quality, and potential use cases. Our findings indicate that Llama 2 holds significant promise for applications involving in-context learning, with notable strengths in both answer quality and inference speed. This research offers valuable insights for the fields of LLMs and serves as an effectivereference for companies and individuals utilizing such large models. The source codes and datasets of this paper are accessible at https://github.com/inflaton/Llama-2-eval.