Efficient inference offloading for mixture-of-experts large language models in internet of medical things

• Main Authors: Yuan, Xiaoming, Kong, Weixuan, Luo, Zhenyu, Xu, Minrui
• Other Authors: School of Computer Science and Engineering
• Format: Article
• Language: English
• Published: 2024
• Subjects: Computer and Information Science; Large language models; Efficient inference offloading
• Online Access: https://hdl.handle.net/10356/179743
• Institution: Nanyang Technological University

Despite recent significant advancements in large language models (LLMs) for medical services, the deployment difficulties of LLMs in e-healthcare hinder complex medical applications in the Internet of Medical Things (IoMT). People are increasingly concerned about e-healthcare risks and privacy prote...