Unified generative and discriminative training for multi-modal Large Language Models

• Main Authors: CHOW, Wei, LI, Juncheng, PAN, Kaihang, YU, Qifan, FEI, Hao, GE, Zhiqi, YANG, Shuai, TENG, Siliang, ZHANG, Hanwang, Qianru SUN
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Machine learning; Generative training; Multimodal Large Language Models; Semantics extraction; Artificial Intelligence and Robotics; Computer Sciences
• Online Access: https://ink.library.smu.edu.sg/sis_research/9743; https://ink.library.smu.edu.sg/context/sis_research/article/10743/viewcontent/NeurIPS_2024_Sugar.pdf
• Institution: Singapore Management University

In recent times, Vision-Language Models (VLMs) have been trained under two predominant paradigms. Generative training has enabled Multimodal Large Language Models (MLLMs) to tackle various complex tasks, yet issues such as hallucinations and weak object discrimination persist. Discriminative trainin...