Multi modal video analysis with LLM for descriptive emotion and expression annotation

• المؤلف الرئيسي: Fan, Yupei
• مؤلفون آخرون: Zheng Jianmin
• التنسيق: Final Year Project
• اللغة: English
• منشور في: Nanyang Technological University 2024
• الموضوعات: Computer and Information Science; Video understanding; Large language model (LLM); Multimodal analysis; Feature extraction; Deep learning; Emotion annotation
• الوصول للمادة أونلاين: https://hdl.handle.net/10356/180715
• المؤسسة: Nanyang Technological University

This project presents a novel approach to multi-modal emotion and action annotation by integrating facial expression recognition, action recognition, and audio-based emotion analysis into a unified framework. The system utilizes TimesFormer, OpenFace, and SpeechBrain to extract relevant features fro...