Multi modal video analysis with LLM for descriptive emotion and expression annotation

• 主要作者: Fan, Yupei
• 其他作者: Zheng Jianmin
• 格式: Final Year Project
• 語言: English
• 出版: Nanyang Technological University 2024
• 主題: Computer and Information Science; Video understanding; Large language model (LLM); Multimodal analysis; Feature extraction; Deep learning; Emotion annotation
• 在線閱讀: https://hdl.handle.net/10356/180715
• 機構: Nanyang Technological University

This project presents a novel approach to multi-modal emotion and action annotation by integrating facial expression recognition, action recognition, and audio-based emotion analysis into a unified framework. The system utilizes TimesFormer, OpenFace, and SpeechBrain to extract relevant features fro...