Online multi-face tracking with multi-modality cascaded matching

• Main Authors: Weng, Zhenyu, Zhuang, Huiping, Li, Haizhou, Ramalingam, Balakrishnan, Mohan, Rajesh Elara, Lin, Zhiping
• Other Authors: School of Electrical and Electronic Engineering
• Format: Article
• Language: English
• Published: 2024
• Subjects: Computer and Information Science; Multi-face tracking; Multi-modality information
• Online Access: https://hdl.handle.net/10356/174458
• Institution: Nanyang Technological University

Tracking multiple faces online in unconstrained videos is a challenging problem as faces may appear drastically different over time and identities can be inferred only based on information available from past frames. Previous tracking methods focus on face information without reference to other modality information such as a person’s overall body appearance, leading to suboptimal performance. In this paper, we propose a new online multi-face tracking method, called online multi-face tracking with multi-modality cascaded matching (OMTMCM), to improve the tracking performance by using both face and body information. The proposed OMTMCM consists of two stages, namely detection alignment and detection association. In the first stage, a detection alignment module is designed to align face detection with body detection from the same person for the subsequent detection association. In the second stage, a cascaded matching module is designed to associate face detections across frames to locate trajectory of each target face by using both face and body information. Specifically, aligned face-body detections in the current frame are matched in a cascade manner with body and face features that are selected from past frames and stored in the designed feature memory. In this way, our method can track multiple faces online with both face and body information while eliminating the possibility of face detection and body detection from the same person being separately assigned with different identities. Experimental results demonstrate our method is on par with or better than other online tracking methods for multi-face tracking.