FaceCollage : a rapidly deployable system for real-time head reconstruction for on-the-go 3D telepresence

• Main Authors: Tan, Fuwen, Fu, Chi-Wing, Deng, Teng, Cai, Jianfei, Cham, Tat-Jen
• Other Authors: School of Computer Science and Engineering
• Format: Conference or Workshop Item
• Language: English
• Published: 2020
• Subjects: Engineering::Computer science and engineering; 3D Telepresence; RGBD Sensors
• Online Access: https://hdl.handle.net/10356/138271
• Institution: Nanyang Technological University

This paper presents FaceCollage, a robust and real-time system for head reconstruction that can be used to create easy-to-deploy telepresence systems, using a pair of consumer-grade RGBD cameras that provide a wide range of views of the reconstructed user. A key feature is that the system is very simple to rapidly deploy, with autonomous calibration and requiring minimal intervention from the user, other than casually placing the cameras. This system is realized through three technical contributions: (1) a fully automatic calibration method, which analyzes and correlates the left and right RGBD faces just by the face features; (2) an implementation that exploits the parallel computation capability of GPU throughout most of the system pipeline, in order to attain real-time performance; and (3) a complete integrated system on which we conducted various experiments to demonstrate its capability, robustness, and performance, including testing the system on twelve participants with visually-pleasing results.