Bitstream-corrupted video recovery: a novel benchmark dataset and method

• Main Authors: Liu, Tianyi, Wu, Kejun, Wang, Yi, Liu, Wenyang, Yap, Kim-Hui, Chau, Lap-Pui
• Other Authors: School of Electrical and Electronic Engineering
• Format: Conference or Workshop Item
• Language: English
• Published: 2024
• Subjects: Computer and Information Science; Video Restoration; Bitstream Corruption
• Online Access: https://hdl.handle.net/10356/174533; https://proceedings.neurips.cc/paper_files/paper/2023
• Institution: Nanyang Technological University

The past decade has witnessed great strides in video recovery by specialist technologies, like video inpainting, completion, and error concealment. However, they typically simulate the missing content by manual-designed error masks, thus failing to fill in the realistic video loss in video communication (e.g., telepresence, live streaming, and internet video) and multimedia forensics. To address this, we introduce the bitstream-corrupted video (BSCV) benchmark, the first benchmark dataset with more than 28,000 video clips, which can be used for bitstream-corrupted video recovery in the real world. The BSCV is a collection of 1) a proposed three-parameter corruption model for video bitstream, 2) a large-scale dataset containing rich error patterns, multiple corruption levels, and flexible dataset branches, and 3) a new video recovery framework that serves as a benchmark. We evaluate state-of-the-art video inpainting methods on the BSCV dataset, demonstrating existing approaches' limitations and our framework's advantages in solving the bitstream-corrupted video recovery problem. The benchmark and dataset are released at https://github.com/LIUTIGHE/BSCV-Dataset.