Towards robust video frame interpolation with long-term propagation

• Main Author: Huang, Ziqi
• Other Authors: Liu Ziwei
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering; Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/157646
• Institution: Nanyang Technological University

Video frame interpolation aims at synthesizing non-existent frames between two consecutive frames in a video, and its importance can be seen from its wide applications in computer vision. The key to video frame interpolation is predicting the intermediate motions between two given frames so that the synthesized frames are coherent to the input video. The existing approaches of imposing assumptions on motions, such as linear and quadratic trajectories, are not generalizable to complex motions in real-world videos. In this project, we consider a different paradigm for robust and general motion prediction. To more thoroughly understand the motion trajectories, we propose to implicitly track the motion paths through long-term propagation. The motion features are then used to refine the motion predicted from the primitive motion assumptions. Our idea of motion propagation can be easily incorporated into existing video frame interpolation approaches. Quantitative and qualitative results demonstrate that our module is able to improve the performance of existing frame interpolation methods. In addition, we conduct analytical experiments to study the mechanism, advantages, and disadvantages of long-term propagation in video frame interpolation to inspire future works.