FPGA-efficient optical flow computation for real-time vision-based applications

• Main Author: Chen, Yu Hang
• Other Authors: Lam Siew Kei
• Format: Final Year Project
• Language: English
• Published: 2017
• Subjects: DRNTU::Engineering::Computer science and engineering
• Online Access: http://hdl.handle.net/10356/70090
• Institution: Nanyang Technological University

Optical flow is an essential image processing algorithm that can be used to acquire motion information of objects in the environment. For example, an automotive vehicle can use the optical flow to determine the motion of the objects in the road scene (such as vehicles and pedestrians) to avoid collision. Today there are increasing number of image processing applications on embedded systems (e.g. smart phones, wearable devices, etc.) many of which runs on stand-alone power source such as batteries. Implementing the optical flow algorithm on battery operated embedded platforms poses a significant challenge in meeting speed and power consumption requirements as the algorithm consists of many complex mathematical operations and requires large memory accesses. This provides the motivation for this project to develop hardware efficient strategies for the optical flow algorithm. In this study, the hardware-based optical flow algorithm was introduced and implemented on FPGA. There are many algorithms used to determine the optical flow of the objects including Horn and Schunck and Lucas Kanade method. In this report, we focus on the Iterative Lucas Kanade method, which is a method introduced by the Lucas Kanade. The software-based iterative Lucas Kanade optical flow algorithm was implemented in MATLAB for algorithm verification. Due to limited time given for the project, only one iteration and one pyramid level of Lucas Kanade optical flow algorithm was investigated for hardware implementation. A Java version of the optical flow algorithm was written for verifying the results of the Verilog implementation. Several challenges were encountered and resolved during the hardware design of the optical flow algorithm. The first Verilog design of the Lucas Kanade optical flow could not be implemented on the De2i-150 development board, which was the targeted board in this study, due to limited SDRAM access capabilities. This led to a second Verilog design of the optical flow algorithm to overcome this limitation by using block memories of the FPGA instead of the external SDRAM. The final implementation of the optical flow was successfully ported and demonstrated on the FPGA platform.