Probability based fast inter mode decision algorithm for H.264/AVC

• Main Authors: Md. Salah Uddin, Yusuf, M. I., Hossain, M. M., Rahman
• Format: Article
• Language: English
• Published: Natural Sciences Publishing Co. 2016
• Subjects: QA75 Electronic computers. Computer science
• Online Access: http://eprints.unisza.edu.my/7503/1/FH02-FSTK-16-06200.jpg; http://eprints.unisza.edu.my/7503/
• Institution: Universiti Sultan Zainal Abidin

In this paper, a new probability based fast inter mode decision algorithm is proposed for H.264/AVC video standard. The main idea is to classify the inter modes into large and small modes category and determine which one is suitable for a macroblock (MB). Depending on this, the probability of each large mode corresponding to the initial Hadamard cost or the probability of each small mode corresponding to the initial Hadamard cost is determined. Then the best mode can be obtained from their probability characteristics. Thus, it is possible to determine the best inter mode only by calculating the Hadamard Transform (H-SAD) cost instead of rate distortion cost. Moreover, it avails the platform to determine the best mode without comparing the cost of each mode to other modes. As a result, this approach can avoid most of the complex computation processes like quantization, variable length coding, pixel reconstruction etc. The experimental results have shown that the proposed algorithm can reduce 60% to 65% of the total encoding time of H.264/AVC with negligible distortion in the rate distortion performance.