Authors

• Seong-Gyun Jeong
• Chul Lee
• Chang-Su Kim

Abstract

A novel motion-compensated frame interpolation (MCFI) algorithm to increase video temporal resolutions based on multihypothesis motion estimation (MHME) and texture optimization is proposed in this work. First, we form multiple motion hypotheses for each pixel by employing different motion estimation parameters, i.e. different block sizes and different directions. Then, we determine the best motion hypothesis for each pixel by solving a labeling problem and optimizing the parameters. In the labeling problem, the cost function is composed of color, shape, and angular difference terms. Finally, we refine the motion hypothesis field based on the texture optimization technique and blend multiple source pixels to interpolate each pixel in the intermediate frame. Simulation results demonstrate that the proposed algorithm provides significantly better MCFI performance than conventional algorithms.

Experiental results

We compare the performance of the proposed algorithm with those of conventional MCFI algorithms. In the experiment, even frames in the test sequences are skipped and then interpolated. These results confirm that the proposed algorithm outperforms the conventional algorithms. Specifically, whereas the conventional algorithms yield flickering artifacts, the proposed algorithm provides smoother transitions between input frames and intermediate frames.

[1] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory of warping,” in Proc. ECCV, May 2004, pp. 25–36.
[2] B.-D. Choi, J.-W. Han, C.-S. Kim, and S.-J. Ko, “Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 4, pp. 407–416, Apr. 2007.
[3] C. Wang, L. Zhang, Y. He, and Y.-P. Tan, “Frame rate up-conversion using trilateral filtering,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 886–893, Jun. 2010.
[4] S.-G. Jeong, C. Lee, and C.-S. Kim, “Exemplar-based frame rate up-conversion with congruent segmentation,” in Proc. IEEE ICIP, Sep.-Oct. 2012, pp. 845–848.