A novel algorithm to remove rain or snow streaks from a video sequence using temporal correlation and low rank matrix completion is proposed in this work. Based on the observation that rain streaks are too small and move too fast to affect the optical flow estimation between consecutive frames, we obtain an initial rain map by subtracting temporally warped frames from a current frame. Then, we decompose the initial rain map into basis vectors based on the sparse representation, and classify those basis vectors into rain streak ones and outliers with a support vector machine. We then refine the rain map by excluding the outliers. Finally, we remove the detected rain streaks, by employing a low-rank matrix completion technique based on the expectation maximization algorithm. Furthermore, we also extend the proposed algorithm to stereo video deraining. Experimental results demonstrate that the proposed algorithm detects and removes rain or snow streaks efficiently, outperforming conventional algorithms.
| [1] | K. Garg and S.K. nayar, "Vision and rain," Int. J. Comput. Vis. vol. 75, no. 1, pp. 3–27, Oct. 2007 |
| [2] | L.-W. Kang, C.-W. Lin, and Y.-H. Fu, "Automatic single-image-based rain streaks removal via image decomposition," IEEE Trans. Image Process. vol. 21, no. 4, pp. 1742–1755, Apr. 2012 |
| [3] | P.C. Barnum, S.G. Narasimhan, and T. Kanade, "Analysis of rain and snow in frequency space," Int. J. Comput. Vis. vol. 86, no. 2/3, pp. 256–274, Jan. 2010 |
| [4] | X. Zhang, H. Li Y. Qi, W.K. Leow, and T.K. Ng, "Rain removal in video by combining temporal and chromatic properties," in Proc. IEEE ICME Jul. 2006, pp. 461–464 |