Segmentation-Based Video Upscaling without Motion Estimation
Abstract
This paper shows an effective method for video upscaling or super resolution (SR) without using an explicit motion estimation step. Exploiting the Non-Local Means (NLM) algorithm in order to bypass motion estimation, which is often complicated, our method proposes some modifications to ensure a good compromise between noise cancelling and detail preservation. A detailed consideration of the NLM algorithm is carried out to propose an efficient distance computation and the best eighbors for the reconstruction of each SR pixel. Moreover, efficient segmentation algorithms are also considered to build a novel upscaling framework that is adapted to spatial contrast. The satisfying results with real videos illustrated the advantages of upscaling without motion estimation compared to motion estimation-based upscaling, as well as the role of segmentation in video super resolution.
Downloads
References
[1] M. Elad and Y. Hel-Or, A Fast Super-resolution Reconstruction Algorithm for Pure Translational Motion and Common Space Invariant Blur, IEEE Trans. Image Process., vol. 10(8), pp. 1187-1193, 2001.
[2] S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, Robust Shift and Add Approach to Super-resolution, in Proc. SPIE Conf. Applications of Digital Signal and Image Processing, pp. 121-130, 2003.
[3] M. Elad and A. Feuer, Restoration of Single Super-resolution Image from Several Blurred, Noisy and Down-sampled Measured Images, IEEE Trans. Image Process., vol. 6(12), pp. 1646-1658, 1997.
[4] A. Buades, B. Coll, and J. M. Morel, Denoising Image Sequences does not Require Motion Estimation, in Proc. IEEE Conf. Advanced Video and Signal Based Surveillance, 2005, pp. 70-74.
[5] M. Protter, M. Elad, H. Takeda, and P. Milanfar, ”Generalizing the Non-Local-Means to Super-Resolution Reconstruction”, IEEE Transactions on Image Processing, Vol. 18(1), pp. 36-51, 2009.
[6] Perrone D. and Favaro P., Total Variation Blind Deconvolution: The Devil is in the Details, IEEE CVPR 2014, pp. 2909-2916.
[7] Kim T. H. and Lee K. M., Segmentation-Free Dynamic Scene Deblurring, IEEE CVPR 2014, pp. 2766-2773.
[8] W. Zuo, L. Zhang, C. Song, and D. Zhang, Gradient Histogram Estimation and Preservation for Texture Enhance Image Denoising, IEEE Transaction on Image Processing, vol. 23(6), pp. 2459-72, 2014.
[9] T. Ho-Phuoc, A. Dupret, and L. Alacoque, Super Resolution Method Adapted to Spatial Contrast, in Proc. IEEE ICIP, 2013, pp. 976-980.
[10] P. Getreuer, Linear Methods for Image Interpolation, IPOL Journal, 2011. http://dx.doi.org/10.5201/ipol.2011.g lmii.
