Document Type
Pre-Print
Publication Date
3-1-2017
Subjects
Interpolation -- Applications to image processing, Image processing -- Digital techniques, Digital video, Convolutions (Mathematics) -- Data processing
Abstract
Video frame interpolation typically involves two steps: motion estimation and pixel synthesis. Such a two-step approach heavily depends on the quality of motion estimation. This paper presents a robust video frame interpolation method that combines these two steps into a single process. Specifically, our method considers pixel synthesis for the interpolated frame as local convolution over two input frames. The convolution kernel captures both the local motion between the input frames and the coefficients for pixel synthesis. Our method employs a deep fully convolu- tional neural network to estimate a spatially-adaptive con- volution kernel for each pixel. This deep neural network can be directly trained end to end using widely available video data without any difficult-to-obtain ground-truth data like optical flow. Our experiments show that the formula- tion of video interpolation as a single convolution process allows our method to gracefully handle challenges like oc- clusion, blur, and abrupt brightness change and enables high-quality video frame interpolation.
Persistent Identifier
http://archives.pdx.edu/ds/psu/19536
Citation Details
Niklaus, S., Mai, L., & Liu, F. (2017). Video Frame Interpolation via Adaptive Convolution. arXiv preprint arXiv:1703.07514.
Description
Originally published in arViv.org. This is the author manuscript of a paper submitted for the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).