Journal ArticleDOI
High-Order Model and Dynamic Filtering for Frame Rate Up-Conversion
Reads0
Chats0
TLDR
A novel frame rate up-conversion method through high-order model and dynamic filtering (HOMDF) for video pixels that minimizes the auto-regressive prediction error of intensity variation by its past samples and minimizes video frame’s reconstruction error along the motion trajectory.Abstract:
This paper proposes a novel frame rate up-conversion method through high-order model and dynamic filtering (HOMDF) for video pixels. Unlike the constant brightness and linear motion assumptions in traditional methods, the intensity and position of the video pixels are both modeled with high-order polynomials in terms of time. Then, the key problem of our method is to estimate the polynomial coefficients that represent the pixel's intensity variation, velocity, and acceleration. We propose to solve it with two energy objectives: one minimizes the auto-regressive prediction error of intensity variation by its past samples, and the other minimizes video frame's reconstruction error along the motion trajectory. To efficiently address the optimization problem for these coefficients, we propose the dynamic filtering solution inspired by video's temporal coherence. The optimal estimation of these coefficients is reformulated into a dynamic fusion of the prior estimate from pixel's temporal predecessor and the maximum likelihood estimate from current new observation. Finally, frame rate up-conversion is implemented using motion-compensated interpolation by pixel-wise intensity variation and motion trajectory. Benefited from the advanced model and dynamic filtering, the interpolated frame has much better visual quality. Extensive experiments on the natural and synthesized videos demonstrate the superiority of HOMDF over the state-of-the-art methods in both subjective and objective comparisons.read more
Citations
More filters
Proceedings ArticleDOI
Depth-Aware Video Frame Interpolation
TL;DR: DAIN as mentioned in this paper proposes a depth-aware flow projection layer to synthesize intermediate flows that preferably sample closer objects than farther ones, and then warps the input frames, depth maps, and contextual features based on the optical flow and local interpolation kernels.
Posted Content
Depth-Aware Video Frame Interpolation
TL;DR: A video frame interpolation method which explicitly detects the occlusion by exploring the depth information, and develops a depth-aware flow projection layer to synthesize intermediate flows that preferably sample closer objects than farther ones.
Posted Content
MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement
TL;DR: A novel adaptive warping layer is developed to integrate both optical flow and interpolation kernels to synthesize target frame pixels and is fully differentiable such that both the flow and kernel estimation networks can be optimized jointly.
Journal ArticleDOI
MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement
TL;DR: In this article, a novel adaptive warping layer is developed to integrate both optical flow and interpolation kernels to synthesize target frame pixels, which is fully differentiable such that both the flow and kernel estimation networks can be optimized.
Journal ArticleDOI
Video Frame Interpolation via Deformable Separable Convolution
Xianhang Cheng,Zhenzhong Chen +1 more
TL;DR: Experimental results demonstrate that the DSepConv method significantly outperforms the other kernel-based interpolation methods and shows strong performance on par or even better than the state-of-the-art algorithms both qualitatively and quantitatively.
References
More filters
Journal ArticleDOI
Image quality assessment: from error visibility to structural similarity
TL;DR: In this article, a structural similarity index is proposed for image quality assessment based on the degradation of structural information, which can be applied to both subjective ratings and objective methods on a database of images compressed with JPEG and JPEG2000.
Book ChapterDOI
A New Approach to Linear Filtering and Prediction Problems
TL;DR: In this paper, the clssical filleting and prediclion problem is re-examined using the Bode-Shannon representation of random processes and the?stat-tran-sition? method of analysis of dynamic systems.
Journal ArticleDOI
Overview of the High Efficiency Video Coding (HEVC) Standard
TL;DR: The main goal of the HEVC standardization effort is to enable significantly improved compression performance relative to existing standards-in the range of 50% bit-rate reduction for equal perceptual video quality.
Journal ArticleDOI
On the limited memory BFGS method for large scale optimization
Dong C. Liu,Jorge Nocedal +1 more
TL;DR: The numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence, and the convergence properties are studied to prove global convergence on uniformly convex problems.
Book ChapterDOI
High Accuracy Optical Flow Estimation Based on a Theory for Warping
TL;DR: By proving that this scheme implements a coarse-to-fine warping strategy, this work gives a theoretical foundation for warping which has been used on a mainly experimental basis so far and demonstrates its excellent robustness under noise.