Restricted affine motion compensation in video coding using particle filtering
12 Dec 2010-pp 479-484
TL;DR: This work proposes a novel particle filter-based motion compensation strategy for video coding that uses a higher order linear model in place of the traditional translational model used in standards such as H.264.
Abstract: We propose a novel particle filter-based motion compensation strategy for video coding. We use a higher order linear model in place of the traditional translational model used in standards such as H.264. The measurement/observation process in the particle filter is a computationally efficient mechanism as opposed to traditional search methods. We use a multi-resolution framework for efficient parameter estimation. Results of our experimentation show reduced residual energy and better PSNR as compared to traditional video coding methods, especially in regions of complex motion such as zooming and rotation.
Citations
More filters
DOI•
04 Jun 2012TL;DR: An automatic approach, SAMBA, that computes a pleasing choreography by using a novel combination of a distance weighted, least-squares registration between a previous and a subsequent frame and of a modified SAM interpolation.
Abstract: Given the start positions of a group of dancers, a choreographer specifies their end positions and says: "Run!" Each dancer has the choice of his/her motion. These choices influence the perceived beauty (or grace) of the overall choreography. We report experiments with an automatic approach, SAMBA, that computes a pleasing choreography. Rossignac and Vinacua focused on affine motions, which, in the plane, correspond to choreographies for three independent dancers. They proposed the inverse of the Average Relative Acceleration (ARA) as a measure of grace and their Steady Affine Morph (SAM) as the most graceful interpolating motion. Here, we extend their approach to larger groups. We start with a discretized (uniformly time-sampled) choreography, where each dancer moves with constant speed. Each SAMBA iteration steadies the choreography by tweaking the positions of dancers at all intermediate frames towards corresponding predicted positions. The prediction for the position of dancer at a given frame is computed by using a novel combination of a distance weighted, least-squares registration between a previous and a subsequent frame and of a modified SAM interpolation. SAMBA is fully automatic, converges in a fraction of a second, and produces pleasing and interesting motions.
3 citations
TL;DR: Results of extensive experimentation show reduced residual energy and better Peak Signal-to-Noise Ratio (PSNR) as compared to H.264/HEVC for instance, especially in regions of complex motion such as zooming and rotation.
Abstract: In this paper, we propose a multi-resolution affine block-based tracker for motion estimation and compensation, compatible with existing video coding standards such as H.264 and HEVC. We propose three modifications to traditional motion compensation techniques in video coding standards such as H.264 and HEVC. First, we replace traditional search methods with an efficient particle filtering-based method, which incorporates information from both spatial and temporal continuity. Second, we use a higher order linear model in place of the traditional translation motion model in these standards to efficiently represent complex motions such as rotation and zoom. Third, we propose a multi-resolution framework that enables efficient parameter estimation. Results of extensive experimentation show reduced residual energy and better Peak Signal-to-Noise Ratio (PSNR, hereafter) as compared to H.264/HEVC for instance, especially in regions of complex motion such as zooming and rotation.
2 citations
Cites methods from "Restricted affine motion compensati..."
...Our previous work [28] was presented more as a proof of concept which we have extended extensively in this submission by: a) integration of the proposed framework into the H....
[...]
...This work builds on some very preliminary ideas of the authors in [28]....
[...]
References
More filters
TL;DR: The combination of affine and long-term memory motion-compensated prediction provides a highly efficient video compression scheme in terms of rate-distortion performance and the two incorporated multipicture concepts complement each other well providing almost additive rate- Distortion gains.
Abstract: Affine motion compensation is combined with long-term memory motion-compensated prediction. The idea is to determine several affine motion parameter sets on subareas of the image. Then, for each affine motion parameter set, a complete reference picture is warped and inserted into the multipicture buffer. Given the multipicture buffer of decoded pictures and affine warped versions thereof, block-based translational motion-compensated prediction and Lagrangian coder control are utilized. The affine motion parameters are transmitted as side information requiring additional bit rate. Hence, the utility of each reference picture and, with that, each affine motion parameter set is tested for its rate-distortion efficiency. The combination of affine and long-term memory motion-compensated prediction provides a highly efficient video compression scheme in terms of rate-distortion performance. The two incorporated multipicture concepts complement each other well providing almost additive rate-distortion gains. When warping the prior decoded picture, average bit-rate savings of 15% against TMN-10, the test model of ITU-T Recommendation H.263, are reported for the case that 20 warped reference pictures are used. When employing 20 warped reference pictures and 10 decoded reference pictures, average bit-rate savings of 24% can be obtained for a set of eight test sequences. These bit-rate savings correspond to gains in PSNR between 0.8-3 dB. For some cases, the combination of affine and long-term memory motion-compensated prediction provides more than additive gains.
89 citations
"Restricted affine motion compensati..." refers background in this paper
...[9], affine motion compensation is combined with long-term memory motion-compensated prediction....
[...]
TL;DR: The analysis of computational complexity and the simulation results indicate that the proposed method achieves significant savings on computation along with smoother motion vector fields and similar picture quality, when compared to the conventional full search algorithm.
Abstract: Motion estimation plays an important role for the compression of video signals. This paper presents a new block-based motion estimation method using Kalman filtering. The new method utilizes the predicted motion and measured motion to obtain an optimal estimate of motion vector. The autoregressive models are employed to fit the motion correlation between neighboring blocks and then achieve predicted motion information. The measured motion information is obtained by the conventional block-based fast search schemes. Several algorithms based on either one- or two dimensional models using either nonadaptive or adaptive Kalman filters are developed. The analysis of computational complexity and the simulation results indicate that the proposed method achieves significant savings on computation along with smoother motion vector fields and similar picture quality, when compared to the conventional full search algorithm.
47 citations
"Restricted affine motion compensati..." refers background or methods in this paper
...The authors in [6] use Kalman filtering, but only for refinement of the motion vector....
[...]
...The authors in [6] consider only a translational model....
[...]
07 May 1996
TL;DR: This paper proposes a multiple layer video codec which uses affine model for large regions to improve motion compensation and shows that the proposed method gives better results in terms of the bit rate under the same PSNR constraint for most of the tested sequences.
Abstract: The aim in variable block size and object based video coding is to motion compensate as large regions as possible. Whereas the translational motion model is adequate to motion compensate small regions even if the image sequence involves complex motion, it is no longer capable of delivering satisfactory results for large regions. A more sophisticated motion model is required in such regions. In this paper, we extend our previous approach by proposing a multiple layer video codec which uses affine model for large regions to improve motion compensation. The translational model is still used for small regions to reduce the bit overhead. A multi-resolution robust Hough transform based motion estimation technique is used to estimate the affine motion parameters. The experimental results show that the proposed method gives better results in terms of the bit rate under the same PSNR constraint for most of the tested sequences as compared with our previous approach and an implementation of the standard H.261.
18 citations
"Restricted affine motion compensati..." refers background in this paper
...[10] propose an affine model for the same....
[...]
07 May 1993
TL;DR: In this paper, Wu et al. presented a method for motion segmentation into regions satisfying a parametric motion constraint performed in parallel with the estimation of motion parameters, which is similar to those of Wu (1990) with a number of modifications which include multiresolution in image space with coarse-to-fine resolution object tracking, a new rule for searching the minima in Hough space, and an improved segmentation algorithm.
Abstract: The authors face the problem of motion segmentation (into regions satisfying a parametric motion constraint) performed in parallel with estimation of motion parameters. Particular attention is attached to the presence of several independently moving objects registered from a non-stationary sensor; the complex motion model; and the parallel segmentation and estimation; speed of the method. The principles of the algorithm presented are similar to those of Wu (1990) with a number of modifications which include multiresolution in image space with coarse-to-fine resolution object tracking, a new rule for searching the minima in Hough space, and an improved segmentation algorithm. Experimental results confirm that these result in an improved, more robust convergence and better accuracy. >
12 citations
16 Dec 2008
TL;DR: This paper proposes a novel spatial multi-scale tracker that tracks at the optimal accuracy-versus-speed operating point, and extends these ideas to a multi-resolution spatio-temporal tracker.
Abstract: This paper proposes efficient and robust methods for tracking a moving object at multiple spatial and temporal resolution levels. The efficiency comes from optimising the amounts of spatial and temporal data processed. The robustness results from multi-level coarse-to-fine state-space searching. Tracking across resolution levels incurs an accuracy-versus-speed trade-off. For example, tracking at higher resolutions incurs greater processing cost, while maintaining higher accuracy in estimating the position of the moving object. We propose a novel spatial multi-scale tracker that tracks at the optimal accuracy-versus-speed operating point. Next, we relax this requirement to propose a multi-resolution tracker that operates at a minimum acceptable performance level. Finally, we extend these ideas to a multi-resolution spatio-temporal tracker. We show results of extensive experimentation in support of the proposed approaches.
10 citations
"Restricted affine motion compensati..." refers background in this paper
...Making a tracker robust to such situations is not difficult in a multi-scale scenario [3]....
[...]