Cordula Heithausen

Bio: Cordula Heithausen is an academic researcher from RWTH Aachen University. The author has contributed to research in topics: Motion compensation & Motion estimation. The author has an hindex of 4, co-authored 8 publications receiving 48 citations.

Motion compensation with higher order motion models for HEVC

Abstract: In recent video compression standards only translational motion is accurately compensated, constraining any higher order motion to be approximated by being split into smaller translational units. The objective of this paper is to improve the coding efficiency for video sequences containing complex motion.Various higher order motion models are considered and evaluated to this end. Motivated by the higher maximum block size introduced by the High Efficiency Video Coding Standard (HEVC) and a suitable combination of motion parameter estimation, interpolation and coding, a higher order motion compensation framework for the HEVC is proposed. Through this an average data reduction of 2.9% as well as an increase in average block size is achieved.

Improved higher order motion compensation in HEVC with block-to-block translational shift compensation

Abstract: Conventionally, complex motion in video sequences is approximated by smaller block units in order to be representable by a translational motion model. This approximation results in a fine block partitioning and a high prediction error, both at cost of more data rate than potentially necessary. A worthwhile data reduction has been shown to be achievable by adding a higher order motion model to the most recent video coding standard, High Efficiency Video Coding (HEVC). The benefit of this additional option of inter-frame prediction is due to the more accurate motion compensation as well as the usage of larger block sizes. This paper deals with more efficient encoding of higher order motion parameters in this context. The geometrically accurate prediction of higher order motion parameters from a neighbored block needs to consider the dependency of the block-to-block parameter difference based on the spatial relation between two block centers. An algorithm is introduced for correcting the translational component and reducing the difference between the actual and the predicted motion when determining higher order parameters from neighbored blocks. Additionally, a further increase of the maximum block size up to 512×512 pixels is investigated.

Segmentation-based partitioning for motion compensated prediction in video coding

Abstract: In video coding standards such as Advanced Video Coding (AVC) and its successor High Efficiency Video Coding (HEVC), motion compensation is performed by partitioning each inter-predicted picture into square or rectangular regions. While HEVC introduces an efficient quad-tree based splitting of square-shaped coding blocks into prediction blocks by symmetric and asymmetric motion partitioning, the boundaries of natural moving objects can only be approximated by a fine block partitioning, resulting in a redundant representation of the motion and therefore a potential coding overhead. This paper studies the method of using a segmentation based partitioning of coding blocks into arbitrary shaped segments, where the segmentation is performed on coded reference pictures. Experimental results show that for cases where a reasonable segmentation can be obtained, bitrate reductions of around 2% can be achieved.

Temporal Prediction of Motion Parameters with Interchangeable Motion Models

Abstract: While the translational motion model remains predominant in motion compensation, better coding efficiency can be achieved by applying a higher order motion model in cases of non-translationally moving video content. But so far, temporal motion vector prediction has not been fully optimized in case of higher order motion compensation. In particular, if the motion model provided by the reference picture differs from the motion model of the current block, the temporal prediction is either not used at all or it is sub-optimal. Also, temporal predictors of a much smaller partition size might not provide the best suited motion parameter prediction to a larger current block. In order to overcome these issues, a new method of temporal motion prediction is introduced in this paper, allowing for flexible switching between motion models. The picture-wise dense translational motion vector field is calculated from both translational and higher order motion parameters with a configurable granularity of 4x4 pixel subpartitions down to pixelwise accuracy. Both a translational and a higher order motion parameter predictor are estimated from that vector field, thus giving the current block two alternatives to choose from. The proposed algorithm achieves rate reductions of about 2% on average compared to the previous higher order motion compensation system it is based on, now resulting in an average of around 20% efficiency gain for non-translational video content, compared to HEVC without such an option.

Geometry-adaptive motion partitioning using improved temporal prediction

Abstract: Current state of the art video codecs such as HEVC are based on rectangular motion partitioning and compensation. To further enhance the compression performance, more flexible block partitioning strategies are needed. We present geometry based motion partitioning (GMP) in a post-HEVC framework with improved temporal prediction of GMP parameters. Our main contribution is a simple yet efficient temporal projection method for GMP parameters using available reference picture motion vectors, which allows the tracking of geometric partitioning lines along a motion trajectory. Average bit rate reductions by 1.35% are reported.

An Efficient Four-Parameter Affine Motion Model for Video Coding

Abstract: In this paper, we study a simplified affine motion model-based coding framework to overcome the limitation of a translational motion model and maintain low-computational complexity. The proposed framework mainly has three key contributions. First, we propose to reduce the number of affine motion parameters from 6 to 4. The proposed four-parameter affine motion model can not only handle most of the complex motions in natural videos, but also save the bits for two parameters. Second, to efficiently encode the affine motion parameters, we propose two motion prediction modes, i.e., an advanced affine motion vector prediction scheme combined with a gradient-based fast affine motion estimation algorithm and an affine model merge scheme, where the latter attempts to reuse the affine motion parameters (instead of the motion vectors) of neighboring blocks. Third, we propose two fast affine motion compensation algorithms. One is the one-step sub-pixel interpolation that reduces the computations of each interpolation. The other is the interpolation-precision-based adaptive block size motion compensation that performs motion compensation at the block level rather than the pixel level to reduce the number of interpolation. Our proposed techniques have been implemented based on the state-of-the-art high-efficiency video coding standard, and the experimental results show that the proposed techniques altogether achieve, on average, 11.1% and 19.3% bits saving for random access and low-delay configurations, respectively, on typical video sequences that have rich rotation or zooming motions. Meanwhile, the computational complexity increases of both the encoder and the decoder are within an acceptable range.

Advanced Spherical Motion Model and Local Padding for 360° Video Compression

Abstract: The 360° video compression has two main challenges due to projection distortions, namely, the geometry distortion and the face boundary discontinuity. There are some tradeoffs between selecting equi-rectangular projection (ERP) and polyhedron projection. In ERP, the geometry distortion is severer than the face boundary discontinuity; while for the polyhedron projections, the face boundary discontinuity is severer than the geometry distortion. These two distortions will have side effects on the motion compensation and undermine the compression efficiency of the 360° video. In this paper, an integrated framework is developed to handle these two problems to improve coding efficiency. The proposed framework mainly has two key contributions. First, we derive a unified advanced spherical motion model to handle the geometry distortion of different projection formats for the 360° video. When fitting the projection between the various projection formats and the sphere into the unified framework, a specific solution can be obtained for each projection format. Second, we propose a local 3D padding method to handle the face boundary discontinuity between the neighboring faces in various projection formats of the 360° video. The local 3D padding method can be applied to different projection formats through setting different angles between neighboring faces. These two methods are independent of each other and can also be combined into an integrated framework to achieve a better rate-distortion performance. The proposed framework can be seamlessly integrated into the latest video coding standard high-efficiency video coding. The experimental results demonstrate that introducing proposed coding tools can achieve significant bitrate savings compared with the current state-of-the-art method.

Geometric Partitioning Mode in Versatile Video Coding: Algorithm Review and Analysis

Abstract: This paper presents an overview of the geometric partitioning mode (GPM) algorithm that is a part of the most recent Versatile Video Coding (VVC) standard. The GPM algorithm aims to increase the partitioning precision of moving objects using non-rectangular and asymmetric rectangular partitions on top of the conventional rectangular block partitioning structure of VVC. Novel features of GPM contributing to the increase in coding efficiency and the reduction in encoder and decoder complexity are detailed and analyzed in this paper. Evaluated with VVC test model version 8.0 under the joint video experts team common test conditions, experimental results show that the presented GPM algorithm provides luma Bjontegaard Delta rate reduction of 0.70% for random access and of 1.55% for low delay with B slices configurations, with roughly 3% to 5% additional encoding time and negligible decoder runtime change. Furthermore, as GPM provides more precise partitions for the boundaries of the moving objects, an improvement of visual quality is seen in GPM coded sequences.

Deformable block-based motion estimation in omnidirectional image sequences

Abstract: This paper presents an extension of block-based motion estimation for omnidirectional videos, based on a translational object motion model that accounts for the spherical geometry of the imaging system. We use this model to design a new algorithm to perform block matching in sequences of panoramic frames that are the result of the equirectangular projection. Experimental results demonstrate that significant gains can be achieved with respect to the classical exhaustive block matching algorithm in terms of accuracy of motion prediction. In particular, average quality improvements up to approximately 6 dB in terms of Peak Signal to Noise Ratio (PSNR), 0.043 in terms of Structural SIMilarity index (SSIM), and 2 dB in terms of spherical PSNR, can be achieved on the predicted frames.

Subblock-Based Motion Derivation and Inter Prediction Refinement in the Versatile Video Coding Standard

Abstract: Efficient representation and coding of fine-granular motion information is one of the key research areas for exploiting inter-frame correlation in video coding. Representative techniques towards this direction are affine motion compensation (AMC), decoder-side motion vector refinement (DMVR), and subblock-based temporal motion vector prediction (SbTMVP). Fine-granular motion information is derived at subblock level for all the three coding tools. In addition, the obtained inter prediction can be further refined by two optical flow-based coding tools, the bi-directional optical flow (BDOF) for bi-directional inter prediction and the prediction refinement with optical flow (PROF) exclusively used in combination with AMC. The aforementioned five coding tools have been extensively studied and finally adopted in the Versatile Video Coding (VVC) standard. This paper presents technical details of each tool and highlights the design elements with the consideration of typical hardware implementations. Following the common test conditions defined by Joint Video Experts Team (JVET) for the development of VVC, 5.7% bitrate reduction on average is achieved by the five tools. For test sequences characterized by large and complex motion, up to 13.4% bitrate reduction is observed. Additionally, visual quality improvement is demonstrated and analyzed.