Residual frame

About: Residual frame is a research topic. Over the lifetime, 4443 publications have been published within this topic receiving 68784 citations.

Iterative Side-Information Generation in a Mixed Resolution Wyner-Ziv Framework

Abstract: We propose a mixed resolution framework based on full resolution key frames and spatial-reduction-based Wyner-Ziv coding of intermediate nonreference frames Improved rate-distortion performance is achieved by enabling better side-information generation at the decoder side and better rate-allocation at the encoder side The framework enables reduced encoding complexity by low resolution encoding of the nonreference frames, followed by Wyner-Ziv coding of the Laplacian residue The quantized transform coefficients of the residual frame are mapped to cosets without the use of a feedback channel A study to select optimal coding parameters in the creation of the memoryless cosets is made Furthermore, a correlation estimation mechanism that guides the parameter choice process is proposed The decoder first decodes the low resolution base layer and then generates a super-resolved side-information frame at full resolution using past and future key frames Coset decoding is carried using side-information to obtain a higher quality version of the decoded frame Implementation results are presented for the H264/AVC codec

Gradient-based residual variance modeling and its applications to motion-compensated video coding

Abstract: This paper analyzes the relationship between the residual frame and the previous frame in motion-compensated video coding. It is found that the variance of the residual signal depends on the gradient magnitude. On average, the variance of the residual signal is larger for pixels with larger gradient magnitude. Two applications of this analysis are presented. In the first one, the relationship between the residual signal variance and the gradient magnitude is used to model the second-order statistics of the residual field in a nonstationary way. This modeling enables more efficient residual signal coding. The other application is for pixel decimation-based fast block matching. It is proposed that pixels with the largest gradient magnitude in a block be chosen to participate in the block matching process. It is demonstrated that such a gradient-adaptive subsampling achieves great advantage over two other known subsampling methods.

Method and apparatus for effectively encoding multi-layered motion vectors

Abstract: An apparatus and method for improving the compression efficiency of a motion vector by efficiently predicting a motion vector in an enhanced layer from a motion vector in a base layer in a video coding method using a multi-layer structure are provided. The method includes obtaining a motion vector in a mother frame of a base layer that is temporally closest to an unsynchronized frame of a current layer, obtaining a predicted motion vector from the motion vector in the mother frame considering the referencing direction in the mother frame and in the unsynchronized frame and distances between the mother frame and a reference frame and between the unsynchronized frame and a reference frame, generating a residual between the motion vector in the unsynchronized frame and the predicted motion vector, and encoding the motion vector in the mother frame and the residual.

Real-time three-dimensional reconstruction of a scene from a single camera

Abstract: A combination of three computational components may provide memory and computational efficiency while producing results with little latency, e.g., output can begin with the second frame of video being processed. Memory usage may be reduced by maintaining key frames of video and pose information for each frame of video. Additionally, only one global volumetric structure may be maintained for the frames of video being processed. To be computationally efficient, only depth information may be computed from each frame. Through fusion of multiple depth maps from different frames into a single volumetric structure, errors may average out over several frames, leading to a final output with high quality.

In-Block Prediction-Based Mixed Lossy and Lossless Reference Frame Recompression for Next-Generation Video Encoding

Abstract: Frame recompression is an efficient way to reduce the huge bandwidth of external memory for video encoder, especially for P/B frame compression. A novel algorithm, which is called mixed lossy and lossless (MLL) reference frame recompression, is proposed in this paper. The bandwidth reduction comes from two sources in our scheme, which differs from its previous designs and achieves a much higher compression ratio. First, it comes from pixel truncation. We use truncated pixels (PR) for integer motion estimation (IME) and acquire truncated residuals for factional motion estimation (FME) and motion compensation (MC). Because the pixel access of IME is much larger than FME and MC, it saves about 37.5% bandwidth under 3-b truncation. Second, embedded compression of PR helps to further reduce data. The truncated pixels in the first stage greatly help to achieve a higher compression ratio than current designs. From our experiments, 3-b truncated PR can be compressed to 15.4% of the original data size, while most current embedded compressions can only achieve around 50%. For PR compression, two methods are proposed: in-block prediction and small-value optimized variable length coding. With these experiments, the total bandwidth can be reduced to 25.5%. Our proposed MLL is hardware/software friendly and also fast IME algorithm friendly frame recompression scheme. It is more suitable to work together with the data-reuse strategy than the previous schemes, and the video quality degradation is controllable and negligible.