Residual frame

About: Residual frame is a research topic. Over the lifetime, 4443 publications have been published within this topic receiving 68784 citations.

Computational Complexity Management of a Real-Time H.264/AVC Encoder

Abstract: The H.264 video coding standard supports efficient coding of video at the expense of high computational complexity. This work addresses the problem of maintaining acceptable video coding performance in a computation-constrained application scenario. A complexity management approach is proposed for an H.264 encoder running in a processor/power-constrained environment. We hypothesize that, in a power-constrained application such as mobile video telephony, good perceptual quality requires a balance between a high frame rate and acceptable image quality. Therefore, the objective of the complexity management approach is to maintain a smooth video frame rate whilst ensuring that the frame quality is not degraded unacceptably. A frame-level algorithm calculates a target coding time for each frame and drops frames when necessary to maintain acceptable image quality. A per-frame algorithm controls the coding complexity of each frame in order to achieve the target coding time. The performance of the approach is evaluated by carrying out subjective tests and comparing the managed complexity encoder with a reference encoder in a computation-constrained scenario. Subjective results show that the managed complexity encoder consistently achieves superior perceptual video quality ratings compared to the reference encoder.

Method for motion compensated frame rate upsampling based on piecewise affine warping

Abstract: Interpolation of a new frame between a previous frame and a current frame of a video stream by motion compensated frame rate upsampling. The interpolation method includes identifying nodes and edges of objects such as triangles present in the previous frame, constructing a superimposed triangular mesh based on the identified nodes and edges, estimating displacement such nodes in the superimposed triangular mesh from the previous frame with respect to the current frame, and rendering the new frame based on the estimated displacement of nodes. Additionally, pixels of the previous frame and the current frame may be classified according to whether a pixel's value has changed from the previous frame to the current frame. This classification may be used during rendering to reduce overall processing time. Pixel-based forward motion estimation may be used to estimate motion of pixels between the previous frame and the current frame and the estimated motion may be used in estimating node displacement.

Optimally Sparse Frames

Abstract: Frames have established themselves as a means to derive redundant, yet stable decompositions of a signal for analysis or transmission, while also promoting sparse expansions. However, when the signal dimension is large, the computation of the frame measurements of a signal typically requires a large number of additions and multiplications, and this makes a frame decomposition intractable in applications with limited computing budget. To address this problem, in this paper, we focus on frames in finite-dimensional Hilbert spaces and introduce sparsity for such frames as a new paradigm. In our terminology, a sparse frame is a frame whose elements have a sparse representation in an orthonormal basis, thereby enabling low-complexity frame decompositions. To introduce a precise meaning of optimality, we take the sum of the numbers of vectors needed from this orthonormal basis when expanding each frame vector as sparsity measure. We then analyze the recently introduced algorithm Spectral Tetris for construction of unit norm tight frames and prove that the tight frames generated by this algorithm are in fact optimally sparse with respect to the standard unit vector basis. Finally, we show that even the generalization of Spectral Tetris for the construction of unit norm frames associated with a given frame operator produces optimally sparse frames.

Bitplane coding for macroblock field/frame coding type information

Abstract: In one aspect, for a first interlaced video frame in a video sequence, a decoder decodes a bitplane signaled at frame layer for the first interlaced video frame. The bitplane represents field/frame transform types for plural macroblocks of the first interlaced video frame. For a second interlaced video frame in the video sequence, for each of at least one but not all of plural macroblocks of the second interlaced video frame, the decoder processes a per macroblock field/frame transform type bit signaled at macroblock layer. An encoder performs corresponding encoding.

Optical flow estimation using high frame rate sequences

Abstract: Gradient-based optical flow estimation methods such as the Lucas-Kanade (1981) method work well for scenes with small displacements but fail when objects move with large displacements. Hierarchical matching-based methods do not suffer from large displacements but are less accurate. By utilizing the high speed imaging capability of CMOS image sensors, the frame rate can be increased to obtain more accurate optical flow with wide range of scene velocities in real time. Further, by integrating the memory and processing with the sensor on the same chip, optical flow estimation using high frame rate sequences can be performed without unduly increasing the off-chip data rate. The paper describes a method for obtaining high accuracy optical flow at a standard frame rate using high frame rate sequences. The Lucas-Kanade method is used to obtain optical flow estimates at high frame rate, which are then accumulated and refined to obtain optical flow estimates at a standard frame rate. The method is tested on video sequences synthetically generated by perspective warping. The results demonstrate significant improvements in optical flow estimation accuracy with moderate memory and computational power requirements.