Showing papers on "Residual frame published in 2006"

An innovative algorithm for key frame extraction in video summarization

Abstract: Video summarization, aimed at reducing the amount of data that must be examined in order to retrieve the information desired from information in a video, is an essential task in video analysis and indexing applications. We propose an innovative approach for the selection of representative (key) frames of a video sequence for video summarization. By analyzing the differences between two consecutive frames of a video sequence, the algorithm determines the complexity of the sequence in terms of changes in the visual content expressed by different frame descriptors. The algorithm, which escapes the complexity of existing methods based, for example, on clustering or optimization strategies, dynamically and rapidly selects a variable number of key frames within each sequence. The key frames are extracted by detecting curvature points within the curve of the cumulative frame differences. Another advantage is that it can extract the key frames on the fly: curvature points can be determined while computing the frame differences and the key frames can be extracted as soon as a second high curvature point has been detected. We compare the performance of this algorithm with that of other key frame extraction algorithms based on different approaches. The summaries obtained have been objectively evaluated by three quality measures: the Fidelity measure, the Shot Reconstruction Degree measure and the Compression Ratio measure.

Steganography in Compressed Video Stream

Abstract: In this paper, a steganographic algorithm in MPEG compressed video stream was proposed. In each GOP, the control information for to facilitate data extraction was embedded in I frame, in P frames and B frames, the actually transmitted data were repeatedly embedded in motion vectors of macro-blocks that have larger moving speed, for to resist video processing. Data extraction was also performed in compressed video stream without requiring original video. On a GOP by GOP basis, control information in I frame should be extracted firstly, then the embedded data in P and B frames can be extracted based on the control information. Experimental results show that the proposed algorithm has the characteristics of little degrading the visual effect, larger embedding capacity and resisting video processing such as frame adding or frame dropping.

Method and apparatus for video encoding and decoding using adaptive interpolation

Abstract: A method and apparatus is disclosed herein for video encoding and/or decoding using adaptive interpolation is described. In one embodiment, the decoding method comprises decoding a reference index; decoding a motion vector; selecting a reference frame according to the reference index; selecting a filter according to the reference index; and filtering a set of samples of the reference frame using the filter to obtain the predicted block, wherein the set of samples of the reference frame is determined by the motion vector.

Fast multiple reference frame motion estimation for H.264/AVC

Abstract: Multiple reference frame motion compensation is a new feature introduced in H.264/MPEG-4 AVC to improve video coding performance. However, the computational cost of multiple reference frame motion estimation (MRF-ME) is very high. In this paper, we propose an algorithm that takes into account the correlation/continuity of motion vectors among different reference frames. We show that the algorithm effectively reduces the computations of MRF-ME, and achieves similar coding gain compared to the motion search approaches in the reference software

Method and system for counting moving objects in a digital video stream

Abstract: A method and system are provided for counting moving objects in a digital video stream In contrast to known computationally-expensive methods, areas of motion are determined by threshold subtracting a current video frame from a short term average video scene An object box surrounding an object is determined by threshold subtracting the current video frame from a long term average video scene Coordinates of the moving object are identified by associating the area of motion with the object box, if it overlaps the area of motion, to define a moving object box An event counter can be incremented when the moving object box is in a buffer zone in the current frame, and was in a detection zone in an earlier frame, and was initially detected in a buffer zone on the opposite side of the detection zone

High dynamic range codecs

Abstract: A method for encoding high dynamic range (HDR) images involves providing a lower dynamic range (LDR) image, generating a prediction function for estimating the values for pixels in the HDR image based on the values of corresponding pixels in the LDR image, and obtaining a residual frame based on differences between the pixel values of the HDR image and estimated pixel values The LDR image, prediction function and residual frame can all be encoded in data from which either the LDR image of HDR image can be recreated

Method and device for efficient frame erasure concealment in speech codecs

Abstract: A method and device for concealing frame erasures caused by frames of an encoded sound signal erased during transmission from an encoder to a decoder and for recovery of the decoder after frame erasures comprise, in the encoder, determining concealment/recovery parameters including at least phase information related to frames of the encoded sound signal. The concealment/recovery parameters determined in the encoder are transmitted to the decoder and, in the decoder, frame erasure concealment is conducted in response to the received concealment/recovery parameters. The frame erasure concealment comprises resynchronizing, in response to the received phase information, the erasure-concealed frames with corresponding frames of the sound signal encoded at the encoder. When no concealment/recovery parameters are transmitted to the decoder, a phase information of each frame of the encoded sound signal that has been erased during transmission from the encoder to the decoder is estimated in the decoder. Also, frame erasure concealment is conducted in the decoder in response to the estimated phase information, wherein the frame erasure concealment comprises resynchronizing, in response to the estimated phase information, each erasure-concealed frame with a corresponding frame of the sound signal encoded at the encoder.

Overlay image contrast enhancement

Abstract: A method of creating an image difference overlay comprises identifying a loop of reference images of a subject and identifying a loop of data images of the subject. The loop of image data can be identified after an event, such as the administration of contrast agent to the subject. A reference loop image frame is compared to one or more data loop image frames and the reference loop frame is associated with a data loop image frame which closely resembles the data loop image frame. Each of the associated frames can then be processed and used to create an image difference overlay frame.

Device and method for fast block-matching motion estimation in video encoders

Abstract: Motion estimation is the science of predicting the current frame in a video sequence from the past frame (or frames), by slicing it into rectangular blocks of pixels, and matching these to past such blocks. The displacement in the spatial position of the block in the current frame with respect to the past frame is called the motion vector. This method of temporally decorrelating the video sequence by finding the best matching blocks from past reference frames—motion estimation—makes up about 80% or more of the computation in a video encoder. That is, it is enormously expensive, and methods do so that are efficient are in high demand. Thus the field of motion estimation within video coding is rich in the breadth and diversity of approaches that have been put forward. Yet it is often the simplest methods that are the most effective. So it is in this case. While it is well-known that a full search over all possible positions within a fixed window is an optimal method in terms of performance, it is generally prohibitive in computation. In this patent disclosure, we define an efficient, new method of searching only a very sparse subset of possible displacement positions (or motion vectors) among all possible ones, to see if we can get a good enough match, and terminate early. This set of sparse subset of motion vectors is preselected, using a priori knowledge and extensive testing on video sequences, so that these “predictors” for the motion vector are essentially magic. The art of this method is the preselection of excellent sparse subsets of vectors, the smart thresholds for acceptance or rejection, and even in the order of the testing prior to decision.

Extended range image processing for electro-optical systems

Abstract: A method and apparatus for processing imagery using images acquired via any known Electro-Optical (EO) system. In accordance with exemplary embodiments of the present invention, a first frame of data is selected as a template frame (e.g., a given frame). A second frame of data can be captured using the EO system. At least a portion of the second frame can be correlated with the template frame to generate a shift vector. The second frame can then be registered with the template frame by interpolating the second frame using the shift vector and re-sampling at least a portion of the second frame to produce a registered frame. The template frame can also be re-sampled. The registered frame and the re-sampled template frame can then be combined to generate an averaged frame. The averaged frame can be spatially filtered to enhance edges within the averaged frame.

Method and system for estimating motion and compensating for perceived motion blur in digital video

Abstract: A method for compensating for perceived blur due to motion between a current frame and a previous frame of a digital video sequence comprises estimating a motion vector between the frames for each of a plurality of pixel blocks in the current and previous frames. A cluster motion vector is then estimated for each of a plurality of clusters of the motion vectors based on one of vectors in each cluster and motion vectors in proximate clusters. The cluster motion vector of its corresponding cluster is allocated to each pixel in the current frame. An initial guess frame is generated based on the current frame and pixels in the guess frame are blurred as a function of their respective allocated cluster motion vectors. Each blurred pixel is compared with a respective pixel in the current frame to generate an error pixel for each respective pixel. Each error pixel is blurred and weighted and then each error pixel and its respective pixel is combined in the initial guess frame thereby to update the guess frame and compensate for blur. A system and computer program for perceived blur compensation is also provided.

Encoder assisted frame rate up conversion using various motion models

Abstract: An Encoder Assisted Frame Rate Up Conversion (EA-FRUC) system that utilizes various motion models, such as affine models, in addition to video coding and pre-processing operations at the video encoder to exploit the FRUC processing that will occur in the decoder in order to improve the modeling of moving objects, compression efficiency and reconstructed video quality. Furthermore, objects are identified in a way that reduces the amount of information necessary for encoding to render the objects on the decoder device.

An error concealment scheme for entire frame losses based on H.264/AVC

Abstract: In this paper, an error concealment algorithm is proposed to conceal an entirely lost frame in an H.264/AVC compressed video bitstream. For a lost frame, the algorithm reconstructs the lost motion information of this frame based on its temporal neighbors. The algorithm is block based and thus has low computation and implementation complexity. It is designed for those hard-to-conceal video sequences, which have abundant active and chaotic motions. Experimental results show that the algorithm compares favorably to other simple error concealment algorithms.

Spectral parameter substitution for the frame error concealment in a speech decoder

Abstract: A method for use by a speech decoder in handling bad frames received over a communications channel a method in which the effects of bad frames are concealed by replacing the values of the spectral parameters of the bad frames (a bad frame being either a corrupted frame or a lost frame) with values based on an at least partly adaptive mean of recently received good frames, but in case of a corrupted frame (as opposed to a lost frame), using the bad frame itself if the bad frame meets a predetermined criterion. The aim of concealment is to find the most suitable parameters for the bad frame so that subjective quality of the synthesized speech is as high as possible.

Electronic image sensor

Abstract: An electronic imaging sensor. The sensor includes an array of photo-sensing pixel elements for producing image frames. Each pixel element defines a photo-sensing region and includes a charge collecting element for collecting electrical charges produced in the photo-sensing region, and a charge storage element for the storage of the collected charges. The sensor also includes charge sensing elements for sensing the collected charges, and charge-to-signal conversion elements. The sensor also includes timing elements for controlling the pixel circuits to produce image frames at a predetermined normal frame rate based on a master clock signal (such as 12 MHz or 10 MHz). This predetermined normal frame rate which may be a video rate (such as about 30 frames per second or 25 frames per second) establishes a normal maximum per frame exposure time. The sensor includes circuits (based on prior art techniques) for adjusting the per frame exposure time (normally based on ambient light levels) and novel frame rate adjusting features for reducing the frame rate below the predetermined normal frame rate, without changing the master clock signal, to permit per frame exposure times above the normal maximum exposure time. This permits good exposures even in very low light levels. (There is an obvious compromise of lowering of the frame rate in conditions of very low light levels, but in most cases this is preferable to inadequate exposure.) These adjustments can be automatic or manual.

Transmission of stream video in low latency

Abstract: The present invention is a new and innovative system and method for the stream transmission of video data. The disclosed invention enables a significant reduction in latency and in start decoding and displaying parameters in the transmission of the video data in comparison with existing stream video transmission methods. According to the some embodiments of the present invention each frame or each field is sliced into a predetermined number of slices. Each slice is compressed separately, using standard video compression technologies. The compressed slices are then compounded together before being transmitted over the communication network. On the receiving end the frame is reassembled and decompressed. Finally the frame is recomposed accordingly and filtered to provide a high quality video image.

Frame rate up-conversion using perspective transform

Abstract: In this paper, we propose a frame rate up-conversion (FRUC) algorithm to increase the temporal resolution of video sequences at the decoder side. First, the proposed algorithm segments a frame into several objects. Then, perspective transforms are used to motion-compensate each object, while the translational block matching algorithm (BMA) is applied to the background. Furthermore, the overlapped block motion compensation (OBMC) technique is used to reduce blocking artifacts in boundary blocks. Experimental results show that the proposed algorithm provides better performance than the conventional approach.

Method of and apparatus for lossless video encoding and decoding

Abstract: Provided are a method of and apparatus for lossless video encoding and decoding, in which a differential residual block generated by calculating a difference between pixels of a residual block resulting from interprediction is encoded, thereby improving the compression rate. The method of lossless video encoding includes performing interprediction between a reference frame and a current frame in units of a predetermined-size block to generate a predicted block of a current block to be encoded, generating a residual block composed of residual signals corresponding to differences between pixels of the predicted block and the current block, calculating differences between the residual signals of the residual block in a predetermined direction and generating a differential residual block based on the calculated differences, and performing entropy-encoding on the differential residual block.

Method of increasing coding efficiency and reducing power consumption by on-line scene change detection while encoding inter-frame

Abstract: A system and method for on-the-fly detection of scene changes within a video stream through statistical analysis of a portion of the macroblocks comprising each video frame as they are processed using inter-frame coding. If the statistical analysis of the selected macroblocks of the current frame differs from the previous frame by exceeding predetermined thresholds, the current video frame is assumed to be a scene change. Once a scene change is detected, the remainder of the video frame is encoded as an intra-frame, intra-macroblocks, or intra slices, through implementation of one or more predetermined or adaptively adjusted quantization parameters to reduce computational complexity, decrease power consumption, and increase the resulting video image quality. As decoding is the inverse of encoding, these improvements are similarly recognized by a decoder as it decodes a resulting encoded video stream.

An Error Concealment Scheme for Entire Frame Losses for H.264/AVC

Abstract: In this paper, an error concealment scheme is proposed to conceal an entirely lost frame in a compressed video bitstream due to errors introduced during transmission. The proposed scheme targets low bit rate video transmission applications using H.264/AVC. The motion field of the lost frame is first reconstructed by copying the co-located motion vectors and reference indices from the last decoded reference frame. After the motion field estimation of the missing frame, motion compensation is performed to reconstruct the frame. This technique reuses existing modules of the video decoder and it does not incur extra complexity compared to decoding a normal frame. It has also been adopted as a non-normative decoder option to the JM reference software at the JVT meeting in Poznan, Poland in July 2005 [1] and has been incorporated into the SA4 video ad hoc group's toolkit at the 3GPP meeting at Paris [2] in September 2005. Simulation results will show its improved performance over other simple error concealment schemes such as "frame copy," both subjectively and objectively, without significant complexity overhead.

Encoder-assisted frame loss concealment techniques for audio coding

Abstract: Encoder-assisted frame loss concealment (FLC) techniques for decoding audio signals are described. A decoder may discard an erroneous frame of an audio signal and may implement the encoder-assisted FLC techniques in order to accurately conceal the discarded frame based on neighboring frames and side-information transmitted from the encoder. The encoder-assisted FLC techniques include estimating magnitudes of frequency-domain data for the frame based on frequency-domain data of neighboring frames, and estimating signs of the frequency-domain data based on a subset of signs transmitted from the encoder as side-information. Frequency-domain data for a frame of an audio signal includes tonal components and noise components. Signs estimated from a random signal may be substantially accurate for the noise components of the frequency-domain data. However, to achieve highly accurate sign estimation for the tonal components, the encoder transmits signs for the tonal components of the frequency-domain data as side-information.

Image processing apparatus and its method

Abstract: To obtain a dodging-like effect in moving-image data without expensive extraction of low-frequency components in playback, for a color-correction frame which does not refer to another frame, brightness components are extracted from the entire frame image to generate brightness-component and low-frequency brightness-component images, and these are used to color-correct the frame. For a color-correction frame which refers to another frame, a transition region of the frame image is detected, and used to identify an update image region in which a low-frequency brightness-component image must be updated. Brightness components of the transition region are extracted from the frame image, and are combined with those of a reference frame image, thus generating a brightness-component image. Low-frequency brightness components of the update region are extracted and combined with those of the reference image, thus generating a low-frequency brightness-component image of the frame. The color-correction frame image is color-corrected using the generated brightness-component and low-frequency brightness-component images.

Enhanced image/video quality through artifact evaluation

Abstract: In an image/video encoding and decoding system employing an artifact evaluator a method and/or apparatus to process video blocks comprising a decoder operable to synthesize an un-filtered reconstructed video block or frame and an artifact filter operable to receive the un-filtered reconstructed video block or frame, which generates a filtered reconstructed video block or frame. A memory buffer operable to store either the filtered reconstructed video block or frame or the un-filtered reconstructed video block or frame, and an artifact evaluator operable to update the memory buffer after evaluating and determining which of the filtered video block or frame, or the un-filtered video block or frame yields better image/video quality.

Nonlinear, in-the-loop, denoising filter for quantization noise removal for hybrid video compression

Abstract: A method and apparatus is disclosed herein for using an in-the-loop denoising filter for quantization noise removal for video compression. In one embodiment, the video encoder comprises a transform coder to apply a transform to a residual frame representing a difference between a current frame and a first prediction, the transform coder outputting a coded differential frame as an output of the video encoder; a transform decoder to generate a reconstructed residual frame in response to the coded differential frame; a first adder to create a reconstructed frame by adding the reconstructed residual frame to the first prediction; a non-linear denoising filter to filter the reconstructed frame by deriving expectations and performing denoising operations based on the expectations; and a prediction module to generate predictions, including the first prediction, based on previously decoded frames.

Connectivity-Guided Predictive Compression of Dynamic 3D Meshes

Abstract: We introduce an efficient algorithm for real-time compression of temporally consistent dynamic 3D meshes. The algorithm uses mesh connectivity to determine the order of compression of vertex locations within a frame. Compression is performed in a frame to frame fashion using only the last decoded frame and the partly decoded current frame for prediction. Following the predictive coding paradigm, local temporal and local spatial dependencies between vertex locations are exploited. In this framework we present a novel angle preserving predictor and evaluate its performance against other state of the art predictors. It is shown that the proposed algorithm improves up to 25% upon the current state of the art for compression of temporally consistent dynamic 3D meshes.

Method of multi-layer based scalable video encoding and decoding and apparatus for the same

Abstract: A method of multi-layer based scalable video encoding and decoding and an apparatus for the same are disclosed. The encoding method includes the steps of estimating motion between a base layer frame that is placed at a temporal location closest to a current frame of an enhancement layer, and a frame that is backwardly adjacent to the base layer frame to acquire a motion vector, generating a residual image by subtracting the backwardly adjacent frame from the base layer frame, generating a virtual forward reference frame using the motion vector, the residual image and the base layer frame, and generating a predicted frame with respect to the current frame using the virtual forward reference frame, and encoding the difference between the current frame and the predicted frame.

System, transmitter, receiver, method and software for transmitting and receiving ordered sets of video frames

Abstract: When distributing a set 101 of frames 102 each with encoded data 103 from a transmitter 104 to a receiver 105, a dependent frame 108 requires an independent frame 107 for decoding, and the transmitting 111 of the frames 102 from a buffer 110 at the transmitter 104 is reordered such that the dependent frame 108 is sent after the independent frame 107. In particular, a transmission policy wherein buffering 109 a further frame 125 takes into account both: if the further frame 125 requires zero, one or two frames 102 of the set 101 for decoding, and the current frames 102 buffered at the first and the second position 123 of the transmit buffer 110.

Method and system for motion compensated fine granularity scalable video coding with drift control

Abstract: An adaptively formed reference block is used for coding a block in a current frame in the enhancement layer, hi particular, the reference block is formed from a reference block in base layer reconstructed frame and a reference block in the enhancement layer reference frame together with a base layer reconstructed prediction residual block. Furthermore, the reference block for coding is adjusted depending on the transform coefficients of the base layer reconstructed residual layer. Moreover, the actual reference signal used for coding is a weighted average of a reference signal from the reconstructed frame in the base layer and a reference signal from the enhancement layer reference frame together with a base layer reconstruction prediction residual.

Method and apparatus for spatio-temporal deinterlacing aided by motion compensation for field-based video

Abstract: The invention comprises devices and methods for processing multimedia data to generate progressive frame data from interlaced frame data. In one aspect, a method of processing multimedia data includes generating spatio-temporal information for a selected frame of interlaced multimedia data, generating motion information for the selected frame, and deinterlacing fields of the selected frame based on the spatio-temporal information and the motion information to form a progressive frame associated with the selected frame. In another aspect an apparatus for processing multimedia data can include a spatial filter module configured to generate spatio-temporal information of a selected frame of interlaced multimedia data, a motion estimator configured to generate motion information for the selected frame, and a deinterlacer configured to deinterlace fields of the selected frame and form a progressive frame corresponding to the selected frame based on the spatio-temporal information and the motion information.

Method and apparatus of temporal error concealment for p-frame

Abstract: A method and apparatus of processing a multimedia frame comprising a first section received in error are disclosed. The method comprises obtaining temporal prediction data corresponding to a first reference frame. The method further comprises determining whether the first reference frame is reliable for estimating the first section. The method further comprises estimating the first section received in error based on one or more sections other than the first section of the multimedia frame and the first reference frame, if it is determined that the first reference frame is reliable.