Showing papers on "Inter frame published in 2010"

Weighted and controlled frames: Mutual relationship and first numerical properties

Abstract: Weighted and controlled frames have been introduced recently to improve the numerical efficiency of iterative algorithms for inverting the frame operator. In this paper, we develop systematically these notions, including their mutual relationship. We will show that controlled frames are equivalent to standard frames and so this concept gives a generalized way to check the frame condition, while offering a numerical advantage in the sense of preconditioning. Next, we investigate weighted frames, in particular their relation to controlled frames. We consider the special case of semi-normalized weights, where the concepts of weighted frames and standard frames are interchangeable. We also make the connection with frame multipliers. Finally, we analyze weighted frames numerically. First, we investigate three possibilities for finding weights in order to tighten a given frame, i.e. decrease the frame bound ratio. Then, we examine Gabor frames and how well the canonical dual of a weighted frame is approximated by the inversely weighted dual frame.

A Hybrid Frame Concealment Algorithm for H.264/AVC

Abstract: In packet-based video transmissions, packets loss due to channel errors may result in the loss of the whole video frame. Recently, many error concealment algorithms have been proposed in order to combat channel errors; however, most of the existing algorithms can only deal with the loss of macroblocks and are not able to conceal the whole missing frame. In order to resolve this problem, in this paper, we have proposed a new hybrid motion vector extrapolation (HMVE) algorithm to recover the whole missing frame, and it is able to provide more accurate estimation for the motion vectors of the missing frame than other conventional methods. Simulation results show that it is highly effective and significantly outperforms other existing frame recovery methods.

Video encoding using previously calculated motion information

Abstract: A video encoder uses previously calculated motion information for inter frame coding to achieve faster computation speed for video compression. In a multi bit rate application, motion information produced by motion estimation for inter frame coding of a compressed video bit stream at one bit rate is passed on to a subsequent encoding of the video at a lower bit rate. The video encoder chooses to use the previously calculated motion information for inter frame coding at the lower bit rate if the video resolution is unchanged. A multi core motion information pre-calculation produces motion information prior to encoding by dividing motion estimation of each inter frame to separate CPU cores.

SURE-LET for Orthonormal Wavelet-Domain Video Denoising

Abstract: We propose an efficient orthonormal wavelet-domain video denoising algorithm based on an appropriate integration of motion compensation into an adapted version of our recently devised Stein's unbiased risk estimator-linear expansion of thresholds (SURE-LET) approach. To take full advantage of the strong spatio-temporal correlations of neighboring frames, a global motion compensation followed by a selective block-matching is first applied to adjacent frames, which increases their temporal correlations without distorting the interframe noise statistics. Then, a multiframe interscale wavelet thresholding is performed to denoise the current central frame. The simulations we made on standard grayscale video sequences for various noise levels demonstrate the efficiency of the proposed solution in reducing additive white Gaussian noise. Obtained at a lighter computational load, our results are even competitive with most state-of-the-art redundant wavelet-based techniques. By using a cycle-spinning strategy, our algorithm is in fact able to outperform these methods.

Motion-Compensated Frame Rate Up-Conversion—Part I: Fast Multi-Frame Motion Estimation

Abstract: Motion-compensated frame rate up-conversion is used to convert video/film materials of low frame rates to a higher frame rate so that the materials can be displayed with smooth motion and high-perceived quality. It consists of two key elements: motion estimation and motion-compensated frame interpolation. It requires accurate motion trajectories to ensure quality results and low computational cost to ensure practical applications. This paper presents a novel motion estimation algorithm that combines the accuracy of maximum a posteriori probability (MAP) estimation with the speed of hierarchical block-matching algorithm (BMA). This MAP estimation uses three consecutive pictures, instead of the conventional two, and one previously estimated motion field to exploit the temporal correlation between motion fields and to determine motion in occluded areas. The optimization of the MAP estimation is performed using full-search and implemented by means of look-up tables. The full search ensures that the optimization converges to the global minimum, while the look-up tables dramatically reduce the computational cost. Experimental results show that the proposed algorithm provides motion trajectories that are much more accurate than those obtained using either the full-search BMA or hierarchical BMA alone. Also, it is much faster than the full-search BMA.

A new interframe difference algorithm for moving target detection

Abstract: In this paper, a new interframe difference algorithm for moving target detection is proposed which is under a static background based on three-frame-difference method in combination with background subtraction method. Firstly, the current frame image subtracts the previous frame and the next frame image separately, their results are added together to get a gray-scale image of the three-frame-difference method. Secondly, the current frame image subtracts the background image to get another gray-scale image of background subtraction method. Thirdly, their sum of the two gray-scale images of above is translated into binary image after being judged by threshold. Finally, this binary image is processed by morphology filtering and connectivity analyzing. Therefore, moving region is obtained. This new algorithm takes advantage of the good performances of three-frame-difference method and background subtraction method adequately. The analysis in theory and experiment results all show that the algorithm is better in efficiency and effect for moving target detection compared to the other similarity method.

2D to 3D video conversion

Abstract: A method for real-time 2D to 3D video conversion includes receiving a decoded 2D video frame having an original resolution, downscaling the decoded 2D video frame into an associated 2D video frame having a lower resolution, and segmenting objects present in the downscaled 2D video frame into background objects and foreground objects. The method also includes generating a background depth map and a foreground depth map for the downscaled 2D video frame based on the segmented background and foreground objects, and deriving a frame depth map in the original resolution based on the background depth map and the foreground depth map. The method further includes providing a 3D video frame for display at a real-time playback rate. The 3D video frame is generated in the original resolution based on the frame depth map.

Method and apparatus for image stabilization

Abstract: A method and device are provided for method for stabilization of image data by an imaging device. In one embodiment, a method includes detecting image data for a first frame and a second frame, performing motion estimation to determine one or more motion vectors associated with global frame motion for image data of the first frame, performing an outlier rejection function to select at least one of the one or more motion vectors, and determining a global transformation for image data of the first frame based, at least in part, on motion vectors selected by the outlier rejection function. The method may further include determining a stabilization transformation for image data of the first frame by refining the global transformation to correct for unintentional motion and applying the stabilization transformation to image data of the first frame to stabilize the image data of the first frame.

Video coding using the most common frame in scene

Abstract: Motion estimation (ME) and motion compensation (MC) using variable block size, fractional search, and multiple reference frames (MRFs) help the recent video coding standard H.264 to improve the coding performance significantly over the other contemporary coding standards. The concept of MRF achieves better coding performance in the cases of repetitive motion, uncovered background, non-integer pixel displacement, lighting change, etc. The requirement of index codes of the reference frames, computational time in ME&MC, and memory buffer for pre-coded frames limits the number of reference frames used in practical applications. In typical video sequence, the previous frame is used as a reference frame with 68∼92% of cases. In this paper, we propose a new video coding method using a reference frame (i.e., the most common frame in scene (McFIS)) generated by the Gaussian mixture based dynamic background modelling. The McFIS is not only more effective in terms of rate-distortion and computational time performance compared to the MRFs but also error resilient transmission channel. The experimental results show that the proposed coding scheme outperforms the H.264 standard video coding with five reference frames by at least 0.5 dB and reduced 60% of computation time.

Method and apparatus for encoding/decoding images using adaptive motion vector resolution

Abstract: The present disclosure relates to a method and apparatus for improving the encoding efficiency by adaptively changing the resolution of the motion vector in the inter prediction encoding and inter prediction decoding of a video. The apparatus includes: a block identification unit for identifying a colocated block included in a reference picture as a block located at a position equal to a position of a current block; a moving block determiner for determining if the current block is a moving block, based on a motion vector of the colocated block; a motion vector determiner for determining a motion vector of the current block according to a result of the determining of if the current block is a moving block; and a resolution converter for converting a resolution of the motion vector of the colocated block.

Systems and methods for preventing the loss of information within a speech frame

Abstract: A method for preventing the loss of information within a speech frame is described. A first speech frame to be encoded is selected. A determination is made as to whether or not a second speech frame is a critical speech frame based on the information within the second speech frame and one or more adjacent speech frames. At least a part of an encoded version of the second speech frame is created according to a selected forward error correction (FEC) mode if the second speech frame is a critical speech frame. The first speech frame and the at least a part of the encoded version of the second speech frame are transmitted.

QoE as a function of frame rate and resolution changes

Abstract: Video bit rate reduction can be very important for all video streaming application. One of the possible ways to reduce bit rate is decreasing change in time or space domain i.e. changing frame rate or resolution. In this paper we present two no reference metrics mapping frame rate or resolution into MOS. Both models use simple to calculate parameters expressed by sequence spatial and temporal information. The models were estimated and verified upon distinctive video sequence sets. The considered frame rate change varies from 5 to 30 frames per second. The considered resolutions changes from SQCIF to SD.

Video matting based on foreground-background constraint propagation

Abstract: A method for propagating user-provided foreground-background constraint information for a first video frame to subsequent frames allows extraction of moving foreground objects with minimal user interaction. Video matting is performed wherein constraints derived from user input with respect to a first frame are propagated to subsequent frames using the estimated alpha matte of each frame. The matte of a frame is processed in order to arrive at a rough foreground-background segmentation which is then used for estimating the matte of the next frame. At each frame, the propagated constraints are used by an image matting method for estimating the corresponding matte which is in turn used for propagating the constraints to the next frame, and so on.

Temporal error concealment for video communications

Abstract: Methods and systems for processing video data are described. A set of candidate motion vectors is selected from motion vectors associated with macroblocks in a first frame of video data and from motion vectors associated with macroblocks in a second frame of the video data. A statistical measure of the set is determined. The statistical measure defines a motion vector for a macroblock of interest in the second frame.

Method and Apparatus for Encoding Surveillance Video

Abstract: Methods and apparatus are provided for encoding video, such as surveillance video. A video frame is encoded by evaluating a level of activity in video frame relative to one or more neighboring frames; encoding the video frame using an inter-coding technique if the evaluation satisfies one or more predefined inter-coding criteria, wherein said inter-coding technique is applied to said video frame without a Group of Picture structure having a maximum number of consecutive inter-coded frames; and encoding the video frame using an intra-coding technique if the evaluation satisfies one or more predefined intra-coding criteria. The evaluating step may comprise, for example, determining whether differences between the video frame and the one or more neighboring frames exceed a predefined threshold.

Object tracking using momentum and acceleration vectors in a motion estimation system

Abstract: There is provided a method and apparatus for motion estimation in a sequence of video images. The method comprises a) subdividing each field or frame of a sequence of video images into a plurality of blocks, b) assigning to each block in each video field or frame a respective set of candidate motion vectors, c) determining for each block in a current video field or frame, which of its respective candidate motion vectors produces a best match to a block in a previous video field or frame, d) forming a motion vector field for the current video field or frame using the thus determined best match vectors for each block, and e) forming a further motion vector field by storing a candidate motion vector derived from the best match vector at a block location offset by a distance derived from the candidate motion vector. Finally, steps a) to e) are repeated for a video field or frame following the current video field or frame. The set of candidate motion vectors assigned at step b) to a block in the following video field or frame includes the candidates stored at that block location at step e) during the current video field or frame The method enables a block or tile based motion estimator to improve its accuracy by introducing true motion vector candidates derived from the physical behaviour of real world objects.

Key frames extraction for video content analysis

Abstract: A method of extracting a key frame from a sequence of frames constituting a shot, each frame being constituted by a matrix of pixels, comprises: for each frame of the sequence of frames: computing (3) the optical flow of the frame compared to the following frame as a matrix of displacement of each pixel from the frame to the following frame; computing (5) a motion entropy measure based on the optical flow of the frame; selecting (7) as key frame the frame of the sequence of frames having the maximum motion entropy measure.

Virtual Focus and Depth Estimation From Defocused Video Sequences

Abstract: In this paper, we present a novel method for virtual focus and object depth estimation from defocused video captured by a moving camera. We use the term virtual focus to refer to a new approach for producing in-focus image sequences by processing blurred videos captured by out-of-focus cameras. Our method relies on the concept of Depth-from-Defocus (DFD) for virtual focus estimation. However, the proposed approach overcomes limitations of DFD by reformulating the problem in a moving-camera scenario. We introduce the interframe image motion model, from which the relationship between the camera motion and blur characteristics can be formed. This relationship subsequently leads to a new method for blur estimation. We finally rely on the blur estimation to develop the proposed technique for object depth estimation and focused video reconstruction. The proposed approach can be utilized to correct out-of-focus video sequences and can potentially replace the expensive apparatus required for auto-focus adjustments currently employed in many camera devices. The performance of the proposed algorithm is demonstrated through error analysis and computer simulated experiments.

Digital Video Tamper Detection Based on Multimodal Fusion of Residue Features

Abstract: In this paper, we propose novel algorithmic models based on feature transformation in cross-modal subspace and their multimodal fusion for different types of residue features extracted from several intra-frame and inter frame pixel sub-blocks in video sequences for detecting digital video tampering or forgery. An evaluation of proposed residue features – the noise residue features and the quantization features, their transformation in cross-modal subspace, and their multimodal fusion, for emulated copy-move tamper scenario shows a significant improvement in tamper detection accuracy as compared to single mode features without transformation in cross-modal subspace.

Successive refinement based Wyner-Ziv video compression

Abstract: Wyner-Ziv coding enables low complexity video encoding with the motion estimation procedure shifted to the decoder. However, the accuracy of decoder motion estimation is often low, due to the absence of the input source frame (at the decoder). In this paper, we propose a novel Wyner-Ziv successive refinement approach to improve the motion compensation accuracy and the overall compression efficiency of Wyner-Ziv video coding. Our approach encodes each frame by multiple Wyner-Ziv coding layers and uses the progressively refined reconstruction frame to guide the motion estimation for progressively improved accuracy. The proposed approach yields competitive results against state-of-the-art low complexity Wyner-Ziv video coding approaches, and can gain up to 3.8dB over the conventional Wyner-Ziv video coding approach and up to 1.5dB over the previous bitplane-based refinement approach. Furthermore, this paper also presents the rate distortion analysis and the performance comparison of the proposed approach and conventional approaches. The rate distortion performance loss (due to performing decoder motion estimation) is at most 2.17dB (or equivalently 14nats/pixel) in our scheme according to our analysis, but can be more than 6dB in the conventional approach according to previous research. For the simplified two-layers case of our approach, we derive the optimal subsampling ratio in the sense of rate distortion performance. We also extend our analysis and conclusions from P frame to B frame. Finally, we verify our analysis by experimental results.

Methods and apparatus for reducing structured noise in video

Abstract: A method of operating a computer system to reduce structured noise in a video is described. Patches in a frame of a sequence of video frames are selected using a modified non-local means (NLM) search algorithm with a reduced search complexity compared to conventional NLM search algorithms. The set of spatial neighboring patches in a frame is extended to include patches in adjacent frames of the video sequence. Patches in the adjacent frames are selected based, at least in part, on a projection of a location of a pixel in the frame into the adjacent frames using a motion estimation algorithm. An average value determined from the combined set of patches is used to replace the pixel currently being processed.

Method and device for decoding/coding of video signal

Abstract: FIELD: information technologies. ^ SUBSTANCE: method for decoding of video signal includes stages of video signal coding scheme, reception of configuration information for video signal according to coding scheme, recognition of total number of frames, using information of configuration, recognition of interframe reference information on the basis of total number of frames, and decoding of video signal on the basis of interframe reference information of frame, besides configuration information includes at least frame information for identification of video signal frame. ^ EFFECT: improved efficiency of coding, decoding of video image by means of video signal coding on the basis of interframe reference information. ^ 8 cl, 49 dwg

Device, system, and method for predicting residual data for intra and inter frame encoding of image or video data

Abstract: A system, processor, and method are provided for encoding a data block, for example, of digital data. A processor may, from among a plurality of intra frame encoding modes each having a different direction for extrapolating already encoded pixels adjacent to the block, select an intra coding mode having a direction that most closely matches a direction of minimum pixel value change of the block. The processor may compute a predicted intra frame encoding residual data for the block associated with the selected mode based on the difference between the direction of the selected intra frame encoding mode and the direction of minimum pixel value change of the block. The processor may compute inter frame encoding residual data and compare the intra and inter frame encoding residual data. The processor may compress the data block using the intra or inter frame encoding having the smaller residual data.

Method And System For Adaptive Rate Video Compression And Transmission

Abstract: For a portion of video data, a video processing device generates a plurality of compressed video bit streams comprising different bit rates, and selects one stream per portion of video data to transmit to a display device. The streams may be generated concurrently. Bit rates are determined based on operating conditions and link performance. The selections are determined dynamically in response to received information. The streams may be compressed utilizing different compression schemes, for example, DPCM and/or transform compression. The bit rates may be dynamically adjusted by adjusting compression scheme, compression parameters, frame size, frame rate, color space and/or chroma sampling. The portion of video data may comprise contiguous horizontal rows or portions of rows of pixels, a full frame and/or a scaled frame. Information is communicated to the display device for control of decompression operations. The display device receives, decompresses, reconstructs and displays the selected bit stream.

Fast motion estimation methods using multiple reference frames

Abstract: Motion estimation methods using multiple reference frames are provided. In the methods, motion estimation is performed on a current block using a single reference frame selected from the multiple reference frames or using the multiple reference frames in order of probability that each reference frame is an optimal reference frame for the current block. Accordingly, faster motion estimation is performed.

Video cut detection using block based histogram differences in RGB color space

Abstract: Shot boundary detection algorithm has the longest and richest history in the area of content based video analysis and retrieval. Various automatic shot boundary detection techniques have been proposed and their performances are reliable, especially for video cut detection. This paper approaches the video cut transition detection based on the block wise histogram differences of the two consecutive frames of a video sequence in RGB color space. Most of the cut identification techniques uses a threshold to discriminate between the inter frame difference values and thus identify the video breakpoints. A new threshold calculation algorithm is proposed for cut identification process. Experimental results show that the proposed method gives better results than the existing methods.

An Efficient Priority-Based Reference Frame Selection Method for Fast Motion Estimation in H.264/AVC

Abstract: The H.264/AVC achieves higher coding performance than the previous video coding standards, such as MPEG-2 and MPEG-4. In order to achieve this improved coding performance, H.264 adopts various advanced techniques including multiple reference frame motion estimation (MRF-ME), which requires very heavy encoding complexity in proportion to the number of reference frames. In this letter, we propose an efficient priority-based reference frame selection (PRFS) method to reduce the computational complexity. The PRFS method computes the priorities of all reference frames using spatial and temporal correlation of the reference frame index and motion vectors, and selects candidate reference frames to reduce the number of reference frames for each ME block. Experimental results show that the proposed method reduces the computational complexity of the MRF-ME and achieves similar coding performance to that of previous methods.

A reformative frame layer rate control algorithm for H.264

Abstract: Rate control is an important part of video coding. There are two disadvantages in frame layer rate control of JVT-G012 proposal for H.264 video coding standard. One is that the bit allocation in frame layer is uniform and another is that it does not take the encoded frame information into account. This paper presents a reformative frame layer rate control algorithm by combining frame complexity and adjusting quantization parameter (QP). The target bits of each frame are allocated based on its complexity coefficient and then its quantization parameter is adjusted by encoded frame information. Experimental results show that the proposed algorithm, in comparison to the original algorithm, reduced the actual bitrate error of video sequences by almost one times, improved average peak signal noise ratio (PSNR) by 0.64 dB and got a smaller PSNR deviation by 29.32%. The algorithm can not only control bitrate more accurately, but also get video image quality better and smoother with lower coding buffer occupancy.

Motion Vector Based Features for Content Based Video Copy Detection

Abstract: In this article, we propose a motion vector based feature set for Content Based Copy Detection (CBCD) of video clips. Motion vectors of image frames are one of the signatures of a given video. However, they are not descriptive enough when consecutive image frames are used because most vectors are too small. To overcome this problem we calculate motion vectors in a lower frame rate than the actual frame rate of the video. As a result we obtain longer vectors which form a robust parameter set representing a given video. Experimental results are presented.