Showing papers on "Inter frame published in 2013"

LightSync: unsynchronized visual communication over screen-camera links

Abstract: A key challenge for smartphone based visual communication over screen-camera links is imperfect frame synchronization. The difficulty arises from frame rate diversity and variability due to camera capability, lighting conditions, and system factors. On the 4 smartphone cameras we tested, the frame rate varies between 8 and 30 frames per second. If the transmit frame rate is too high, the receiver might lose original frames or capture mixed frames, which are normally not decodable. Previous systems simply reduce the effective screen frame rate to be half the camera frame capture rate, to guarantee receiving a decodable frame every other frame. This under-utilizes the transmitter side capacity and is inefficient.We achieve frame synchronization with LightSync, which features in-frame color tracking to decode imperfect frames and a linear erasure code across frames to recover lost frames. LightSync allows smooth communication between the screen and the camera at any combination of the transmit and receive frame rates, as long as the receive rate is at least half the transmit rate. This means that each receiver can scale the decoding performance with its own camera capability. Across several phones, our system can more than double the average throughput compared to previous approaches.

Motion-Compensated Frame Interpolation Based on Multihypothesis Motion Estimation and Texture Optimization

Abstract: A novel motion-compensated frame interpolation (MCFI) algorithm to increase video temporal resolutions based on multihypothesis motion estimation and texture optimization is proposed in this paper. Initially, we form multiple motion hypotheses for each pixel by employing different motion estimation parameters, i.e., different block sizes and directions. Then, we determine the best motion hypothesis for each pixel by solving a labeling problem and optimizing the parameters. In the labeling problem, the cost function is composed of color, shape, and smoothness terms. Finally, we refine the motion hypothesis field based on the texture optimization technique and blend multiple source pixels to interpolate each pixel in the intermediate frame. Simulation results demonstrate that the proposed algorithm provides significantly better MCFI performance than conventional algorithms.

Motion-Aware Decoding of Compressed-Sensed Video

Abstract: Compressed sensing is the theory and practice of sub-Nyquist sampling of sparse signals of interest. Perfect reconstruction may then be possible with much fewer than the Nyquist required number of data. In this paper, in particular, we consider a video system where acquisition is carried out in the form of direct compressive sampling (CS) with no other form of sophisticated encoding. Therefore, the burden of quality video sequence reconstruction falls solely on the receiver side. We show that effective implicit motion estimation and decoding can be carried out at the receiver or decoder side via sparsity-aware recovery. The receiver performs sliding-window interframe decoding that adaptively estimates Karhunen-Loeve bases from adjacent previously reconstructed frames to enhance the sparse representation of each video frame block, such that the overall reconstruction quality is improved at any given fixed CS rate. Experimental results included in this paper illustrate the presented developments.

Novel True-Motion Estimation Algorithm and Its Application to Motion-Compensated Temporal Frame Interpolation

Abstract: In this paper, a new low-complexity true-motion estimation (TME) algorithm is proposed for video processing applications, such as motion-compensated temporal frame interpolation (MCTFI) or motion-compensated frame rate up-conversion (MCFRUC). Regular motion estimation, which is often used in video coding, aims to find the motion vectors (MVs) to reduce the temporal redundancy, whereas TME aims to track the projected object motion as closely as possible. TME is obtained by imposing implicit and/or explicit smoothness constraints on the block-matching algorithm. To produce better quality-interpolated frames, the dense motion field at interpolation time is obtained for both forward and backward MVs; then, bidirectional motion compensation using forward and backward MVs is applied by mixing both elegantly. Finally, the performance of the proposed algorithm for MCTFI is demonstrated against recently proposed methods and smoothness constraint optical flow employed by a professional video production suite. Experimental results show that the quality of the interpolated frames using the proposed method is better when compared with the MCFRUC techniques.

Video rate control for video coding standards

Abstract: A method and device for improving rate controlling in video coding of sequences including a series of Inter frames separated by Intra frames, when a decoding delay is considered, comprise for each Inter frame of the series: computing a target frame size, computing a maximum buffer level related to a position of each Inter frame relative to a previous Intra frame and an upcoming Intra frame, and optimizing a transmission buffer level in response to the computed target frame size and the computed maximum buffer level.

Video codec architecture for next generation video

Abstract: A system, apparatus, and method related to content adaptive partitioning for prediction and coding for next generation video (NGV) coding are described. NGV coding differs from standards-based approaches as it naturally incorporates significant content-based adaptivity in video coding process to achieve higher compression. On the other hand, NGV exploits interframe differences due to motion and other types of interframe difference (gain, blur, registration) that naturally exist in typical video scenes.

High-efficiency video coding inter-frame mode judging method based on temporal relativity

Abstract: The invention discloses a high-efficiency video coding (HEVC) inter-frame mode judging method based on temporal relativity, which comprises the steps of forecast method configuration and forecast mode selection. The forecast mode selection adopts the temporal relativity between two adjacent frames; the similarity of PU (Physical Unit) modes of a large size CU (Control Unit) of a corresponding piece in the previous frame and a small size CU in the current piece is analyzed according to the relativity; and finally, a PU mode selection method of the current CUs in various dimension is designed for the corresponding pieces in various dimensions according to the similarity. Compared with the HEVC standard in the prior art, with the adoption of the HEVC inter-frame mode judging method, the complexity of codding computation is reduced to a greater extent on the premise that the bit rate and video quality are almost unchanged.

Video shot detection method based on histogram improvement and clustering algorithm

Abstract: The invention discloses a video shot detection method based on histogram improvement and a clustering algorithm, and relates to image processing techniques. According to the method, the histogram improvement and the clustering algorithm are adopted to compute an intersection of histograms of two adjacent frames of images, and whether a shot change happens is judged according to histogram similarity; if the shot change happens, secondary detection on a shot boundary of the intersection of the histograms of the two adjacent frames of images is carried out by utilizing interframe gray scale/color difference values, pixel difference value computation on each block is carried out respectively by adopting non-uniform block weighting processing, pixel difference values and preset block frame differential threshold values are compared to obtain identification variables, the identification variable of each block is weighted and summarized, and the weighing and summarizing values and the preset block differential threshold values are compared to carry out shot detection. The video shot detection method improves shot detection accuracy and solves the problems of shot false detection, frame number discontinuity, and the like.

Quick detecting method for moving objects in dynamic scene

Abstract: Provided is a quick detecting method for moving objects in a dynamic scene. The quick detecting method for the moving objects in the dynamic scene comprises carrying out sequence interframe registration on moving images by utilizing CenSurE feature points and a homography transformation model, obtaining a registering frame of a former frame taking a current frame as reference, carrying out subtraction on the registering frame with the current frame to obtain a frame difference image to generate a foreground mask, building a dynamic background updated in real time according to space distribution information of the foreground mask in the current frame, obtaining a background subtraction image based on a background subtraction method, carrying out statistics on the probability density of the gray level of each pixel in the frame difference image, when the sum of the probability density of the gray level of a pixel is larger than 2phi(k)-1, taking the gray level as a self-adaptation threshold value, judging pixels with values of gray levels larger than the threshold value as foreground pixels, and otherwise judging the pixels as background pixels. The quick detecting method for the moving objects in the dynamic scene can reach the processing speed of 15frame/s and can obtain relatively integral moving objects under the premise that the detecting speed is ensured, and therefore, index requirements such as rapidity, noise immunity, illumination adaptation, target integrity and the like of the detection of the moving objects in the dynamic scene can be met.

Video coding with dynamic background

Abstract: Motion estimation (ME) and motion compensation (MC) using variable block size, sub-pixel search, and multiple reference frames (MRFs) are the major reasons for improved coding performance of the H.264 video coding standard over other contemporary coding standards. The concept of MRFs is suitable for repetitive motion, uncovered background, non-integer pixel displacement, lighting change, etc. The requirement of index codes of the reference frames, computational time in ME & MC, and memory buffer for coded frames limits the number of reference frames used in practical applications. In typical video sequences, the previous frame is used as a reference frame with 68–92% of cases. In this article, we propose a new video coding method using a reference frame [i.e., the most common frame in scene (McFIS)] generated by dynamic background modeling. McFIS is more effective in terms of rate-distortion and computational time performance compared to the MRFs techniques. It has also inherent capability of scene change detection (SCD) for adaptive group of picture (GOP) size determination. As a result, we integrate SCD (for GOP determination) with reference frame generation. The experimental results show that the proposed coding scheme outperforms the H.264 video coding with five reference frames and the two relevant state-of-the-art algorithms by 0.5–2.0 dB with less computational time.

Block Matching Algorithms for motion estimation

Abstract: The Block Matching is a temporal compression technique used in the video encoding. The main purpose of this method is to determine the displacements of each block of pixels between two successive frames. This technique, performed in the step of motion estimation, occupies the majority of the total time of video coding. The aim of this work is to give a comparative study various search algorithms by Block Matching. This study does not focus only on the complexity and computation time of each algorithm, but it also gives objective and subjective quality assessment of decoded video frame by means of each search algorithm.

Bandwidth reduction in video coding through applying the same reference index

Abstract: Techniques for encoding and decoding video data are described. A method of coding video may include determining a plurality of motion vector candidates for a block of video data for use in a motion vector prediction process, wherein each of the motion vector candidates points to a respective reference frame index, performing the motion vector prediction process using the motion vector candidates to determine a motion vector for the block of video data, and performing motion compensation for the block of video data using the motion vector and a common reference frame index, wherein the common reference frame index is used regardless of the respective reference frame index associated with the determined motion vector.

Intra-and-Inter-Constraint-Based Video Enhancement Based on Piecewise Tone Mapping

Abstract: Video enhancement plays an important role in various video applications. In this paper, we propose a new intra-and-inter-constraint-based video enhancement approach aiming to: 1) achieve high intraframe quality of the entire picture where multiple regions-of-interest (ROIs) can be adaptively and simultaneously enhanced, and 2) guarantee the interframe quality consistencies among video frames. We first analyze features from different ROIs and create a piecewise tone mapping curve for the entire frame such that the intraframe quality can be enhanced. We further introduce new interframe constraints to improve the temporal quality consistency. Experimental results show that the proposed algorithm obviously outperforms the state-of-the-art algorithms.

Method for quick inter-frame transcoding from H. 264/AVC standard to HEVC standard and transcoder thereof

Abstract: The invention discloses a method for quick inter-frame transcoding from H. 264/AVC standard to HEVC standard. The method comprises the following steps: imbedding an H.264/AVC standard code stream into a decoder for decoding, abstracting various information of subblocks during a decoding process, converting extracted information of the subblocks into CU and PU information for HEVC standard in ways of direct correspondence and small blocks forming large blocks, and finally using CU and PU information in HEVC standard coding directly. Since definite CU and PU information is available, quadtree division of CU, performed according to original encoding mode, is not required, ergodic of all possible PU modes under the depth of each CU is not required, and all that is needed is to calculate a PU mode and an intra-frame predicting mode under the depth of a corresponding CU. Therefore, calculating complexity in quadtree division of CU and ergodic under PU mode during transcoding is avoided. According to the transcoding method in the invention, the coding calculation complexity and coding time can be lowered and reduced considerably under the condition of very small loss of bit rate and video quality.

Compressible Motion Fields

Abstract: Traditional video compression methods obtain a compact representation for image frames by computing coarse motion fields defined on patches of pixels called blocks, in order to compensate for the motion in the scene across frames. This piecewise constant approximation makes the motion field efficiently encodable, but it introduces block artifacts in the warped image frame. In this paper, we address the problem of estimating dense motion fields that, while accurately predicting one frame from a given reference frame by warping it with the field, are also compressible. We introduce a representation for motion fields based on wavelet bases, and approximate the compressibility of their coefficients with a piecewise smooth surrogate function that yields an objective function similar to classical optical flow formulations. We then show how to quantize and encode such coefficients with adaptive precision. We demonstrate the effectiveness of our approach by comparing its performance with a state-of-the-art wavelet video encoder. Experimental results on a number of standard flow and video datasets reveal that our method significantly outperforms both block-based and optical-flow-based motion compensation algorithms.

Color Video Denoising Based on Combined Interframe and Intercolor Prediction

Abstract: An advanced color video denoising scheme which we call CIFIC based on combined interframe and intercolor prediction is proposed in this paper. CIFIC performs the denoising filtering in the RGB color space, and exploits both the interframe and intercolor correlation in color video signal directly by forming multiple predictors for each color component using all three color components in the current frame as well as the motion-compensated neighboring reference frames. The temporal correspondence is established through the joint-RGB motion estimation (ME) which acquires a single motion trajectory for the red, green, and blue components. Then the current noisy observation as well as the interframe and intercolor predictors are combined by a linear minimum mean squared error (LMMSE) filter to obtain the denoised estimate for every color component. The ill condition in the weight determination of the LMMSE filter is detected and remedied by gradually removing the “least contributing” predictor. Furthermore, our previous work on the LMMSE filter applied in the adaptive luminance-chrominance space (LAYUV for short) is revisited. By reformulating LAYUV and comparing it with CIFIC, we deduce that LAYUV is a restricted version of CIFIC, and thus CIFIC can theoretically achieve lower denoising error. Experimental results verify the improvement brought by the joint-RGB ME and the integration of the intercolor prediction, as well as the superiority of CIFIC over LAYUV. Meanwhile, when compared with other state-of-the-art algorithms, CIFIC provides competitive performance both in terms of the color peak signal-to-noise ratio and in perceptual quality.

Motion-based object segmentation using hysteresis and bidirectional inter-frame change detection in sequences with moving camera

Abstract: We present an unsupervised motion-based object segmentation algorithm for video sequences with moving camera, employing bidirectional inter-frame change detection. For every frame, two error frames are generated using motion compensation. They are combined and a segmentation algorithm based on thresholding is applied. We employ a simple and effective error fusion scheme and consider spatial error localization in the thresholding step. We find the optimal weights for the weighted mean thresholding algorithm that enables unsupervised robust moving object segmentation. Further, a post processing step for improving the temporal consistency of the segmentation masks is incorporated and thus we achieve improved performance compared to the previously proposed methods. The experimental evaluation and comparison with other methods demonstrate the validity of the proposed method.

Compression of 3D mesh sequences by temporal segmentation

Abstract: We describe a compression method for three-dimensional animation sequences that has notable advantages over existing techniques. We first aggregate the frame data by similarity and reorganize them into clusters, which results in the sequence split into several motion fragments of varying lengths. To minimize the number of clusters and obtain optimal clustering, we perform frame alignment, which eliminates the “global” rigid transformation from each frame data and use only “pose” when evaluating the similarity between frames. We then apply principal component analysis for each cluster, from which we get coordinates of corresponding frames in a reduced dimension. Because similar frames are considered, the number of coefficients required for each frame becomes smaller; thus, we obtain better dimension reduction for a given reconstruction error. Further, we perform intracluster compression based on linear coding. Because every motion fragment presents similar frames, conventional linear predictive coding can be replaced by key frame-based linear coding to achieve minimal reconstruction error. Results show that our method can obtain a high compression ratio, with a limited reconstruction error. Copyright © 2013 John Wiley & Sons, Ltd.

Spatio-Temporal Iterative Source–Channel Decoding Aided Video Transmission

Abstract: Low-complexity uncompressed video transmission meets the requirements of home networking and quality/delay-sensitive medical applications. Hence, it has attracted research attention in recent years. The redundancy inherent in the uncompressed video signals may be exploited by joint source–channel decoding to improve the attainable error resilience. Hence, in this treatise, we study the application of iterative joint source–channel decoding aided uncompressed video transmission, where correlation inherent in the video signals is modeled by a first-order Markov process. First, we propose a spatiotemporal joint source–channel decoding system using a recursive systematic convolutional codec, where both the horizontal and vertical intraframe correlations, as well as the interframe correlations, are exploited by the receiver, hence relying on 3-D information exchange. This scheme may be combined with arbitrary channel codecs. Then, we analyze the three-stage decoder's convergence behavior using 3-D extrinsic information transfer (EXIT) charts. Finally, we benchmark the attainable system performance against a couple of video communication systems, including our previously proposed 2-D scheme, where only intraframe correlations were exploited without invoking a channel codec. Our simulation results show that substantial $E_{b}/N_{0}$ improvements are attainable by the proposed technique.

Adaptive up-sampling filter for scalable video coding

Abstract: Methods and systems for video image coding are provided. Sets of filters may be selected and applied to video information at least partially based on the type of inter layer prediction implemented in coding the video information. Different filters, or filter sets, may be used for inter layer intra prediction, difference domain intra prediction, and/or difference domain inter prediction. Filter selection information may be embedded in the video bit stream.

Methods and systems for seeking to non-key frames

Abstract: Methods and systems for seeking to a non-key frame in a compressed video. The compressed video has a plurality of frames, each with a playback timestamp. At least one of the frames is a key frame. A selection of a non-key frame is received, and a temporally proximate key frame is identified. The set of frames between the identified key frame and the selected frame is played in a reduced time interval, and the selected frame is displayed as if it were a key frame.

An Method for Vehicle-Flow Detection and Tracking in Real-Time Based on Gaussian Mixture Distribution

Abstract: Vehicle-flow detection and tracking by digital image are one of the most important technologies in the traffic monitoring system. Gaussian mixture distribution method is used to eliminate the influence of moving vehicle firstly in this text, and then we built the background images for vehicle flow. Combining the advantages of background difference algorithm with inter frame difference operator, the real-time background is segmented integrally and dynamically updated accurately by matching the reconstructed image with current background. In order to ensure the robustness of vehicle detection, three by three window templates are adopted to remove the isolated noise spot in the image of vehicle contour. The template structural element is used to do some graphical morphological filtering. So, the corrosion and expansion sets are obtained. To narrow the target search scope and improve the calculation speed and precision of the algorithm, Kalman filtering model is used to realize the tracking of fast moving vehicles. Experimental results show that the method has good real-time and reliable performance.

Quick inter-frame prediction mode selection method for AVS-M video coding

Abstract: Provided is a quick intra-frame prediction mode selection method for AVS-M video coding. As shown in a figure, an algorithm uses the textural features in the image edge direction to select a predication mode corresponding to the direction with the maximum module in an edge orientation histogram and a DC mode without orientation as candidate predication modes of a current block. The space-time correlation of an image is utilized to select the optimal prediction mode of a block B0n-1 corresponding to a previous frame, the optimal predication mode of a left block B2n of the current block serves as a candidate prediction mode of the current block, and the correlation of textures of adjacent image blocks in partial space is utilized to select the most possible mode of the current block as a candidate predication mode of the current block. A mode with the minimum CostR is selected as the optimal prediction mode by calculating the rate-distortion cost CostR of the candidate prediction modes of the current block. The quick intra-frame prediction mode selection method can effectively reduce the number of modes to be judged and can quickly confirm the optimal mode. Compared with a full-mode selection algorithm, coding time can be reduced by 17% to 21%.

Inter-frame prediction using motion hints

Abstract: We recently proposed a novel approach that employs motion hints for inter-frame prediction. Motion hints are a loose and global description of motion communicated as metadata; they specify motion but they leave it to the client/decoder to find the exact locations where motion is applicable. This work proposes a multi-scale approach for identifying these exact locations, which are then used with the available reference frames to generate an inter-frame prediction. The proposed approach is localized and robust to noise and illumination changes. The scheme of this work is applicable to close-loop prediction, but it is more useful in open-loop prediction scenarios, such as using prediction in conjunction with remote browsing of surveillance footage, communicated by a JPIP server. We show that, with reasonably accurate motion, it is possible to produce good inter-frame predictions visually and in terms of PSNR.

Weighted Inter-frame Averaging-Based Channel Estimation for CO-OFDM System

Abstract: We propose an efficient channel estimation method for coherent optical orthogonal frequency-division multiplexing (CO-OFDM) based on weighted inter-frame averaging (WIFA), in which the channel estimation results of the adjacent frames are averaged to increase the estimation accuracy. The principle of the proposed WIFA method is theoretically analyzed, and the benefit is demonstrated by a polarization-division-multiplexed CO-OFDM transmission experiment. The channel magnitude and phase differences among adjacent frames are investigated. The effectiveness of WIFA method, combined with conventional least square method, time-domain averaging method, and intra-symbol frequency-domain averaging method, respectively, is demonstrated through 240-km standard single-mode fiber transmission with different launched power values. The experimental results show that the WIFA method can significantly enhance transmission performance.

Method and apparatus for concealing frame error and method and apparatus for audio decoding

Abstract: Disclosed is a frame error concealment (FEC) method. The method includes: selecting an FEC mode based on states of a current frame and a previous frame of the current frame in a time domain signal generated after time-frequency inverse transform processing; and performing corresponding time domain error concealment processing on the current frame based on the selected FEC mode, wherein the current frame is an error frame or the current frame is a normal frame when the previous frame is an error frame.

An Analytic Framework for Frame-Level Dependent Bit Allocation in Hybrid Video Coding

Abstract: In this paper, we address the frame-level dependent bit allocation (DBA) problem in hybrid video coding. In most existing methods, the DBA solution is achieved at the expense of high, sometimes even unbearable, computational complexity because of the multipass coding involved. Motivated by this, we propose a model-based approach as an attempt to solve this problem analytically. Leveraging the predictive nature in hybrid video coding, we develop a novel interframe dependency model (IFDM), which enables a quantitative measure of the coding dependency between the current frame and its reference frame. Based on the IFDM, the buffer-constrained frame-level DBA problem is carefully formulated. Finally, the model-based DBA method called IFDM-DBA is derived, in which successive convex approximation techniques are employed to convert the original optimization problem into a series of convex optimization problem s of which the optimal solutions can be obtained efficiently. Experimental results suggest that the proposed IFDM-DBA method can achieve up to a 23% bitrate reduction over the JM reference software of H.264.

An Interframe Prediction Technique Combining Template Matching Prediction and Block-Motion Compensation for High-Efficiency Video Coding

Abstract: This paper introduces an interframe prediction technique that combines two motion vectors (MVs) derived respectively from template and block matching for overlapped block motion compensation (OBMC). It has a salient feature of not having to signal the template MV, while achieving a prediction performance close to that of bi-prediction. We begin by studying template matching prediction (TMP) from a theoretical perspective. Based on two signal models, the template MV is shown to approximate the pixel true motion around the template centroid, through which we explain why TMP generally outperforms SKIP prediction but is inferior to block-based motion compensation in terms of prediction performance. We then approach the problem of finding another MV to best complement the template MV from both deterministic and statistical viewpoints, the latter leading to the search of its optimal sampling location in the motion field. The result is a search criterion with OBMC window functions forming a geometry-like motion partitioning when the template area is straddled on the top and to the left of a target block. Generalizations to adaptive template design, multihypothesis prediction and motion merging are made to explore the complexity and performance trade-offs. Extensive experiments based on the HM-6.0 software show that the best of them, in terms of compression performance, achieves 1.7-2.0% BD-rate reductions at a cost of 26% and 39% increases in encoding and decoding times, respectively.

Spatiotemporal texture synthesis and region-based motion compensation for video compression

Abstract: In this paper, a content-based approach for video compression is proposed. The main novelty relies on the complete texture analysis/synthesis framework, which enables the use of multiple algorithms, depending on texture characteristics. The idea comes from the efficient MPEG prediction based on a best mode selection. Existing synthesis algorithms cannot be efficient in synthesizing every kind of texture but a certain range of them. This approach is designed to be jointly used with current and future standard compression schemes. At encoder side, texture analysis includes segmentation and characterization tools, in order to localize candidate regions for synthesis: motion compensation or texture synthesis. The corresponding areas are not encoded. The decoder fills them using texture synthesis. The remaining regions in images are classically encoded. They can potentially serve as input for texture synthesis. The chosen tools are developed and adapted in order to ensure the coherency of the whole scheme. Thus, a texture characterization step provides required parameters to the texture synthesizer. Two texture synthesizers, including a pixel-based and a patch-based approach, are used on different types of texture, complementing each other. The scheme is coupled with a motion estimator in order to segment coherent regions and to interpolate rigid motions using an affine model. Inter frame adapted synthesis is therefore used for non-rigid texture regions. The framework has been validated within an H.264/MPEG4-AVC video codec. Experimental results show significant bit-rate saving at similar visual quality levels, assessed using subjective tests. The method can be coupled with the future HEVC in which blocks can be skipped by the encoder to be synthesized at decoder side.

I frame rate control method based on stable area video quality

Abstract: An I frame rate control method based on stable area video quality includes: diving a video image into a motion area and non-motion area, and respectively building interframe dependence models aiming at different interframe dependence of different areas; and comprehensively considering relation of I frame and front and rear P frame, building stable video quality optimized models aiming at different areas, and selecting appropriate quantization parameters for each macro block in each area by solving models. On the premise that objective mass is close, video 'flashing' is restrained effectively, and I frame rate control of stable video quality is achieved.