Showing papers on "Residual frame published in 2013"

LightSync: unsynchronized visual communication over screen-camera links

Abstract: A key challenge for smartphone based visual communication over screen-camera links is imperfect frame synchronization. The difficulty arises from frame rate diversity and variability due to camera capability, lighting conditions, and system factors. On the 4 smartphone cameras we tested, the frame rate varies between 8 and 30 frames per second. If the transmit frame rate is too high, the receiver might lose original frames or capture mixed frames, which are normally not decodable. Previous systems simply reduce the effective screen frame rate to be half the camera frame capture rate, to guarantee receiving a decodable frame every other frame. This under-utilizes the transmitter side capacity and is inefficient.We achieve frame synchronization with LightSync, which features in-frame color tracking to decode imperfect frames and a linear erasure code across frames to recover lost frames. LightSync allows smooth communication between the screen and the camera at any combination of the transmit and receive frame rates, as long as the receive rate is at least half the transmit rate. This means that each receiver can scale the decoding performance with its own camera capability. Across several phones, our system can more than double the average throughput compared to previous approaches.

Method and system for fast dense stereoscopic ranging

Abstract: A stochastic method and system for fast stereoscopic ranging includes selecting a pair of images for stereo processing, in which the pair of images are a frame pair and one of the image is a reference frame, seeding estimated values for a range metric at each pixel of the reference frame, initializing one or more search stage constraints, stochastically computing local influence for each valid pixel in the reference frame, aggregating local influences for each valid pixel in the reference frame, refining the estimated values for the range metric at each valid pixel in the reference frame based on the aggregated local influence, and post-processing range metric data. A valid pixel is a pixel in the reference frame that has a corresponding pixel in the other frame of the frame pair. The method repeats n iterations of the stochastically computing through the post-processing.

Rate-GOP Based Rate Control for High Efficiency Video Coding

Abstract: In this paper, a Rate-GOP based frame level rate control scheme is proposed for High Efficiency Video Coding (HEVC). The proposed scheme is developed with the consideration of the new coding tools adopted into HEVC, including the quad-tree coding structure and the new reference frame selection mechanism, called reference picture set (RPS). The contributions of this paper mainly include the following three aspects. Firstly, a RPS based hierarchical rate control structure is designed to maintain the high video quality of the key frames. Secondly, the inter-frame dependency based distortion model and bit rate model are proposed, considering the dependency between a coding frame and its reference frame. Thus the distortion and bit rate of the coding frame can be represented by the distortion and bit rate of its reference frame. Accordingly, the Rate-GOP based distortion model and rate model can be achieved via the inter-frame dependency based distortion model and bit rate model. Thirdly, based on these models and a mixed Laplacian distribution of residual information, a new ρ-domain Rate-GOP based rate control is proposed. Experimental results demonstrate the proposed Rate-GOP based rate control has much better R-D performance. Compared with the two state-of-the-art rate control schemes for HEVC, the coding gain with BD-PSNR can be up to 0.87 dB and 0.13 dB on average respectively for all testing configurations. Especially for random access low complexity testing configuration, the BD-PSNR gain can be up to 1.30 dB and 0.23 dB respectively.

Motion-Compensated Frame Interpolation Based on Multihypothesis Motion Estimation and Texture Optimization

Abstract: A novel motion-compensated frame interpolation (MCFI) algorithm to increase video temporal resolutions based on multihypothesis motion estimation and texture optimization is proposed in this paper. Initially, we form multiple motion hypotheses for each pixel by employing different motion estimation parameters, i.e., different block sizes and directions. Then, we determine the best motion hypothesis for each pixel by solving a labeling problem and optimizing the parameters. In the labeling problem, the cost function is composed of color, shape, and smoothness terms. Finally, we refine the motion hypothesis field based on the texture optimization technique and blend multiple source pixels to interpolate each pixel in the intermediate frame. Simulation results demonstrate that the proposed algorithm provides significantly better MCFI performance than conventional algorithms.

Novel True-Motion Estimation Algorithm and Its Application to Motion-Compensated Temporal Frame Interpolation

Abstract: In this paper, a new low-complexity true-motion estimation (TME) algorithm is proposed for video processing applications, such as motion-compensated temporal frame interpolation (MCTFI) or motion-compensated frame rate up-conversion (MCFRUC). Regular motion estimation, which is often used in video coding, aims to find the motion vectors (MVs) to reduce the temporal redundancy, whereas TME aims to track the projected object motion as closely as possible. TME is obtained by imposing implicit and/or explicit smoothness constraints on the block-matching algorithm. To produce better quality-interpolated frames, the dense motion field at interpolation time is obtained for both forward and backward MVs; then, bidirectional motion compensation using forward and backward MVs is applied by mixing both elegantly. Finally, the performance of the proposed algorithm for MCTFI is demonstrated against recently proposed methods and smoothness constraint optical flow employed by a professional video production suite. Experimental results show that the quality of the interpolated frames using the proposed method is better when compared with the MCFRUC techniques.

Systems and methods for tracking human hands using parts based template matching within bounded regions

Abstract: Systems and methods for tracking human hands using parts based template matching within bounded regions are described. One embodiment of the invention includes a processor; an image capture system configured to capture multiple images of a scene; and memory containing a plurality of templates that are rotated and scaled versions of a finger template. A hand tracking application configures the processor to: obtain a reference frame of video data and an alternate frame of video data from the image capture system; identify corresponding pixels within the reference and alternate frames of video data; identify at least one bounded region within the reference frame of video data containing pixels having corresponding pixels in the alternate frame of video data satisfying a predetermined criterion; and detect at least one candidate finger within the at least one bounded region in the reference frame of video data.

Rotation of prediction residual blocks in video coding with transform skipping

Abstract: Techniques are described for coding residual data of a prediction residual block with transform skipping. A transform may be skipped for a residual block when the residual block is coded using either a lossless coding mode or a lossy coding mode in a transform skip mode. According to the techniques, based on a transform being skipped for a residual block, a rotation unit included in a video encoder or a video decoder determines whether to rotate the residual block prior to coding residual data of the residual block. In some examples, a rotation value may be explicitly signaled between the video encoder and the video decoder. In other examples, the video encoder and the video decoder may each independently determine whether to rotate the residual block based on a type of boundary at two or more edges of the residual block.

Video rate control for video coding standards

Abstract: A method and device for improving rate controlling in video coding of sequences including a series of Inter frames separated by Intra frames, when a decoding delay is considered, comprise for each Inter frame of the series: computing a target frame size, computing a maximum buffer level related to a position of each Inter frame relative to a previous Intra frame and an upcoming Intra frame, and optimizing a transmission buffer level in response to the computed target frame size and the computed maximum buffer level.

System and Method for Image Enhancement and Improvement

Abstract: A method and system for improving picture quality of images by providing a series of frames of a given region of interest. One embodiment for a method includes: determining the value of each pixel at each location within each frame to form a first array of pixel values for each frame; determining the overall intensity of each frame; determining the product of the overall intensity and the array of pixel values for each frame; determining the sum of the products by adding together the products of the overall frame intensity and first array of pixel values for each frame; determining the average of the sum of products by dividing the sum of products by the number of frames in the series of frames; determining the average value of each pixel at each pixel location for the series of frames to form a second array of average pixel values; determining the average overall frame intensity for the series of frames; determining a second product of the second array of average pixel values and the average overall frame intensity; subtracting the second product from the first product to provide an improved image of the region of interest. Other improvement embodiments are also disclosed.

Adaptive filtering mechanism to remove encoding artifacts in video data

Abstract: A method includes determining, through a processor and/or a hardware engine, edge pixels and flat pixels of a video frame of a video sequence during decoding thereof or post-processing associated with the decoding based on a predetermined threshold, and quantifying spatial correlation of pixels of the video frame around edges thereof to estimate strength of ringing artifacts and spatial and temporal persistence thereof across the video frame and across video frames of the video sequence. The method also includes adaptively and spatially filtering the pixels around the edges of the video frame, adaptively and temporally filtering the video frame, and blending an output of the adaptive spatial filtering and the adaptive temporal filtering to generate an output with suppressed ringing artifacts, spatial and temporal persistence thereof and artifacts resulting from the cumulative effect of compression therein.

Image processing device, image processing method, image processing program, and storage medium

Abstract: An image processing device for setting a region inside a frame image, and generating an output frame image by correcting the position or shape of the region according to the motion of an imaging device. This image processing device is equipped with: a motion acquisition unit for acquiring motion data between a first frame image and a second frame image; an estimation unit for estimating a first, a second, and a third matrix by dividing a projection matrix, which projects the output frame image onto the second frame image, into a first matrix including a parallel-motion component in the direction perpendicular to the imaging direction, and a rotation component centered around an axis extending in the imaging direction, a second matrix including a rolling-shutter distortion component, and a third matrix including an error component; a correction unit for correcting the first matrix and the third matrix; and a drawing unit for calculating the projection matrix and generating the output frame image on the basis of the corrected first matrix, the second matrix, and the corrected third matrix.

Systems and methods for phase rotating duplicate frames in wireless lan transmissions

Abstract: Methods, devices, and computer program products for optimally phase rotating duplicate frames in wireless LAN transmissions are disclosed. In one aspect, phase rotation sequences may be chosen in order to minimize a peak-to-average power ratio (PAPR) of a frame or data unit, or of a portion of a frame or data unit, where the frame contains a plurality of identical frequency segments, such as a duplicate frame. The method involves selecting a frame bandwidth, and then selecting a phase rotation sequence based upon the frame bandwidth. The method further includes generating a frame including a number of identical 1 MHz frequency segments, and rotating some of those segments relative to other segments, based on the selected phase rotation sequence. The method further includes transmitting the frame.

Video coding with dynamic background

Abstract: Motion estimation (ME) and motion compensation (MC) using variable block size, sub-pixel search, and multiple reference frames (MRFs) are the major reasons for improved coding performance of the H.264 video coding standard over other contemporary coding standards. The concept of MRFs is suitable for repetitive motion, uncovered background, non-integer pixel displacement, lighting change, etc. The requirement of index codes of the reference frames, computational time in ME & MC, and memory buffer for coded frames limits the number of reference frames used in practical applications. In typical video sequences, the previous frame is used as a reference frame with 68–92% of cases. In this article, we propose a new video coding method using a reference frame [i.e., the most common frame in scene (McFIS)] generated by dynamic background modeling. McFIS is more effective in terms of rate-distortion and computational time performance compared to the MRFs techniques. It has also inherent capability of scene change detection (SCD) for adaptive group of picture (GOP) size determination. As a result, we integrate SCD (for GOP determination) with reference frame generation. The experimental results show that the proposed coding scheme outperforms the H.264 video coding with five reference frames and the two relevant state-of-the-art algorithms by 0.5–2.0 dB with less computational time.

Bandwidth reduction in video coding through applying the same reference index

Abstract: Techniques for encoding and decoding video data are described. A method of coding video may include determining a plurality of motion vector candidates for a block of video data for use in a motion vector prediction process, wherein each of the motion vector candidates points to a respective reference frame index, performing the motion vector prediction process using the motion vector candidates to determine a motion vector for the block of video data, and performing motion compensation for the block of video data using the motion vector and a common reference frame index, wherein the common reference frame index is used regardless of the respective reference frame index associated with the determined motion vector.

Using a combination of 2D and 3D image data to determine hand features information

Abstract: A method of determining hand features information using both two dimensional (2D) image data and three dimensional (3D) image data is described. In one implementation, a method includes: receiving a 2D image frame; receiving 3D image data corresponding to the 2D image frame; using the 3D image data corresponding to the 2D image frame, transforming the 2D image frame; and using the 3D image data corresponding to the 2D image frame, scaling the 2D image frame, where the transforming and scaling results in a normalized 2D image frame, where the normalized 2D image frame is a scaled and transformed version of the 2D image frame, and where the scaling and transforming is performed using a computer.

Audio Signal Encoder

Abstract: An apparatus comprising: a coding rate determiner configured to determine a first coding bitrate for at least one first frame audio signal multi-channel parameter and a second coding bitrate for at least one second frame audio signal multi-channel parameter, wherein the combined first and second coding bitrate is less than a bitrate limit; a channel analyser configured to determine for a first frame the at least one first frame audio signal multi-channel parameter and configured to determine for a second frame the at least one second frame audio signal multi-channel parameter; a multi-channel parameter determiner configured to generate an encoded first frame audio signal multi-channel parameter within the first coding bitrate from the at least one first frame audio signal multi-channel parameter and configured to generate an encoded at least one second frame audio signal parameter within the second coding bitrate from the at least one second frame audio signal multi-channel parameter; and a multiplexer configured to combine the encoded at least one first frame audio signal multi-channel parameter and the encoded at least one second frame audio signal multi-channel parameter.

Compression of 3D mesh sequences by temporal segmentation

Abstract: We describe a compression method for three-dimensional animation sequences that has notable advantages over existing techniques. We first aggregate the frame data by similarity and reorganize them into clusters, which results in the sequence split into several motion fragments of varying lengths. To minimize the number of clusters and obtain optimal clustering, we perform frame alignment, which eliminates the “global” rigid transformation from each frame data and use only “pose” when evaluating the similarity between frames. We then apply principal component analysis for each cluster, from which we get coordinates of corresponding frames in a reduced dimension. Because similar frames are considered, the number of coefficients required for each frame becomes smaller; thus, we obtain better dimension reduction for a given reconstruction error. Further, we perform intracluster compression based on linear coding. Because every motion fragment presents similar frames, conventional linear predictive coding can be replaced by key frame-based linear coding to achieve minimal reconstruction error. Results show that our method can obtain a high compression ratio, with a limited reconstruction error. Copyright © 2013 John Wiley & Sons, Ltd.

Method and system for controlling video frame encoding

Abstract: A method of enabling iterative encoding of a video frame by a video encoder, comprising obtaining a video-encoder-state resulting from encoding of a previous input video frame and previous to encoding of a current input video frame, copying the video-encoder-state giving rise to a reserved state and obtaining a candidate current encoded video frame from the video encoder, and in case the candidate current encoded video frame does not meet an encoding criterion, copying the reserved state back to the video encoder to enable the video encoder to re-encode the current input video frame.

Multi-Pass Video Noise Filtering

Abstract: A method of noise filtering of a digital video sequence is provided that includes computing a motion image for a frame, wherein the motion image includes a motion value for each pixel in the frame, and wherein the motion values are computed as differences between pixel values in a luminance component of the frame and corresponding pixel values in a luminance component of a reference frame, applying a first spatial noise filter to the motion image to obtain a final motion image, computing a blending factor image for the frame, wherein the blending factor image includes a blending factor for each pixel in the frame, and wherein the blending factors are computed based on corresponding motion values in the final motion image, generating a filtered frame, wherein the blending factors are applied to corresponding pixel values in the reference frame and the frame, and outputting the filtered frame.

Mobile device based text detection and tracking

Abstract: Embodiments disclosed pertain to mobile device based text detection and tracking. In some embodiments, a first reference frame is obtained by performing Optical Character Recognition (OCR) on an image frame captured by a camera to locate and recognize a first text block. A subsequent image frame may be selected from a set of subsequent image frames based on parameters associated with the selected subsequent image and a second reference frame may be obtained by performing OCR on the selected subsequent image frame to recognize a second text block. A geometric relationship between the first and second text blocks is determined based on a position of the first text block in the second reference frame and a pose associated with the second reference frame.

Devices for redundant frame coding and decoding

Abstract: A method for redundant frame coding by an electronic device is described. The method includes determining an adaptive codebook energy and a fixed codebook energy based on a frame. The method also includes coding a redundant version of the frame based on the adaptive codebook energy and the fixed codebook energy. The method further includes sending a subsequent frame.

Methods and systems for seeking to non-key frames

Abstract: Methods and systems for seeking to a non-key frame in a compressed video. The compressed video has a plurality of frames, each with a playback timestamp. At least one of the frames is a key frame. A selection of a non-key frame is received, and a temporally proximate key frame is identified. The set of frames between the identified key frame and the selected frame is played in a reduced time interval, and the selected frame is displayed as if it were a key frame.

Selection of transform size in video coding

Abstract: Blocks of a frame of a video stream can be encoded using various prediction modes followed by transforming the predicted blocks. Increasing coding efficiency may be achieved by indicating a default transform mode for a frame in a frame header. The default transform mode can be used to encode blocks of the frame. When the default transform mode is a per-block transform mode, the transform size can be determined on a per-block basis and may be included in block headers. When all available transform sizes are not tested for a current block, estimates of a rate distortion value can be performed for the current block using its prediction mode and the remaining transform sizes. Using these estimates and the actual calculated rate distortion values for blocks of the current frame, the default transform mode can be updated for the next frame.

Method and apparatus for discontinuous reception in user equipment for power savings

Abstract: The present aspects relate to methods and apparatuses to power off a modem receiver or components of the receiver at a time prior to the end of the frame based on data in the frame being received and properly decoded before the end of the frame. In an aspect, the apparatuses and methods provide power saving in a wireless device, and include receiving data within a frame at a user equipment, determining whether all payload packet data has been correctly decoded prior to an end of the frame, and powering down a receiver component for a portion of a remainder of the frame in response to determining that all payload packet data has been correctly decoded and where a first time period to a next scheduled overhead bit transmission period of a slot in the frame is greater than a second time period corresponding to a warm-up period for the receiver component.

Spatially adaptive video coding

Abstract: A video signal comprises a sequence of source frames to be encoded. A pre-processing stage determines a region of interest for a plurality of the source frames, and spatially adapts each of the plurality of the source frames to produce a respective warped frame. In the respective warped frame, the region of interest comprises a higher spatial proportion of the warped frame than in the source frame. The pre-processing stage supplies the warped frames to an encoder to be encoded into an encoded version of the video signal.

Method and apparatus for concealing frame error and method and apparatus for audio decoding

Abstract: Disclosed is a frame error concealment (FEC) method. The method includes: selecting an FEC mode based on states of a current frame and a previous frame of the current frame in a time domain signal generated after time-frequency inverse transform processing; and performing corresponding time domain error concealment processing on the current frame based on the selected FEC mode, wherein the current frame is an error frame or the current frame is a normal frame when the previous frame is an error frame.

Distributed adaptive compressed video sensing using smoothed projected landweber reconstruction

Abstract: A novel Compressed-Sensing-based (CS-based) Distributed Video Coding (DVC) system, called Distributed Adaptive Compressed Video Sensing (DISACOS), is proposed in this paper. In this system, the input frames are divided into key frames and non-key frames, which are encoded by block CS sampling. The key frames are encoded as CS measurements at substantially higher rates than the non-key frames and decoded by the Smoothed Projected Landweber (SPL) algorithm using multi-hypothesis predictions. For the non-key frames, a small number of CS measurements are first transmitted to detect blocks having low-quality Side Information (SI) generated by the conventional interpolation or extrapolation at the decoder; then, another group of CS measurements are sampled again upon the decoder's request. To fully utilise the CS measurements, we adaptively allocate these measurements to each block in terms of different edge features. Finally, the residual frame is reconstructed using the SPL algorithm and the decoded non-key frame is simply determined as the sum of the residual frame and the SI. Experimental results have revealed that our CS-based DVC system yields better rate-distortion performance when compared with other schemes.

Block error compensating apparatus of image frame and method thereof

Abstract: An apparatus and method are provided for compensating a block error in an image frame. This may include a video codec decoder for decoding an inputted image frame, and outputting a decoded image frame. An error concealment block may detect an error-generated block in the decoded image frame and compensate the detected error block through a median filter, and output the compensated image frame.

Systems and methods for compressing video data using image block matching

Abstract: Systems and methods described herein compress video data using image block matching. A server is configured to access a reference frame of an image in a video, separate the reference frame into a plurality of reference blocks of pixels, calculate a hash value for each of the plurality of reference blocks of pixels, receive a current frame of an image in the video, separate the current frame into a plurality of current blocks of pixels, and calculate a hash value for each of the plurality of current blocks of pixels. Further, the server is configured to compare the reference frame hash values with the current frame hash values, identify a hash value in the reference frame that matches a hash value in the current frame, and store the indication that the hash value in the reference frame matches the hash value in the current frame.