Showing papers on "Inter frame published in 2000"

Analysis of video transmission over lossy channels

Abstract: A theoretical analysis of the overall mean squared error (MSE) in hybrid video coding is presented for the case of error prone transmission. Our model covers the complete transmission system including the rate-distortion performance of the video encoder, forward error correction, interleaving, and the effect of error concealment and interframe error propagation at the video decoder. The channel model used is a 2-state Markov model describing burst errors on the symbol level. Reed-Solomon codes are used for forward error correction. Extensive simulation results using an H.263 video codec are provided for verification. Using the model, the optimal tradeoff between INTRA and INTER coding as well as the optimal channel code rate can be determined for given channel parameters by minimizing the expected MSE at the decoder. The main focus of this paper is to show the accuracy of the derived analytical model and its applicability to the analysis and optimization of an entire video transmission system.

New frame rate up-conversion using bi-directional motion estimation

Abstract: We propose a new frame rate up-conversion algorithm for high quality video. In the proposed scheme, bi-directional motion estimation (ME) is performed to construct the motion vector (MV) field for the frame to be interpolated. Unlike conventional motion-compensated interpolation (MCI) algorithms, the proposed technique does not produce any overlapped pixel and hole region in the interpolated frame, and thus can utilize the overlapped block motion compensation technique to reduce the blocking artifacts. The proposed algorithm is very simple to implement on consumer products when compared to conventional MCI methods. Computer simulation shows a high visual performance of the proposed frame rate up-conversion algorithm.

Key frame selection to represent a video

Abstract: This paper describes a technique to automatically extract a single key frame from a video sequence. The technique is designed for a system to search video on the World Wide Web. For each video returned by a query, a thumbnail image that illustrates its content is displayed to summarize the results. The proposed technique is composed of three steps. Shot boundaries detection, shot selection, and key frame extraction within the selected shot. The shot and key frame are selected based on measures of motion and spatial activity and the likeliness to include people. The latter is determined by skin-color detection and face detection. Simulation results on a large set of video from the Internet, including movie trailers, sports, news, and animation, show the efficiency of the method. Furthermore, this is achieved at a very low complexity cost.

System and method for motion compensation and frame rate conversion

Abstract: The invention pertains to a method of temporal processing (Fig. 10) of motion picture image frames (100) each comprising a plurality of pixels, including the steps of comparing each pixel of a current frame (n) to at least one previous (n-1, n-2, ...) or subsequent image frame (n+1, n+2, ...); determining at least one motion vector (106) corresponding to each such pixel relative to such at least one previous or subsequent image frame; and saving the determined motion vectors; applying motion vectors corresponding to multiple image frames to define a new pixel for each pixel of the current frame, and outputting all of such new pixels as a constructed image frame.

Method for efficiently tracking object models in video sequences via dynamic ordering of features

Abstract: An object model has a plurality of features and is described by a model state. An unregistered feature of the object model, and an available frame from a sequence of images are selected to minimize a cost function of a subsequent search for a match of the selected model feature to the image in the selected frame. Upon a match, the feature is registered in that frame. The model state is then updated for each available frame. The steps of selecting, searching and updating are repeated. A video storage module may contain only one frame corresponding to a single time instance, in which case the framework used is based on integrated sequential feature selection. Alternatively, the video store may contain the entire video sequence, in which case feature selection is performed across all video frames for maximum tracking efficiency. Finally, the video store may contain a small number of previous frames plus the current frame, in which case feature selection spans only a subset of the entire video sequence for each feature matching cycle.

A frame-layer bit allocation for H.263+

Abstract: In typical block-based video coding, the rate-control scheme allocates a target number of bits to each frame of a video sequence and selects the block quantization parameters to meet the frame targets. In this work, we present a new technique for assigning such targets. This method has been adopted in the test model TMN10 of H.263+, but it is applicable to any video coder and is particularly useful for those that use B frames. Our approach selects the frame targets using formulas that result from combining an analytical rate-distortion optimization and a heuristic technique that compensates for the distortion dependency among frames. The method does not require pre-analyses, and encodes each frame only once; hence, it is geared toward low-complexity real-time video coding. We compare this new frame-layer bit allocation in TMN10 to that in MPEG-2's TM5 for a variety of bit rates and video sequences.

Methods for transforming streaming video data

Abstract: A method for forming an output stream of data includes determining an output resolution for the output stream of data, determining an output frame rate for the output stream of data, determining an output color depth for the output stream of data, retrieving a first frame of data, a second frame of data, and a third frame of data from an input stream of data, the input stream of data having an input resolution, an input frame rate, and an input color depth, subsampling the first frame of data, the second frame of data, and the third frame of data to respectively form a first subsampled frame of data, a second subsampled frame of data, and a third subsampled frame of data, when the output resolution is lower than the input resolution, dropping the second subsampled frame of data, when the output frame rate is lower than the input frame rate, reducing color depth for the first subsampled frame of data and the second subsampled frame of data to respectively form a first reduced frame of data and a second reduced frame of data, when the output color depth is smaller than the input color depth, and converting the first reduced frame of data and the second reduced frame of data into the output stream of data.

Image sequence stabilisation based on DFT filtering

Abstract: An image sequence stabilisation system based on DFT filtering of absolute frame displacements to compensate for undesired fluctuations in the sequence by shifting image frames into correct positions is reported. The system compensates for undesired jitter, while preserving desired global camera motions. Robustness is introduced to interframe motion estimation by averaging motion vectors detected in the phase correlation surface.

Device, method and digital video encoder for block-matching motion estimation

Abstract: A device or method for video compression uses a technique in which changes in the image are encoded by motions of block of the image and signals indicating evolutions in the block. To determine the motions of the blocks of a each frame, a search is performed for a similar block of a previous frame based on points of the previous frame which are arranged in successive diamond shaped zones. The diamond shaped zones may be centred on the position of the block in previous frame, or one or more predicted motions of the block. The method terminates according to criteria defined using thresholds.

Frame construction method, frame construction device and data transfer system capable of accommodating STM traffic and best effort traffic in common frame format

Abstract: A layer 1 frame, capable of accommodating data of any protocol selected from an STM (Synchronous Transfer Mode) signal, ATM (Asynchronous Transfer Mode) cells, a primary IP (Internet Protocol) packet and a best effort IP packet in a common frame format, is constructed by an edge node and transmitted to a data transfer system composed of edge nodes, core nodes, etc. The layer 1 frame includes a layer 1 frame header, a layer 1 frame payload and a payload CRC (Cyclic Redundancy Check) field. The layer 1 frame header includes a “Packet Length” identifier, a “Priority” identifier, a “Protocol” identifier indicating the type of the data, a “Frame Mode” identifier, a “Stuff” identifier indicating whether or not stuff data is contained in the layer 1 frame, and a “Header CRC” identifier. The layer 1 frame payload contains a layer 2 frame having a layer 2 frame header and a layer 2 frame payload in which the data is packed. STM layer 1 frames containing the STM signals are transferred with the highest priority at fixed intervals (125 μsec). Best effort IP layer 1 frames containing the best effort IP packets are transferred with the lowest priority. Partitioning of the best effort IP layer 1 frame, stuffing, dummy frames, etc. are employed for implementing the periodical transmission of the STM layer 1 frames.

Systems and methods for tracking objects in video sequences

Abstract: A method for tracking one or multiple objects from an input video sequence allows a user to select one or more regions that contain the object(s) of interest in the first and the last frame of their choice. An initialization component selects the current and the search frame and divides the selected region into equal sized macroblocks. An edge detection component computes the gradient of the current frame for each macroblock and a threshold component decides then which of the macroblocks contain sufficient information for tracking the desired object. A motion estimation component computes for each macroblock in the current frame its position in the search frame. The motion estimation component utilizes a search component that executes a novel search algorithm to find the best match. The mean absolute difference between two macroblocks is used as the matching criterion. The motion estimation component returns the estimated displacement vector for each block. An output component collects the motion vectors of all the predicted blocks and calculates the new position of the object in the next frame.

Digital camera for capturing a sequence of full and reduced resolution digital images and storing motion and still digital image data

Abstract: A method for simultaneously recording motion and still images, includes the steps of: capturing a motion image sequence and accompanying audio of a scene with a digital video camera adapted to record both motion and higher resolution still images; simultaneously capturing a still image sequence having a higher resolution and lower frame rate than the motion capture sequence; compressing the motion image sequence using interframe compression and the accompanying audio and storing the compressed motion image and audio data; and compressing the still images using intraframe coding and storing the compressed still image data.

Block-based adaptive lossless image coder

Abstract: With an exponential growth in the use of digital documents and photographic and medical images, the interest in lossless image compression has increased. Coders, such as CALIC and JPEG-LS, using context modeling have raised the bar on achievable compression performance. However, the computation required for these coders is significant and naturally serial. Parallelizable and compute efficient compression and decompression algorithms have attractive features such as cost effective hardware and scalable software implementations. Hence, we propose a compute friendly, parallel algorithm for image compression with compression performance comparable to that of the state-of-the-art schemes. Furthermore, we also show that the proposed method is applicable to interframe lossless coding for image sequences such as medical, graphical, and video contents.

Compressed video signal including information for independently coded regions

Abstract: In a MPEG or other video system, a “region map” is preferably defined in header information for each frame which indicates for each independently coded region in the frame which “image slices” contain data for that region. Using a system of globally and locally defined regions (region groups), the map permits automatic identification of an object through multiple frames (notwithstanding movement) and for select decoding and extraction of that object only, without necessarily decoding all irrelevant image data. An edited region can be re-inserted into the original encoded bit stream in place of the extracted data using minimal computational resources. Using this signal format, objects or locations within a video sequence can be organized into region groups such that they can be tracked from frame to frame and associated with each other; each region group is independently coded, such that motion vectors and residuals of a dependent frame (such as a MPEG “P” or “B” frame) point only to a corresponding region in an anchor frame.

Method of estimating motion in interlaced video

Abstract: A method of estimating motion in interlaced video involves, firstly, a frame search ( 61 ), where a search is conducted for the frame structure using a sub-sampled block matching metric (e.g. sub-sampled SAD). The locations to be searched are either fixed or dynamically determined based on the minimum frame SAD (ie best frame block match). Next, pixel errors are calculated ( 62 ) for all pixels included in the sub-sampled block matching metric. Each pixel error is first identified as belonging to one of four field patterns (e.g. even-even, even-odd, odd-even and odd-odd). For each location, pixel errors can be classified into two field patterns, either even-even and odd-odd or even-odd and odd-even ( 63 ). The pixel errors belonging to the same field pattern are added together ( 64 ) to obtain field error values (eg field SAD values). The individual field error (or field SAD) values are used to determine ( 77-80 ) the field Motion Vectors. The location of the lowest field SAD value is taken as the position of best match. All pixel errors are then summed together ( 65 ) to obtain the frame error and the location of the lowest frame error (or frame SAD) is taken as the position of best match for the frame. The frame and field Motion Vectors can be refined by using a full block matching metric within the small search window ( 66, 73-76 ).

System and method for generating video frames

Abstract: The system includes a frame generator which generates one or more intermediate frames based upon one base frames. Each of the base frames are comprised of a plurality of macroblocks. In the frame generation process, the frame generator performs a number of steps such as: (i) determines whether frame generation is appropriate, (ii) examines the first and second base frames to check for the presence of textual characters, (iii) selects a frame generation method based upon information in the first and second frames, (iv) filters the generated frames. This application focuses on analysing the first and second base frames and determining the method of frame interpolation.

Video frame rate control for non-guaranteed network services with explicit rate feedback

Abstract: An adaptive video frame rate control method is proposed for the network of time-varying rate channel with explicit rate feedback. It consists of a prediction module of future channel rate and an adaptive frame discarding and skipping module. We derive an encoder buffer constraint which guarantees an end-to-end delay bound of video frames. The recursive least-squares (RLS) method is used as a tool to predict the low frequency component of channel rate. The adaptive frame discarding method prevents delay violation of frames due to the channel rate prediction error. Also, the frame skipping method adapts the encoder output rate to the channel rate while keeping a constant level of video quality. From the simulation result, it is shown that the proposed method keeps the decoded image quality nearly constant with much less number of skipped frames and no delay violating frame during the congestion period.

Method and apparatus for digital data compression

Abstract: The quality of digital images recovered from compressed data in an inter-frame redundancy-removing scheme is enhanced using a self-adaptive feedback scheme in an image compression/decompression system so as to include means for the compensation of the distortion component from prior frame compression in subsequent difference frame compression. Each transmitted frame is stored after a full compress/decompress cycle, and difference data (which includes the inverse of the distortion component from compression of the transmitted frame) representing the difference between the stored frame and the incoming new frame is transmitted. Consequently, the quality of static regions in the recovered images may be improved with each subsequent iteration by taking the distortion component in the prior frame into consideration along with the inter-frame motion information. The feedback loop thus forms a self-adaptive iterative cycle.

Indexing for motion video that is compressed using interframe and intraframe techniques

Abstract: Random access to arbitrary images, whether frames or fields, of a video segment compressed using both interframe and intraframe techniques may be enhanced by including state information, for decoding and display, at appropriate points in the compressed bitstream to enable random access to each intraframe compressed image. The state information may be inserted during compression or by processing the bitstream of compressed data. An image index also may be generated that maps each temporal image in a decompressed output image sequence to an offset in the compressed bitstream of the data used to decode the image. The index may be created during compression or by processing the bitstream of compressed data. To access one or more samples starting with a specified point in time in a decompressed output image sequence, the index is accessed using the specified point in time to identify another sample in the decompressed output image sequence for which data is used to produce the specified sample. The identity of the other sample is used to access the index to identify a location in the compressed data for the data used to produce the specified sample.

Methods and apparatus for reduction of prediction modes in motion estimation

Abstract: In image encoding methods or systems, first and second motion vectors are determined for the two fields comprising a frame based on a field search. The first and second motion vectors then may be used to determine whether a frame prediction search is necessary. If the difference between motion vectors for the two fields of the frame is less than a threshold, the frame search should be performed. A suitable threshold may be calculated or described in any number of ways, however, in one embodiment of the present invention, the threshold is dynamic that changes in response to the changing information in either preceding or succeeding fields.

Method and apparatus for selecting image data to skip when encoding digital video

Abstract: Block and frame skipping decides which image regions, blocks or frames in a video frame or series of frames should be encoded. Block skipping detects in advance which of the blocks in a video frame will produce little or no bits, so that the encoder can skip the encoding process for these blocks. Block skipping uses an iterative procedure to determine the optimal energy threshold value for each image frame. All the image blocks whose pixel energy is below the optimal threshold for the frame are skipped. Frame skipping predicts a distortion or image quality value for each frame. If some frames have a predicted image quality smaller than a preset value, these frames are allocated more bits and if, as a result, too few bits are left for future frames, the latter frames are skipped. This selective frame skipping guarantees that the quality of the non-skipped frames is above the preset value.

A contex-based predictive coder for lossless and near-lossless compression of video

Abstract: We propose a new approach to context-based predictive coding of video, where the interframe or intraframe coding mode is adaptively selected on a pixel basis. We perform the coding mode selection using only the previously reconstructed samples which are also available at the decoder, so that any overhead information on the coding mode selection does not need to be transmitted to the decoder. The proposed coder also provides the lossless concatenated coding property when applied to multigeneration of video sequences since the same coding mode information is available at the second time encoding. The proposed coding mode selection enables the coder to easily incorporate error modeling and context modeling by performing the intraframe coding with one of the existing image coders such as the JPEG-LS standard. Experiments show that the proposed approach in conjunction with the JPEG-LS standard provides significant improvements in compression efficiency.

Method and apparatus for transcoding coded video image data

Abstract: A transcoding apparatus converts first coded data obtained from a plurality of frames forming a video image by means of a coding algorithms including an interframe prediction coding algorithm with motion compensation into second coded data having a different structure. More specifically, the transcoding apparatus takes a portion of frames which have had the interframe prediction coding and are included in the plurality of frames forming the video image as processing object frames, and from at least the first coded data corresponding to the processing object frames, generates the second coded data corresponding to coded data which are to be obtained when performing the interframe prediction coding on the processing object frames with reference to second reference frames which are different from first reference frames referenced in the interframe prediction coding of the processing object frames. In order to the second coded data, the transcoding apparatus has a motion information calculating portion for calculating motion information to be included in the second coded data using at least motion information contained in said processing object data.

Rate-constrained multi-hypothesis motion-compensated prediction for video coding

Abstract: Multi-hypothesis prediction extends motion compensation with one prediction signal to the linear superposition of several motion-compensated prediction signals with the result of increased coding efficiency. The multiple hypotheses in this paper are blocks in past decoded frames. These blocks are referenced by individual motion vectors and picture reference parameters incorporating long-term memory motion-compensated prediction. In this work, we at most employ two hypotheses similar to B-frames. However, they are obtained from the past. Due to the increased rate for the motion vectors, rate-constrained coder control is utilized. For this scheme, we demonstrate the coding efficiency of multi-hypothesis prediction in combination with variable block size and long-term memory and present bit-rate savings up to 32% when compared to standard variable block size prediction without long-term memory motion compensation.

Motion vector detection apparatus, method of the same, and image processing apparatus

Abstract: A control circuit specifies a first search area by using a motion vector found in units of macroblocks between a plurality of frames located previous to first frame, specifies a second search area by using a simplified motion vector indicating the motion of an image between the first frame, the second frame, and a frame located in time between the first frame and the second frame, and designates a search area by switching the first search area and the second search area suitably.

Method and system for energy based frame rate determination

Abstract: A sequence of code symbols is supplied to a rate decision block. The sequence of code symbols can be output from a Viterbi decoder, and may correspond to block encoded and convolutional encoded output from a vocoder, produced in data frames at varying data rates, or frame rates, unknown to the decoder or to the rate decision block. The rate decision block determines a number of frame energies based on the sequence of code symbols, where each frame energy corresponds to a tentative frame rate. Each frame energy can be determined, by measuring the power and duration of each code symbol based on the tentative frame rate, to determine the tentative energy of each code symbol in a frame, and then adding the tentative energies of all the code symbols in the frame. The rate decision block determines a final frame rate when the frame energies meets a desired condition.

Nonlinear motion-compensated interpolation for low-bit-rate video

Abstract: By augmenting the ITU-T H.263 standard bit stream with supplementary motion vectors for to-be-interpolated frames, a new deformable block-based fast motion-compensated frame interpolation (DB-FMCI) scheme is presented. Unlike other motion-compensated interpolation methods, which assume a constant motion velocity between two reference P frames, the proposed scheme takes into account the non-linearity of motion to achieve a better interpolation result. The supplementary motion information for the so-called M frame (motion frame) is defined, which consists of compressed residues of linear and non-linear motion vectors. The non-linear motion vectors of skipped frames are used at the decoder to determine the 6- parameter affine-based DB-FMCI. Experimental results show that the proposed non-linear enhancement scheme can achieve a higher PSNR value and better visual quality in comparison with traditional methods based only on the linear motion assumption.

A novel codec for thin client computing

Abstract: In this paper, we describe thin client compression (TCC), a novel codec for screendumps and sequences of such images that exploits both local and global redundancy as well as interframe redundancy. Our method extends textual image compression to non-bilevel images, and uses three piecewise-constant models to separately code bilevel marks, non-bilevel marks, and the residue. It also speeds up pattern matching and substitution by exploiting the absence of noise in synthetic images, and shares its codebook across images. Our method compresses a series of test images 2.6 to 8.2 times better than state-of-the-art methods. Its speed is adequate for interactive logins, and it achieves the best end-to-end latency over low bandwidth connections.

System for lost packet recovery in voice over internet protocol based on time domain interpolation

Abstract: A lost packet recovery device, method and computer program for use in a VoIP system in which lost packets containing voice information are replaced using time domain interpolation techniques. These time domain interpolation techniques employ two different approaches to interpolate missing data packets. The first approach relies on time domain harmonic scaling to interpolate a replacement frame for a missing frame using the frames that come before and after the missing frame. The second approach replicates a frame immediately prior to the missing frame. This replicated frame then has an energy reduction function applied to it to gradually reduce the energy output level of the data samples in the frame. This replicated frame is then used to replace the missing frame. In the second approach, the process of duplicating the prior frame and reducing its energy levels using an energy reduction function is repeated until no further missing frames are detected. Once no further missing frames are detected, an energy restoration function is applied to the next available frame to gradually increase its energy level and provide for a smooth transition. Using these techniques, missing frames of voice data may be replaced to mask the effects of missing frames to a listener.

Exploiting interframe redundancies in the lossless compression of 3D medical images

Abstract: Summary form only given. An evaluation is made of different approaches to removing interframe redundancies. The test images we used are the MRI and CT images available from the Visual Human Project. Firstly, we found that linear predictive techniques are not able to provide any compression improvement. Even an optimal linear predictor (least-squares optimization), which holds pixels from the current and the previous frames, does not outperform the very simple 2D lossless JPEG predictor number 7. However nonlinear techniques, such as context-modeling do yield better results. In a simple experiment, lossless JPEG was estimated to code intraframe prediction errors using an interframe context. The context for coding the current pixel is formed by the magnitude of the prediction error of the corresponding pixel in the previous frame. Note that this prediction error comes from the intraframe prediction step in the previous frame. Using this interframe context-modeling approach, a reduction of the bit rate of up to 1 bit per pixel is obtainable. It is obvious that the context serves as an edge-detector, albeit a very simple and straightforward one. A new technique is proposed based on the 2D lossless image compressor JPEG-LS. The existing context-modeling scheme is extended to also catch the interframe redundancies. The interframe context is very similar to the abovementioned one, however the sign of the prediction error in the previous frame is also used.