A transformation method provides a multi-dimensional affine transformation for representing motion between corresponding image components of successive video image frames. The multi-dimensional affine transformations can represent complex motion that includes any or all of translation, rotation, magnification, and shear. The transformation method of this invention includes determining motion transformations between corresponding pixels in the image components of the first and second video image frames. From the motion transformations between corresponding pixels, multi-dimensional affine motion transformations between the corresponding image components are determined. This transformation method increases the accuracy with which complex motion is represented and results in fewer compression or encoding errors in comparison to conventional methods, thereby increasing compression efficiency.

Image compression and affine transformation for image motion compensation

A method implemented in an object-based video encoder or decoder uses shape information that describes the boundary of a group of pixels representing an object in a sequence of video frames to identify transparent blocks (e.g., macroblocks or blocks so that coding/decoding of these blocks can be skipped. In the object-based video coding method, encoders code shape separately from motion and texture, and shape information is available before the encoder/decoder codes/decodes texture and motion data. The encoder and decoder use this shape information to identify transparent macroblocks or blocks so that texture coding and possible motion coding can be skipped. This method for transparent block skipping reduces coding and decoding operations and reduces the number of bits needed to store a bitstream representing a compressed video sequence.

Transparent block skipping in object-based video coding systems

Homogeneous moving objects of arbitrary shapes are segmented and tracked with respect to the motion of the objects. In an intraframe mode of operation, a segmentation method includes obtaining a motion representation of corresponding pixels in the selected video image frame and a preceding video image frame to form motion-segmented video image features. Video image features are also segmented according to their spatial image characteristics (e.g., color) to form spatially-segmented video image features. Finally, the video image features are jointly segmented as a weighted combination of the motion-segmented video image features and the spatially-segmented video image features. The joint motion and spatial segmentation of image features provides enhanced accuracy in representing moving image features. This enhanced accuracy is particularly beneficial because the motion of image features is a significant display characteristic for human observers.

Segmentation of image features using hierarchical analysis of multi-valued image data and weighted averaging of segmentation results

A semantic video object extraction system using mathematical morphology and perspective motion modeling. A user indicates a rough outline around an image feature of interest for a first frame in a video sequence. Without further user assistance, the rough outline is processed by a morphological segmentation tool to snap the rough outline into a precise boundary surrounding the image feature. Motion modeling is performed on the image feature to track its movement into a subsequent video frame. The motion model is applied to the precise boundary to warp the precise outline into a new rough outline for the image feature in the subsequent video frame. This new rough outline is then snapped to locate a new precise boundary. Automatic processing is repeated for subsequent video frames.

Semantic video object segmentation and tracking

A semantic object tracking method tracks general semantic objects with multiple non-rigid motion, disconnected components and multiple colors throughout a vector image sequence. The method accurately tracks these general semantic objects by spatially segmenting image regions from a current frame and then classifying these regions as to which semantic object they originated from in the previous frame. To classify each region, the method perform a region based motion estimation between each spatially segmented region and the previous frame to computed the position of a predicted region in the previous frame. The method then classifies each region in the current frame as being part of a semantic object based on which semantic object in the previous frame contains the most overlapping points of the predicted region. Using this method, each region in the current image is tracked to one semantic object from the previous frame, with no gaps or overlaps. The method propagates few or no errors because it projects regions into a frame where the semantic object boundaries are previously computed rather than trying to project and adjust a boundary in a frame where the object's boundary is unknown.

Tracking semantic objects in vector image sequences

A method of determining the noise content of a video signal by digitizing a frame based on the video signal to provide a digitized signal which is then filtered via a high pass filter. The filtered image is then divided into a plurality of subimages and the noise signal in each of the subimages is determined from a selected number of subimages having the lowest noise signal are further processed to determine the average noise in the selected subimages and the signal-to-noise ratio determined. The carrier-to-noise ratio may also be determined if desired.

System for determining noise content of a video signal in the disclosure

A cable television signal for example may be monitored and impairments determined and in many cases classified by grabbing at least portion of a transmitted frame of video information somewhere on the network to provide a video signal, performing a two dimensional Fourier transform on the video signal and analyzing the Fourier transform so obtained to determine the location of areas indicating above average occurrence of specific frequencies to provide an indication of an impairment.

Signal quality monitoring system

We describe a real-time digital image processing approach for correcting the impulse noise impairment in television transmission. The described algorithm identifies the pixels impaired with impulse noise and subsequently corrects them. A real-time processing board is constructed to demonstrate the algorithm. >

Impulse noise correction in TV transmission

A novel method which measures the thermal noise content in any television picture is presented. Our method has the following advantages (1) it is non-intrusive and does not require the use of a test signal, (2) the noise assessment is carried in the spatial domain and not in the frequency domain, thus the use of the expensive spectrum analyzer instrument is not required. The other proposed non-intrusive and spatial domain methods are limited in their accuracy up to around 40 dB signal-to-noise ratios. Above that the resulting accuracies are unacceptable. Our method yields accurate results up to 48 dB. The new FCC regulations require the operating signals-to-noise ratios to be at least equal to 40 dB. Prior to estimating the noise our method applies a filter on the picture. This filter suppresses the originally intended (non-noisy) contents of the picture but retains enough information about the noise present. From this information, an estimate of the noise present is calculated. >

Automatic assessment of signal-to-thermal noise ratio of television images

We describe a method which automatically measures the signal-to-noise ratio (SNR) of a television image. Our method does not require test signals or the use of a spectrum analyzer. In our scheme, we use a second order two-dimensional digital filter to separate the snow noise and the picture contents. Then, the SNR of the picture is evaluated by the snow noise contents left in the filtered picture. Computer simulation shows a good agreement between the results of our method and the theoretical values. >

Qin Zhang

Papers

System for determining noise content of a video signal in the disclosure

Signal quality monitoring system

Impulse noise correction in TV transmission

Automatic assessment of signal-to-thermal noise ratio of television images

A non-intrusive scheme for measuring the signal-to-noise ratio in television images