Showing papers on "Residual frame published in 1996"

Extraction of high-resolution frames from video sequences

Abstract: The human visual system appears to be capable of temporally integrating information in a video sequence in such a way that the perceived spatial resolution of a sequence appears much higher than the spatial resolution of an individual frame. While the mechanisms in the human visual system that do this are unknown, the effect is not too surprising given that temporally adjacent frames in a video sequence contain slightly different, but unique, information. This paper addresses the use of both the spatial and temporal information present in a short image sequence to create a single high-resolution video frame. A novel observation model based on motion compensated subsampling is proposed for a video sequence. Since the reconstruction problem is ill-posed, Bayesian restoration with a discontinuity-preserving prior image model is used to extract a high-resolution video still given a short low-resolution sequence. Estimates computed from a low-resolution image sequence containing a subpixel camera pan show dramatic visual and quantitative improvements over bilinear, cubic B-spline, and Bayesian single frame interpolations. Visual and quantitative improvements are also shown for an image sequence containing objects moving with independent trajectories. Finally, the video frame extraction algorithm is used for the motion-compensated scan conversion of interlaced video data, with a visual comparison to the resolution enhancement obtained from progressively scanned frames.

Method and system for a user to manually alter the quality of previously encoded video frames

Abstract: A system (31, 41, 32, 33) in which a user manually changes the quality of specific time periods of encoded video (180). After the automatic encoding (50) of video into a compressed digital format, a person editing (31, 33) the encoded video reviews the quality of the video and manually indicates that the quality of specific time periods (180) of the video is to be altered. As the digital storage medium such as an optical disc (84, 90) which stores the encoded video has finite storage capacity, the total number of bits for the encoded video (50) and the quality of the video is limited. A percentage of bits are removed from time sequence and placed in a bit pool (368). Using an exponential function (368) new bits are calculated and the bits in the bit pool are proportionally distributed to the video frames (376). The frames must then be checked (394) and possibly adjusted (398, 400) to prevent underflow of the video buffer.

Video encoding method and system using a rate-quantizer model

Abstract: A system and method for determining quantization level versus bit-rate characteristics of raw video signals in video frames during a pre-encoding phase for video technologies such as MPEG and MPEG-2. During a pre-encoding phase, various quantization levels are assigned to various parts of a frame, and the frame is then pre-encoded to determine a bit-rate for each quantization level used in the pre-encoding phase. Depending on the embodiment, quantization levels are assigned in one of many ways: checkerboard style, block style or any other distribution that avoids statistical anomalies. The method and system repeat the pre-encoding for plural frames, recording all quantization level versus bit-rate statistics on a frame by frame basis. These statistics are then used during encoding or re-encoding of a digital video to control the number of bits allocated to one segment of the digital video as compared to another segment, based on a target quality and target storage size for each segment. The resulting encoded digital video is stored on a digital storage medium, such as a compact disc.

Apparatus and method for detecting motion in a video signal

Abstract: A system detects motion in a video signal by identifying differences between a current image frame and a reference image frame. A difference profile is calculated that represents the differences between the current image frame and the reference image frame. A motion detection signal is generated by the system if the difference profile exceeds a threshold. In one configuration, the system replaces the reference image frame with each successive current image frame. In other configurations, the system replaces the reference image frame with the current image frame if the difference profile exceeds the threshold. The system generates difference data associated with the differences between the current image frame and the reference image frame. This difference data can be partitioned into multiple blocks. An average value is calculated for each block of difference data. Upon generation of a motion detection signal, the system may be configured to record the current image frame if the difference profile exceeds the threshold.

Method to reduce perceptual contouring in display systems

Abstract: A method of pulse width modulation using a spatial light modulator (40) with a finite transition time. The method uses m bits per sample to digitize the incoming data, but apportions the LSB times for pulse width modulation based upon m-1 bits. The current video frame displays all of the bits for each sample, except for the LSBs for each sample. The next video frame displays all of the bits for each sample, adding one more LSB for dividing up the frame time. The first frame could use either the additional LSB time and display no data, or it could use only that number of LSB times it needs. In the latter, the system will have to adjust to different partitions of the frame time for alternating frames. The system includes a spatial light modulator (40), a memory (42), a formatter (48), a sequence controller (44) and a toggle circuit (46), to perform this method.

Compression based reduced memory video decoder

Abstract: A compressed video decoder receives a stream of compressed video data in a channel buffer, and decompresses it. A portion of the decompressed video data is recompressed as intraframe data to serve as an anchor frame. The intraframe data is motion compensated, and then a reference picture is selected. The reference frame data is decompressed. Next, a region of interest is selected and stored in display memory for the decompressed form. The recompression of data to be used as an anchor frame can be done in a substantially lossless manner by using DCT with the same quantization matrix, quantizer scale, and field/frame coding type.

Method and apparatus for decoding B frames in video codecs with minimal memory

Abstract: An MPEG decoder system and method for decoding frames of a video sequence The MPEG decoder includes frame reconstruction or decoder logic which operates to reconstruct a bi-directionally encoded (B) frame with minimal memory requirements The MPEG decoder operates to decode or reconstruct the frame twice, once during each field display period The picture reconstruction unit operates to decode or reconstruct the B frame twice, once each during a first field time and a second field time The first field time substantially corresponds to the time when the first or top field of the picture is displayed, and the second field time substantially corresponds to the time when the second or bottom field of the picture is displayed This obviates the necessity of storing the reconstructed B frame data, thus reducing memory requirements The present invention also eliminates the storage requirement of the B-frame without substantially increasing the bandwidth required from external semiconductor memory when compared to prior art methods

Method and apparatus for bit rate control in a digital video system

Abstract: In a video image compression and transmission system, quantization parameters for a block transform based video compression algorithm can be controlled by a quantizer selector so as to control compressed video frame size. The selection of the appropriate quantization parameter for the nth macroblock of a current frame is based on the cumulative number of compressed bits appearing in the first n-1 macroblocks of a current frame and a previous frame. By controlling the quantization parameter is such a manner, the overall system reacts more quickly to changes in complexity in the video sequence and allocates bits more accurately to different parts of the video frame according to a past history of bit allocation. To efficiently utilize the bandwidth of a transmission medium (such as POTS), a bit count of the contents of the transmit buffer is sent to a buffer regulator in a video controller where it is compared to a low water mark threshold. If the bit count falls below the threshold, an uncompressed video frame is scheduled for compression by a video compressor. By using the low water mark threshold, latency in the overall system is reduced and an efficient use of transmission medium bandwidth is achieved.

Subsequent frame variable data rate indication method for various variable data rate systems

Abstract: In a synchronous fixed frame boundary system with variable data rates, a transmitter (10) inserts into a current frame an indication of the data rate of the next frame. As a station modem (18) assembles a current frame for encoding, the station modem (18) inserts a rate indication for the subsequent frame in accordance with information from a vocoder (22) and CPU (20) of the appropriate data rate for the subsquent frame. On the receiver (30) side, rather than decoding multiple times to determine the appropriate data rate for every frame, the receiving station modem (38) discovers the rate of each frame subsequent to the first frame.

Method and apparatus for detecting a point of change in a moving image

Abstract: A system for detecting a point of change between video shots from a video having a plurality of succeeding frames. The system includes video playback apparatus for playing a video chronologically one frame at a time, and a display for displaying the video. The a processing device for calculating a feature quantity of video image data for each frame, determining a first correlation coefficient between a feature quantity of a current frame and a feature quantity calculated from an immediately preceding frame and determining a second correlation coefficient between the feature quantity of the current frame and a feature quantity of at least two frames preceding the current frame, and indicating on the display a point of change between video shots when the first correlation coefficient and the second correlation coefficient are out of predetermined allowable ranges. The correlation coefficients of each frame is stored and can be used by the processing device to dynamically change a reference used for detecting a point of change between video shots. The change in the reference is performed based on the stored correlation coefficients or feature quantities of past frames.

Variable rate transmission method, transmitter and receiver using the same

Abstract: A variable rate transmission method that can vary the transmission rate of data. A transmitting side supplies a transmitted data sequence to an error detecting encoder 105 and a frame memory 103. The frame memory 103 stores data of a variable length to be transmitted in one frame. The error detecting encoder 105 calculates an error detecting code (such as CRC code) for each frame of the transmitted data. A multiplexer 104 adds the calculated error detecting code ahead of the transmitted data to place it at the initial position of the frame, and sequentially outputs the data sequence frame by frame. A receiving side calculates an error detecting code of the data in each transmitted frame in the same manner as the transmitting side, and compares the calculated error detecting code with the error detecting code at the initial position of the frame. The end bit of the frame data is decided as a position at which the two error detecting codes coincide. This makes it possible to transmit variable length data without informing the receiving side of the data length in each frame. This is equivalent that the transmission rate can be varied freely.

Apparatus and method for managing a frame buffer for MPEG video decoding in a PC environment

Abstract: A four-buffer MPEG decoder is provided for decoding MPEG video frames. A four-buffer frame controller and control method manage the four frame buffers including decoding, displaying and discarding of I-frames, P-frames and B-frames so that video data decoding is accelerated. The four-buffer frame controller and control method frees one frame buffer when the frame buffer contains obsolete data, defined as data which is no longer useful for decoding additional frames and for which storage is not necessary for displaying pictures in a correct temporal order. One example of an obsolete frame is a B-frame that is displayed. Another example is a P-frame for I-frame which is no longer used for motion compensation and has been displayed.

Method and apparatus for block-based motion estimation

Abstract: This disclosure provides a method of block-based motion estimation used in video compression. The compression process, derives change data for a new frame of data (with respect to a reference frame) by first dividing the frame structure into data tiles (or data blocks) of identical size. Each tile in the new frame is compared to a localized window (about the tile's expected position) in the reference frame to search for a best fit, and thereby provide motion data for the particular tile. Once the best fit is determined, motion-compensated difference data is determined, and stored with the motion data for each tile to complete the process. To achieve computation efficiency, each tile under analysis is preferably converted to single-bit value data and searching and comparisons are performed based on such transformed single-bit data. The single-bit data is computed by convolving the original image data with a low-pass filter to obtain a threshold matrix. The original image data is then compared with the threshold matrix, and converted to single-bit values in dependence on whether the values of the data exceed counterparts in the threshold matrix. Comparison is performed using an exclusive-or function and bit-summation of results.

Variable rate transmitting method, and transmitter and receiver using it

Abstract: A variable rate transmitting method in which the data transmission rate can be changed freely. The transmitting side supplies a transmission data sequence to an error detection encoding circuit (105) and a frame memory (103). The frame memory (103) holds variable-length data which are transmitted in frames. The error detection encoding circuit (105) calculates error detecting codes (for example, CRC codes for every frame of the transmission data). A multiplexing circuit (104) adds the calculated error detecting codes to the frame immediately before the transmission data, namely, at the head of the frame, and successively outputs data sequences one frame by one frame. On the receiving side, the error detecting codes of the data in the transmitted frame are calculated in the same way as that on the transmitting side and the calculated error detecting codes are compared with the error detecting codes at the head of the frame. The position where both error detecting codes coincide with each other is determined to be the last bit of the data in the frame. Therefore, variable-length data can be transmitted without informing the receiving side of the data length of each frame. This is equivalent to the case where the transmission rate can be freely changed.

Method and apparatus for picture coding and method and apparatus for picture decoding

Abstract: An excellent picture coding scheme, adapted to suit any particular picture movement, divides an input picture data into two-dimensional small blocks including frame small blocks and field small blocks, then calculates the block activity of the frame and field small blocks. By comparing field block activity with frame block activity, a total of two-dimensional blocks indicating that field block activity is greater than frame block activity is calculated. When the total calculated above is equal to or greater than a predetermined value, frame unit coding is performed. When the total calculated above, however, is below the predetermined value, the frame is divided into odd-and even-numbered fields and a field unit coding is performed for the frame.

Frame erasure compensation techniques

Abstract: In a speech coding system which encodes speech parameters into a plurality of frames, each frame having a predetermined number of bits, a predefined number of bits per frame are employed to transmit a speech parameter delta The speech parameter delta specifies the amount by which the value of a given parameter has changed from a previous frame to the present frame According to a preferred embodiment disclosed herein, a speech parameter delta representing change in pitch delay from the present frame to the immediately preceding frame is transmitted in the present frame, and the predefined number of bits is in the approximate range of four to six The speech parameter delta is used to update a memory table in the speech coding system when a frame erasure occurs

Varied frame rate video

Abstract: A method and apparatus wherein raw video and audio are captured and digitized and stored at a frame rate different from the playback stand frame rate. The audio and the video are separated, the video as a frame and the audio as a data stream wherein there are time links to the video frame. The playback entails multiplexing the video and the audio, and where the multiplexer is operating at the standard frame rate synchronization is not maintained. The present invention discloses modifications of the audio header and/or data to accommodate the differences in the multiplexing frame rate and the original capture frame rate. If the originating frame rate is one half the standard then the audio header information is changed to indicate that twice as much audio information exists, whereupon the multiplexer will take the video frames and properly multiplex the proper and therefore synchronized audio data. For example, the existence of 150 video frames originally taken at 15 fps or 10 seconds worth of video, causes the multiplexer and the associated software to presume that the 150 frames were taken at 30 frames per second and the 5 seconds worth of audio was required. The header is changed to indicate there is half as much audio or five seconds worth, which is the correct value. The resulting multiplexing of video and audio is in synchronization.

Method of video encoding by accumulated error processing and encoder therefor

Abstract: The present invention relating to a method of video coding associated with processing accumulated errors and a encoder therefor, the method comprising the steps of: (a) generating motion vectors of an input image in a predetermined unit and the difference image between an image of filtering a motion-compensated image on a reconstructed previous frame and the input image on current frame, and then performing discrete cosine transform (DCT), quantization and variable length coding on the difference image; (b) generating the motion-compensated image on the reconstructed previous frame from the reconstructed previous frame and the motion vectors; and (c) filtering off accumulated errors while preserving the edges within the motion-compensated image on the reconstructed previous frame. Therefore, random distributed noises due to accumulated errors can be removed and bit generation amounts by filtering off random accumulated errors with a high frequency characteristics before coding can be reduced.

Scene change detecting device

Abstract: A scene change detecting device of a relatively small circuit scale is capable of accurately detecting a scene change using data which have already been compressed and coded. The scene change detecting device has an extracting circuit for extracting frame type information, frame identification information, and block type information from a coded signal of a moving picture which has been processed by block-adaptive interframe predictive coding. The block type information is sent to a counter, and the frame type information and the frame identification information are sent to a frame estimating circuit. Based on the block type information, the counter counts macroblocks with respect to each of the types of predictive processes over one frame. The frame estimating circuit estimates a frame immediately following a scene change based on the counted number of macroblocks with respect to each of the types of predictive processes, the frame type information, and the frame identification information.

Image information processor

Abstract: An image information processor for a liquid crystal display by a digital driver is intended to provide image display near to a source image by making continuous-tone image display by pseudo representations. The image information processor generates L-bit image display data based on P-bit source image data, L being less than P. Added to source image data of a picture element of the Nth frame (where N is a natural number of 2 or greater) is error data corresponding to the picture element of the (N-1)th frame at the same position as the picture element of the Nth frame. The high-order L bits of the P-bit data resulting from the addition are used as image display data of the picture element of the Nth frame and at least one bit of the remaining low-order bit data is held as error data of the picture element of the Nth frame. Thus, error data of a picture element of one frame is added to the same picture element of the next frame, thereby lessening the brightness difference between both the picture elements and smoothing a time change in brightness.

System and method for representing a video sequence

Abstract: A system and method is disclosed for representing each frame in a sequence of video frames in terms of a correlation between successive video frames. The system and method of the present invention segment the current video frame data and a previous frame into blocks for which motion vectors and/or affine mapping coefficients may be determined. The motion vectors may then be used to represent the current frame data without requiring periodic transmission of a base, self-referential frames of video data. Preferably, the compressor which generates the codewords to represent the current frame data subdivides the blocks of the current frame and previous frame buffers to further evaluate motion factors which may be used to represent current frame data. This evaluation is performed in a recursive manner and preferably includes evaluation of a bit rate/distortion factor to optimally select codewords for representing the current frame data. The bit rate parameters are generated by an entropy encoder which encodes the codewords generated by a compressor in a known entropy coding manner, such as Huffman encoding. Preferably, the decompressor for regenerating previous frames of video data includes a weighted mask filtering of the regenerated frames to reduce blocking artifacts in the regenerated frame data.

Video compression interface

Abstract: An apparatus and method for asynchronous compression of video information in a computer system The apparatus includes data structures and functions for specifying synchronous or asynchronous compression of at least one uncompressed video frame, for specifying latency in the compression of the at least one uncompressed video frame, for initiating asynchronous compression of the at least one uncompressed video frame by a video compressor, for asynchronously retrieving from the video compressor at least one compressed video frame, for terminating asynchronous compression of the at least one uncompressed video frames by the video compressor, and for immediately retrieving from the video compressor the at least one compressed video frame when the at least one compressed video frame remains unretrieved after asynchronous compression is terminated

Confidence and frame signal quality detection in a soft decision convolutional decoder

Abstract: Received signals are processed (111) to generate confidence measure signals. Before these are forwarded to a soft decision decoder (116), correction factors are applied, e.g. by a look-up table (113), to reduce the extent to which the signals differ from a logarithmic representation of the error statistics of the received signals. Alternatively, or as well, the confidence measure signals may be summed (122) and thresholded (123) to provide a quality-indicating signal. In a coder, error check bits for a frame of bits to be convolutionally coded are generated from bits taken from the beginning and end of the frame but not the middle.

Motion compensated video processing

Abstract: A motion compensated video processing method includes encoding a video frame group I 0 -I 7 into one in which one frame I 0 of the group is a basic input frame or `I `frame, and the other frames B 1 -B 7 are derived from calculating the difference between a current frame and two neighbouring frames so as to form `B ` frames. The `B ` frames are derived in a logarithmic progression. The `B ` frames are quantized by applying selective quantization weighting to predetermined input frames. Encoding can be of the forms described as open loop or closed loop, preferably open loop. Decoding can be achieved by applying inverse quantization weighting to the input frames prior to selective summnation in order to achieve a constant reconstruction error for each frame even though quantization errors carry over from each reconstruction stage.

Apparatus for and method of providing interrupts to a host processor in a frame receiving system

Abstract: In an interrupt-driven data frame stream receiver, an interrupt to a host processor is applied by an interrupt system that comprises a timer logic circuit, an address match circuit, a data frame counter, a cyclic redundancy check (CRC) logic circuit, a frame length logic circuit, and a frame content detector. Each of these circuits is coupled to an OR gate. A Start Frame Delimiter (SFD) detector is connected to the inputs of the other circuits. The interrupt system provides the interrupts when any one or more of the following conditions arise: 1) a time interval between successive frames in the data frame stream is equal to or greater than a predetermined value, 2) a predetermined maximum number of frames have been received, 3) a frame is not addressed to the receiver, 4) an error frame is received, 5) the length of a frame is less than a minimum preset value, 6) the length of a frame is more than a maximum preset value, 7) frames have been received for a time period that exceeds a predetermined time period, and 8) the content of a frame requests the frame to be handled in a prescribed manner.

Scene cut frame detector and scene cut frame group detector

Abstract: An input image and a reference image which are input to a contracted image processing unit, are converted into contracted image, which are constituted by average values of blocks of the input and reference frames. With respect to the contracted image, an inter-frame difference unit determines the inter-frame difference Dn between the input and reference images and the temporal change ΔDn in the inter-frame difference. A first judging unit judges, according to the above data, the input frame to be a non-cut frame, a cut frame or a cut frame candidate. When the input frame is judged to be a cut frame, an output unit effects a cut frame output, and then the next input frame processing is performed. With respect to a cut frame candidate, a chrominance histogram correlation unit calculates the correlation ρ of the color distribution of the chrominance histogram by using contracted image of chrominance signal. A second judging unit makes an overall judgment on the inter-frame difference Dn, temporal change ΔDn therein, and correlation ρ for the cut frame detection.

Method and system for locating a frame position in an MPEG data stream

Abstract: A method for locating a frame position in an MPEG data stream within a computer system is disclosed. MPEG standard is a set of defined algorithms and techniques for the compression and decompression of moving pictures (video) and sound (audio), and the formation of a multiplexed data stream that includes the compressed video and audio data plus any associated ancillary service data. Although the MPEG standard is extremely flexible, there is a fundamental deficiency associated with the packet-oriented nature of the MPEG format, and that is there being no information about the position of each video frame encoded in the data stream. Even though such information can be deduced from the byte-rate, but because the calculation of a frame position depends on a constant byte-rate, a problem may still arise when the byte-rate is non-existent, incorrectly encoded, or constantly changed due to the presence of several packs with varying rates. This invention presents an improved method for locating a frame position in the MPEG data streams regardless of the byte-rate condition.

Hierarchical motion estimation for interlaced video

Abstract: A system and method for performing motion estimation. Interlaced frames are reduced in resolution using field based scaling, and a first search is performed on the reduced resolution frames. This first search is performed by computing motion errors between the fields of a current frame and the fields of a reference frame to obtain vectors that approximately describe inter-field motion. These same motion errors are used to compute frame errors, which are used to obtain vectors that approximately describe inter-frame motion. The approximate field and frame vectors are then refined to get more precise vectors.

Gradation display method

Abstract: PROBLEM TO BE SOLVED: To reduce spurious contour without impairing quality of a still picture and to improve display quality of a time serial picture with an indefinite degree of movement by varying a frame form in accordance with a degree of movement of a display object between two adjacent frames. SOLUTION: A movement detection circuit 85 reads video data DR, DG, DB of two adjacent frames from a frame memory 82 at each transmission cycle, detects a degree of movement of a display object, and outputs a signal S85 presenting whether or not the degree of movement exceeds a set value, namely, whether or not a spurious contour is in danger of being generated and supplies the signal to a controller 81, an address generator 83, and a picture processing circuit 84. When the degree of movement is relatively small, the detection signal is inactive, the following frame is operated as 'a normal frame', and when the signal is active, the following frame is processed as 'special frame' splitting a specific sub-frame to a field for preventing spurious contour. COPYRIGHT: (C)1998,JPO

Content-based indexing of images by coding levels defined as a function of reduced entropy

Abstract: A video image indexing system includes a method for identifying categories in compressed digital data for improved retrieval of video images. An encoder receives a frame and digitizes the image data while a sub-band coder codes each digitized frame at a plurality of transform levels to derive categorized data segments. Data segments are stored and then retrieved according to minimum entropy of Eigenvalues in a first transform level. Preferably, the compression is improved by creating a residual image of each frame by subtracting a predicted frame image from the actual frame image. This compression of data enhances the recreation of lossless video images at the receiver.