A computer-implemented method of image generation that makes efficient use of reprojective techniques to reduce the cost of image generation. The method employs a technique of primitive reprojection in which convex graphics primitives are the reprojected elements. The visibility of elements known to be visible in a previous frame is first determined by transformation and depth-comparison rasterization of these elements. Regions of the image that may contain newly visible elements are located by occlusion-exposure transitions in the depth buffer and from incremental view volume motion. In these regions a depth-prioritized, data-access method of visible surface determination, spatial-subdivision ray casting, is employed to identify newly visible primitives which are added to the list of previously visible primitives for rasterization. The method employs a system of classifying objects based on their dynamic occlusive properties to increase the accuracy, efficiency and versatility of the reprojective approach. Because the method employs a hybrid approach to visible surface determination in which newly visible primitives are identified for each frame it can be implemented as a graphics server employing an efficient on-demand, progressive geometry transmission protocol for client-server image generation. This client-server system employs a method of visibility event encoding in which data representing newly visible and newly invisible primitives for each frame are transmitted to a client unit which is a conventional graphics display system. The visibility event codec method can also be used to encode and store information representing a computer animation for later interactive playback.

System and method of image generation and encoding using primitive reprojection

A computer implemented method of image generation that employs adaptive, progressive, perception-based spatio-temporal importance sampling to reduce the cost of image generation. The method uses an adaptive approach to sampling which employs refinement criteria based on specific spatio-temporal limits of human vision. By using these refinement criteria the method produces an image with a spatio-temporal structure that is closely matched to the spatio-temporal limits of human vision. Using one sampling criteria the spatial sampling density is adjusted in proportion to the sampled region's exposure duration in a manner that substantially reflects the visual systems's increase in acuity as a function of exposure time. Using other criteria the spatial and temporal sampling frequencies of a region of the image stream are adjusted based on the measured or estimated retinal velocity of the sampled element in a manner that substantially reflects both dynamic visual acuity limits of vision and the critical temporal sampling frequency for the perception of smooth motion. The method includes image parallel, shared-memory multiprocessor implementations based on sample reprojection and primitive reprojection, the latter using a technique of adaptive rasterization. In these implementations the natural temporal image coherence and temporal visibility coherence of the image stream produce a temporal locality of data reference that enhances performance of the system. Because temporal image coherence and temporal visibility coherence increase the spatial resolving performance of the human visual system, the performance of the present method parallels the performance of human vision making performance degradations relatively invisible to the user.

System and method of perception-based image generation and encoding

A pipeline video decoder and decompression system handles a plurality of separately encoded bit streams arranged as a single serial bit stream of digital bits and having separately encoded pairs of control codes and corresponding data carried in the serial bit stream. The pipeline system employs a plurality of interconnected stages to decode and decompress the single bit stream, including a start code detector. When in a search mode, the start code detector searches for a specific start code corresponding to one of multiple compression standards. The start code detector responding to the single serial bit stream generates control tokens and data tokens. A respective one of the tokens includes a plurality of data words. Each data word has an extension bit which indicates a presence of additional words therein. The data words are thereby unlimited in number. A token decode circuit positioned in certain of the stages recognizes certain of the tokens as control tokens pertinent to that stage and passes unrecognized control tokens to a succeeding stage. A reconfigurable decode and parser processing means positioned in certain of the stages is responsive to a recognized control token and reconfigures a particular stage to handle an identified data token. Methods relating to the decoder and decompression system include processing steps relating thereto.

Multistandard video decoder and decompression system for processing encoded bit streams including a video formatter and methods relating thereto

A video frame arithmetical context adaptive encoding and decoding scheme is presented which is based on the finding, that, for sake of a better definition of neighborhood between blocks of picture samples, i.e. the neighboring block which the syntax element to be coded or decoded relates to and the current block based on the attribute of which the assignment of a context model is conducted, and when the neighboring block lies beyond the borders or circumference of the current macroblock containing the current block, it is important to make the determination of the macroblock containing the neighboring block dependent upon as to whether the current macroblock pair region containing the current block is of a first or a second distribution type, i.e., frame or field coded.

Video frame encoding and decoding

When dividing inputted image data to be coded into image data of a plurality of small regions which are adjacent to each other and coding the image data of an objective small region to be processed among the image data of the plurality of divided small regions which are adjacent to each other, reconstructed image data of a reproduction small region adjacent to the image data of the objective small region to be processed is used as image data of an intra-frame prediction small region of the objective small region to be processed, the image data of the intra-frame prediction small region is used as image data of an optimum prediction small region and image data of a difference small region which are differences between the image data of the objective small region to be processed and the image data of the optimum prediction small region is generated. Then, the generated image data of the difference small region is coded and outputted, and then the coded image data of the difference small region is decoded, so that the reconstructed image data of the reproduction small region is generated by adding the decoded image data of the difference small region to the image data of the optimum prediction small region.

Image predictive coding method

A sequential process-pipeline (12) has a first processing stage (30) coupled to a CODEC (24) through a plurality of buffers, including an image data input buffer (28), an image data output buffer (26), and an address buffer (34). The address buffer stores addresses, each of which identifies an initial address of a block of addresses within an image memory (22). Each block of addresses in the image memory stores a block of decompressed image data. A local controller (18) is responsive to the writing of an address into the address buffer to initiate the operation of the CODEC to execute a Discrete Cosine Transformation Process and a Discrete Cosine Transformation Quantization Process.

A process-pipeline architecture for image/video processing

A system and method for improving the quality of decoded images, without any cost to compression efficiency, in both progressive still frame and motion video applications wherein the images are coded with intraframe transform based coding techniques, such as Discrete Cosine Transform (DCT) coding. Prediction of the AC components is utilized and operations are performed on the AC coefficient prediction in the decoder portions of the system in combination with a thresholding technique. Input images are subdivided into blocks of pixels, which are changed into blocks of coefficients by forward transform coding resulting in DC coefficients, representative of the average value in a block of pixels, and AC coefficients, representative of harmonic frequencies in a block of pixels. AC coefficient prediction is performed using both DC and AC coefficients from the block, whereupon the predicted coefficient values are thresholded and the thresholded coefficient values are added to their corresponding transform coefficient values for the given block. These steps are repeated for all of the blocks of the image and the image representation is recovered by applying an inverse transform operation to the results of all of the additions for all of the blocks of the image.

Transform coding using coefficient prediction techniques

A sequential process-pipeline has a first processing stage (30) coupled to a CODEC (24) through a plurality of buffers, including an image data input buffer (28), an image data output buffer (26), and an address buffer (34). The image data input buffer stores, for each block of image data, control information for controlling the processing of an associated block of image data. The address buffer stores addresses, each of which identifies an initial address of a block of addresses within an image memory (22). A local controller (18) is responsive to the writing of an address into the address buffer to read the control information for a block to be processed, and to initiate the operation of the CODEC, in accordance with the read-out information, to execute a Discrete Cosine Transformation Process and a Discrete Cosine Transformation Quantization Process.

Sequential pipelined processing for the compression and decompression of image data

We describe a motion compensated hybrid DCT/DPCM video compression scheme, which has been proposed for the storage of compressed digital video. The scheme, an extension of a current CCITT/ISO proposal for compressing still-frame images, differs from other hybrid DCT proposals in several ways. (1) Reduction of DCT block-artifacts by implementing inverse AC-quantizers which are of finer granularity than the corresponding encoder quantizers. This process, which we label ‘AC-Correction’, is applied to intraframe coded frames. (2) Entropy coding by means of an adaptive binary arithmetic coder. Arithmetic coding results in better compression efficiency than Huffman coding, thus improving image quality when a fixed bandwidth channel is available. (3) Non-linear loop filtering that preserves edge definition. (4) Tight tolerances in the rate control of groups of frames.

DCT coding for motion video storage using adaptive arithmetic coding

Systems and methods are provided for allocating bits to pictures in accordance with the bit allocation constraints for operation at both constant and variable bit rates A statistical complexity measure which is an estimate of the average encoding complexity of the entire data is updated after encoding each picture This set of parameters, along with an estimate of the encoding complexity of the current picture and the desired average coding rate is used to allocate target bits for encoding the current picture One method of allocating bits to a picture is to used the statistical complexity measure to vary the instantaneous rate of encoding and allocate bits for this picture so as to generate encoded data at this rate Another method is to modulate the target generated by a conventional encoder using the current level of the Virtual Buffer Verifier and the deviation of the estimated encoding complexity of the current picture from the statistical complexity measures

Thomas Mccarthy

Papers

A process-pipeline architecture for image/video processing

Transform coding using coefficient prediction techniques

Sequential pipelined processing for the compression and decompression of image data

DCT coding for motion video storage using adaptive arithmetic coding

Single pass target allocation for video encoding