Showing papers in "Signal Processing-image Communication in 1991"

Object-oriented motion estimation and segmentation in image sequences

Abstract: This contribution presents a method for segmenting video scenes hierarchically into several differently moving objects and subobjects. To this end, both contour and texture information from the single images and information from successive images is used to split up a scene into various objects. Furthermore, each of these objects is characterized by a transform h ( x,T ) with a parameter vector T which implicitely describes the surface shape and the three-dimensional motion of the objects in the scene. In order to estimate T of these transforms, an efficient algorithm is introduced. Thus, we obtain an object-oriented segmentation and a prediction of the image contents from one image to the next, which can be used in low bit-rate image coding.

3D-television: A survey of recent research results on subjective requirements

Abstract: At present the foundations are being laid out for the implementation of three-dimensional television. Apart from the engineering problems to be solved there are a number of questions connected with human factors which have to be answered. These questions are related to fundamental requirements with regard to certain system features such as screen size, viewing distance and the spatio-temporal resolution of the image in the three dimensions of space. Since the implementation of data-compressing techniques is essential for the recording and transmission of 3DTV picture signals, it is necessary to show the possibilities of irrelevance reduction as well as to define the demands made on the quality of reconstructed images. The present paper summarizes the latest results of research and shows in which fields work must be carried out on hitherto unanswered questions.

Constrained disparity and motion estimators for 3DTV image sequence coding

Abstract: This paper presents two-dimensional motion estimation methods which take advantage of the intrinsic redundancies inside 3DTV stereoscopic image sequences. Most of the previous studies extract, either disparity vector fields if they are involved in stereovision, or apparent motion vector fields to be applied to motion compensation coding schemes. For 3DTV image sequence analysis and transmission, we can jointly estimate these two feature fields. Locally, initial image data are grouped within two views (the left and right ones) at two successive time samples and spatio-temporal coherence has to be used to enhance motion vector field estimation. Three different levels of ‘coherence’ have been experimented subject to the crucial assumption that an initial calibration of the stereoscopic sensors provides us with geometric change of coordinates for two matched features.

A three camera approach for calculating disparity and synthesizing intermediate pictures

Abstract: This article presents a method for stereo analysis and picture synthesis which allows the calculation of intermediate pictures from a virtual camera in a position between three actual cameras. Determining the depth (disparities) is the focal point here. We describe a segment-matching method for determining them which uses the principle of maximal cliques and performs a motion estimation for picture sequences. This procedure can achieve the data reduction necessary for a 3D-TV system. Synthesized intermediate pictures from natural scenes are shown.

1.15 Mbit/s coding of video signals including global motion compensation

Abstract: A coder for television video signals working at a bit-rate of 1.15 Mbit/s is presented. It uses a special coding structure facilitating trick modes and a separated global and local motion compensation. Global motion is described by a central zoom and pan model. For the estimation of global motion a frame matching algorithm is explained. The advantages of using global motion parameters are given by the reduction of the amount of coded motion information as well as the reduction of regions to be replenished in the predicted pictures. Experimental results show that for the given bit-rate of 1.15 Mbit/s the quantization step size for prediction error coding can be reduced by the factor three in case of existing global motion. Also at the decoder side global motion parameters are used for a synthesis of omitted frames resulting in a higher resolution in time for global motions.

Optimized quantization for image subband coding

Abstract: Subband coding of images, which can be seen as an extension of orthogonal transformations, has been introduced by Woods and O'Neil. In their paper, the subbands are further encoded by DPCM. In this study, it is proposed to use subband coding as a stand alone coding technique. The emphasis is put on the question of subband quantization. A methodology is proposed where the quantization steps of the subbands are computed so as to minimize a quantization noise power weighted by a sensitivity function of the eye. The methodology which is proposed is also valid for unequal size subbands.

Classified block truncation coding-vector quantization: An edge sensitive image compression algorithm

Abstract: Block truncation coding-vector quantization (BTC-VQ) is an extremely simple non-adaptive block-based image compression technique. It has a relatively low compression ratio; however, the simplicity of the algorithm makes it an attractive option. Its main drawback is the fact that the reconstructed pictures suffer from ragged edges. In this paper we show that by allowing the algorithm to adapt to the local picture statistics and by paying particular attention to the nature and reproduction of edges in the picture we are able to substantially improve the visual picture quality and at the same time allow for a moderate increase in the compression ratio.

A comparison of vector quantization techniques in transform and subband coding of imagery

Abstract: Transform image coding is considered using scalar quantization, pyramid vector quantization (PVQ), weighted pyramid vector quantization (WPVQ) and generalized Lloyd algorithm vector quantization (VQ). Subband image coding is considered using scalar quantization and pyramid vector quantization. A modification to the PVQ encoding algorithm is described and shown to provide improvement in encoded image peak signal-to-noise ratio (PSNR) of 0.5 to 1.0 dB. The WPVQ encoding algorithm is described and an adaptive WPVQ image coder is developed which provides improvements of up to 1.5 dB over the PVQ coder. In a simulation study, at low bit-rates (0.5 bit/pel) the transform VQ coder generally provided the largest PSNR, although the PVQ subband coder was superior for some images, and th was competitive for most images. At larger encoding rates (for which, because of complexity, the generalized Lloyd algorithm VQ was not considered) the scalar quantizer subband coder generally yielded larger PSNR than did the scalar quantizer transform coder. The PVQ transform and subband coders provided similar values of PSNR, but the subband coded image had less blockiness than the transform coded image. At an encoding rate of 0.5 bit/pel, the subband coders and the full search VQ and adaptive WPVQ transform coders are roughly an order of magnitude more complex than the PVQ and tree-searched VQ transform coders.

Two-layer video coding for ATM networks

Abstract: This paper highlights the cell loss problem associated with coding video for variable bit-rate transmission over asynchronous transmission mode networks. Countermeasures are discussed from which a 2-layer coding algorithm has been chosen for a more detailed study. Simulation results are presented for a 2-layer codec using a constant bit-rate base layer and a variable rate enhancement layer compared to a 1-layer model. These results are then used further to investigate achievable network loadings.

DCT-based embedded coding for packet video

Abstract: We present an image compression algorithm that is useful for transmitting video over packet networks. The algorithm modifies those in references [4–6, 8, 10] by incorporating motion-compensated prediction. The low and high priority images can be computed in parallel, and the algorithm provides improved motion-compensated prediction of low-frequency discrete cosine transform (DCT) coefficients.

A two layers video coding scheme for ATM networks

Abstract: In the transmission of image sequences over an ATM network variable bit-rate coding techniques can be implemented to optimize performances and to improve channel sharing efficiency. On the other hand the use in ATM networks of a packet switching technique introduces some problems related to packet delay, cell loss and packet delay jitter. One possible solution to these problems is the transmission of the video information over the network with different levels of protection with respect to the packet losses. The information used to guarantee a video connection with minimum acceptable quality is transmitted in a very protected way. On the contrary, the other information necessary to increase the quality of the coded images up to the desired level is not intrinsically protected against data loss. In the paper two different realizations of this type of video coder are presented and their performances are compared.

Image reconstruction from partial subband images and its application in packet video transmission

Abstract: This paper addresses the problem of image reconstruction in a subband coding system when certain parts of one or several down-sampled sub-images are missing. By requiring that the sub-images produced from the reconstructed image be similar to those interpolated from the received sub-images, the loss recovery problem has been formulated as a quadratic optimization problem. Two reconstruction algorithms have been developed: a relaxational algorithm that achieves the optimal solution and a fast algorithm that leads to a sub-optimal solution. The interpolation scheme for the sub-images has been derived by characterizing each small image region by a texture or edge model. The proposed algorithms can be applied to any subband system and can accomodate various loss patterns. For the algorithm to work well, the analysis filters should have substantial overlap in their passbands such that the sub-images before down-sampling are correlated. Very good results have been obtained with some short kernel filter banks. The reconstructed image is satisfactory even when many parts of the low-low image is missing. It becomes unacceptable only if the lost regions contain certain periodic line structures which can cause Moire patterns in the down-sampled sub-images. The results of our investigation suggest that signal loss problem such as packet loss can be combatted by using subband systems with overlapping filters. Although the coding efficiency is reduced compared to the conventional subband system using non-overlapping filters, the more disastrous signal loss can be prevented.

Video subband VQ coding at 64 kbit/s using short-kernel filter banks with an improved motion estimation technique

Abstract: A motion compensated interframe prediction (MCIP) coder incorporating short-kernel subband filter-banks with an improved motion estimation technique is described. The coder is based on two-dimensional subband decomposition of the error image obtained after motion compensated interframe prediction. The luminance and chrominance subbands, generated after the decomposition process, form image vectors which are then quantized using vector quantization. The block matching method works well with low velocities and low noise content in the image frames. For a motion compensation estimation algorithm to cope with higher activity image frames, it is important that rapid movements, having high-amplitude motion vectors, are effectively recognized. It is also essential that the motion estimation performance does not degrade due to noise in the previously decoded frame. To achieve this, a combination of phase correlation and block search methods is applied sequentially to a hierarchy of low-pass filtered and subsampled images, while the motion vectors are progressively refined at each step of the process. Details of the proposed coding algorithm and simulation results operating at 64 kbits/s are presented.

Layered coding for ATM based video distribution systems

Abstract: This paper discusses packet loss and its protection in an asynchronous transfer mode (ATM) based video distribution system. Packet losses in ATM based networks have such a great impact on the design of coding algorithms and network architectures that they should be exhaustively discussed and resolved. In this paper, first basic configuration of the ATM based video transmission system and its packet-loss protection schemes are discussed. The DCT based layered coding scheme with packet priority classification is proposed as an effective packet-loss protection scheme. Burstiness characteristics of the broadcast video sources are evaluated and modeled to clarify statistical multiplexing performance and packet-loss properties. The quality degradation caused by the packet losses is also evaluated by the SNR, and the superior performance of the proposed layered coding scheme is verified.

Modeling activity in VBR video sources

Abstract: Variable bit-rate (VBR) video sources are significantly characterized by the statistics of ‘scene changes’, which determine the activity exhibited by the pictures. In a source coded with an interframe technique, in fact, this characteristic is related to the length of the emission periods having a very high bit-rate. It therefore represents a fundamental feature in evaluating performance of a network supporting video traffic. In this paper an emission process that is able to fit different activity measurements for VBR video sources characterized by a predefined bit-rate probability density function is introduced. Such a process is based on a suitable linear combination of continuous state autoregressive Markov processes. It is analyzed with respect to its capability to match the actual values of some temporal parameters which have been introduced to characterize the source. Its modeling power is compared with that of a single autoregressive Markov process. In particular is demonstrated that the process introduced here can model faster motion within a scene than a single autoregressive Markov process. Finally a case study is introduced to point out the flexibility of the proposed process.

Filter design for the interpolation of highly subsampled pictures

Abstract: In some of the proposed HDTV bandwidth reduction schemes, such as MUSE, HDMAC and wideband MAC, images are highly subsampled. Interpolation filters have to be applied for the reconstruction. Dependent on the subsampling grid, they are composed of several polyphase components. The gain of each polyphase component has to be one. It is also desirable that the polyphase component fulfill an additional constraint: In case the same filtering and subsampling process has to be applied twice or many times, the values of the picture elements selected by the subsampling process for transmission must not be affected by the applied filters. A method for construction of these filters is presented. The additional constraint and the implications from the recovery requirement are discussed and similarities to the Nyquist pulse-shaping criterion in digital signaling are pointed out.

Adaptive algorithms for image coding using vector quantization

Abstract: Vector quantization (VQ) is a powerful technique for low bit-rate image coding. The two basic steps in vector quantization are codebook generation and encoding. In VQ, a universal codebook is usually designed from a training set of vectors drawn from many different kinds of images. The coding performance of vector quantization can be improved by employing adaptive techniques. The applicability of vector quantization is, however, limited by its computational complexity. In this paper, we propose two adaptive algorithms for image vector quantization which provide a good compromise between coding performance and computational complexity resulting in a very good performance at a reduced complexity. In the first algorithm, a subset of codewords from a universal codebook is used to code an image. The second algorithm starts with the reduced codebook and requires one iteration to adapt the codewords to the image to be coded. Simulation results demonstrate the gains in coding performance and the savings in computational complexity.

Universal VBR videocodecs for ATM networks in the Belgian Broadband Experiment

Abstract: This paper deals with digital television coding schemes for transmission on ATM networks. The proposed TV codec aims to reduce the incoming TV bit-rate of 216 Mbit/s by a factor of 6 to 100 to eliminate most of the redundant information. Most of the first videocodecs were designed to transmit on STM channels at constant bit-rate (CBR); therefore, this paper intends to present all the adaptations to be realized to transform this type of videocodecs into a variable bit-rate (VBR) universal videocodec suitable in the ATM environment for both distributive and interactive video services. Throughout the paper, the hardware (VLSI) videocodec realized for the ATM Belgian Broadband Experiment is taken as an example of a VBR videocodec.

A stereoscopic television system (3D-TV) and compatible transmission on a MAC channel (3D-MAC)

Abstract: A new television service, in this case stereoscopic television or 3D-TV, cannot be credible unless there has been research into the analysis, transmission and restitution of 3D-TV signals. Specific studies of the quality of 3D images must give some idea of system efficiency, or even control and direct each of the above three stages. Analysis requires the use of pickup equipment capable of filming at least 2 views of the same scene. The quality of the pictures acquired in this way depends on the mechanical, optical, geometric and electronic parameters of the system. Disparity evaluation algorithms used for transmission purposes require prior knowledge of certain internal and external parameters. In this article, we propose transmission systems on a MAC type channel, which is divided into an analog and a digital section. The analog section transmits a video signal (625/50/2:1) that can be watched by any viewer with a conventional MAC receiver; this defines the notion of compatibility. The digital section is used to transmit, at a limited rate (of the order of 1 Megabit/sec), digital assistance data (‘DA’) which, in conjunction with the compatible picture, rebuilds the 3D image for a 3D receiver. Band reduction algorithms are based on frame sub-sampling and multiplexing techniques, motion and disparity compendation techniques. Their layout depends on the importance ascribed to the quality of compatible and 3D pictures, respectively, and on the output system. Display is a key element and is vital for the acceptance of 3D-TV by the general public. The present solution is the wearing of special glasses in order to synchronise the right (respectively left) eye and right (respectively left) view. However, there is no doubt that the future lies with systems that do not require special glasses, e.g. lenticular networks.

A video coding method considering cell losses in ATM-based networks

Abstract: This paper describes a video coding method which is capable of compensating for information losses due to cell discard. The proposed method separates picture elements into two groups by using quincunx subsampling. Separated picture elements are coded severally. Picture elements in one group are called higher priority pixels and their coded data are transmitted in higher priority cells, while ones in the other called lower priority pixels and their coded data are transmitted in lower priority cells. In case of network congestion, the proposed method prevents fatal degeneration in image quality by selectively discarding lower priority cells. Furthermore we take into consideration that higher priority cells are not always free from discard. As a countermeasure against losses of higher priority cells, we examine two schemes, one of which is called intrablock prediction and the other is cyclic refreshing. Experimental results show that the proposed method effectively suppresses degradation in image quality compared with conventional coding methods.

Variable bit-rate image sequence coding with adaptive quantization

Abstract: In this paper a variable bit-rate image sequence coding scheme for ATM will be described. The code bit data structure is fully compatible to the CCITT H.261 recommendation for a p × 64 kbit/s constant bit-rate codec. In order to keep the average bit-rate as low as possible without quality degradation, the quantization of the DCT coefficients will be adapted according to the visibility of quantization errors. Such errors are most easily visible in homogeneous areas of medium brightness, especially around objects in front of a homogeneous background. According to the Weber/Fechner law and the γ-characteristic of the cathode ray tube, the visibility of errors decreases for very bright or dark areas. Structured areas are also less sensitive in this respect. A segmentation of the images into critical and less critical areas will be performed by the determination of the local activity and the average brightness of homogeneous regions. Some experimental results will be reported and compared with constant quantization using standard sequences and a sequence taken from regular PAL television. Obviously, the savings which can be obtained by this method depend on the content of the scene, but for most of the sequences the average bit-rate could be reduced by more than 30% without quality degradation.

Variable bit-rate coding based on human visual system

Abstract: One of the merits with the introduction of variable bit-rate coding to moving pictures is the constancy of picture quality. However, considering the fact that conventional SNR cannot reflect the result of subjective assessment sufficiently, other quality measures should be considered to design the variable bit-rate coders. In this paper, weighted SNR (WSNR), which is calculated by weighting coding errors with the spatial frequency response of human vision, is introduced as a quality measure in place of SNR, and the variable bit-rate coder based on this measure is considered. Then, subjective assessment is carried out and the effectiveness of our approach is confirmed.

ATM video signal multiplexer with congestion control function

Abstract: ATM (asynchronous transfer mode) is a key technology in the construction of the broadband ISDN. In the ATM network, encoded video signals with burstiness are divided into short fixed length packets called cells, and can be transferred efficiently maintaining high quality by the statistical multiplexing of cells. However, in the congestion state, an appropriate congestion control is needed to avoid deterioration of transfer quality due to the loss of cells. This paper proposes a new congestion control scheme and evaluates characteristics of an ATM video signal multiplexer with the proposed congestion control function. In this paper, the amount of encoded video information per frame is approximated by the auto-regressive model, and the characteristics are evaluated by computer simulation. Particularly, the obtained circuit gain, the effect of congestion control and the optimum cell length are shown from the results of evaluation. Nearly double circuit gain can be obtained by the statistical multiplexing of cells and a throughput improvement of nearly 10% can be obtained by introducing the proposed congestion control in the multiplexing of cells.

Hardware trials for verifying recommendation H.261 on p × 64 kbit/s video codec

Abstract: ISDN deployment around the world is providing a good opportunity for realizing the long awaited visual telephone service. A worldwide unique video coding standard plays another essential role in achieving that goal. This paper highlights the hardware trials and their results which verified the recently established draft Recommendation H.261 for the sub-primary rate (64 kbit/s-2 Mbit/s) video coding. These trials were succesfully conducted in and among several countries using Flexible Hardware which is a test bed for verification and the optimized CCITT specifications.

Compatibility issues and selection devices for stereoscopic television

Abstract: A flickerless stereoscopic video system employing a time multiplexing technique using 120 fields per second has been developed and marketed by StereoGraphics Corporation. The product uses unmodified NTSC recording and transmission equipment operating at the nominally standard 60 fields per second and doubles the number of fields at playback. To improve image quality, an off-the-shelf scan converter is used to double the number of lines per field per eye. The Universal Camera Controller was developed to multiplex the signals from two unmodified cameras, fitting the composite within the existing NTSC bandwidth. The system, designed for industrial applications, comprises, from camera to monitor, an upwardly compatible approach. The design of a downwardly compatible system, on the other hand, would be appropriate for a consumer product. The question of compatibility with regard to all aspects of the broadcast service and video infrastructure, including existing consumer video tape recorders, needs to be considered. In addition, in order for an entertainment service to be viable, a low cost selection technique of high performance must be offered. The author believes that wireless active eyewear, using liquid crystal shutters, can be made available at prices attractive to the consumer.

A motion vector replenishment video codec for ATM networks

Abstract: A low bit-rate video codec for ATM networks is described. It is based on two-layer coding principles. The base layer comprises the motion vectors plus a strip of interframe coded video data. The remaining video data are coded by a second layer. Transmission of the base layer cells is assumed to be guaranteed. The required guaranteed channel rate can be as low as that needed for a speech signal, such that networks like Orwell can handle them equally. The second layer cells may be lost, if congestion arises. Simulation results demonstrate the performance of the codec for a range of cell loss rates from the second layer.

HDTV transmission system in an ATM-based network

Abstract: This paper proposes an ATM-based nationwide HDTV transmission and distribution network architecture and a picture coding algorithm, which are of key importance to high qualtiy HDTV transmission systems. This HDTV coding algorithm is based on sub-band coding, and its ability to maintain the high picture quality of the original is confirmed. Computer simulation results are presented in terms of bit-per-pel and the quality of the reconstructed picture. This paper also proposes a cell loss compensation method for video signals using random error correcting codes and an interleave structure in an ATM-based network. Analysis results show that the proposed cell loss compensation method reduced cell loss probability to the point that the decoded sequence can be regarded as cell loss free.

Stereoscopic display for design visualization

Abstract: Two experiments were performed to rank different types of display formats in terms of geometric information conveyed and perceived realism of objects. Display types tested were wireframe, wireframe with hidden lines removed (HLR), shaded solid, orthogonal views, stereoscopic wireframe, stereoscopic HLR and stereoscopic shaded solid. Prior to the running of the experiments a review of the current literature on stereoscopic rendering and display was conducted to provide guidelines for the creation of the stereoscopic images. The results of the geometric information experiment indicated that the orthogonal views display was judged inferior to both the non-stereo and stereo pictorial presentation displays and that the stereo displays were judged superior to the non-stereo ones. Individual preferences among subjects, however, varied widely. The results of the realism experiment were that the flat shaded stereo, HLR stereo and the flat shaded display types were judged to be equivalent and most realistic. The wireframe stereo, HLR and wireframe displays were judged to be equivalent and less realistic. The orthogonal views display was judged overwhelmingly to be the least realistic.

Piecewise linear motion-adaptive interpolation

Abstract: A new motion-adaptive interpolation algorithm is presented in this paper. Based upon the characteristics of typical video-conferencing scenes, a simple yet effective technique called background subtraction is proposed to refine the segmentation of the frame for interpolation. In this scheme, the outlines of missing frames are transmitted if necessary. The adaptivity depends on the magnitude of the representative vector, which is determined by the distribution of motion vectors on the outlines. At cost of small overhead data, the algorithm improves the edge integrity and the quality of reconstruction of the interpolated frame.

Hybrid coding with pre-buffering and pre-analysis in a software-based codec environment

Abstract: A concept for a software oriented video telephone codec compatible to CCITT draft recommendation H.261 is presented. An overview is given of the hybrid DPCM/transform coding which has been adopted for video telephone applications in digital networks at 64 kbit/s up to 2.048 Mbit/s data rate. The principle of an effective coding procedure is described leading to minimized and optimally distributed processing power while allowing high picture quality and minimum delay in order to retain interactivity of the video telephone service. The paper is focused on the coding strategy which is based on a pre-analysis of the input pictures in terms of change detection and structure analysis. Thus the processing power available can be efficiently utilized to get a constant picture quality at low hardware expense.