Showing papers by "Aggelos K. Katsaggelos published in 1996"

Rate-Distortion Based Video Compression: Optimal Video Frame Compression and Object Boundary Encoding

Abstract: From the Publisher: The book contains a review chapter on video compression, a background chapter on optimal bit allocation and the necessary mathematical tools, such as the Lagrangian multiplier method and Dynamic Programming. These two introductory chapters make the book self-contained and a fast way of entering this exciting field. Rate-Distortion Based Video Compression establishes a general theory for the optimal bit allocation among dependent quantizers. The minimum total (average) distortion and the minimum maximum distortion cases are discussed. This theory is then used to design efficient motion estimation schemes, video compression schemes and object boundary encoding schemes. For the motion estimation schemes, the theory is used to optimally trade the reduction of energy in the displaced frame difference (DFD) for the increase in the rate required to encode the displacement vector field (DVF). These optimal motion estimators are then used to formulate video compression schemes which achieve an optimal distribution of the available bit rate among DVF, DFD and segmentation. This optimal bit allocation results in very efficient video coders. In the last part of the book, the proposed theory is applied to the optimal encoding of object boundaries, where the bit rate needed to encode a given boundary is traded for the resulting geometrical distortion. Again, the resulting boundary encoding schemes are very efficient. Rate-Distortion Based Video Compression is ideally suited for anyone interested in this booming field of research and development, especially engineers who are concerned with the implementation and design of efficient video compression schemes. It also represents a foundation for future research, since all the key elements needed are collected and presented uniformly. Therefore, it is ideally suited for graduate students and researchers working in this field.

Spatially Adaptive Wavelet-B ased Multiscale Image Restoration

Abstract: Abstruct- In this paper, we present a new spatially adaptive approach to the restoration of noisy blurred images, which is particularly effective at producing sharp deconvolution while suppressing the noise in the flat regions of an image. This is accomplished through a multiscale Kalman smoothing filter applied to a prefiltered observed image in the discrete, separable, 2-D wavelet domain. The prefiltcering step involves constrained leastsquares filtering based on optimal choices for the regularization parameter. This leads to a reduction in the support of the required state vectors of the multiscale restoration filter in the wavelet domain and improv’ement in the computational efficiency of the multiscale filter. The proposed method has the benefit that the majority of the regularization, or noise suppression, of the restoration is accomplished by the efficient multiscale filtering of wavelet detail coefficients airdered on quadtrees. Not only does this lead to potential parallel implementation schemes, but it permits adaptivity to the local edge information in the image. In particular, this method changes filter parameters depending on scale, local signal-to-noise ratio (SNR), and orientation. Because the wavelet detail coefficients are a manifestation of the multiscale edge information in an image, this algorithm may be viewed as an “edge-adaptive” multiscale restoration approach.

Spatially adaptive wavelet-based multiscale image restoration

Abstract: In this paper, we present a new spatially adaptive approach to the restoration of noisy blurred images, which is particularly effective at producing sharp deconvolution while suppressing the noise in the flat regions of an image. This is accomplished through a multiscale Kalman smoothing filter applied to a prefiltered observed image in the discrete, separable, 2-D wavelet domain. The prefiltering step involves constrained least-squares filtering based on optimal choices for the regularization parameter. This leads to a reduction in the support of the required state vectors of the multiscale restoration filter in the wavelet domain and improvement in the computational efficiency of the multiscale filter. The proposed method has the benefit that the majority of the regularization, or noise suppression, of the restoration is accomplished by the efficient multiscale filtering of wavelet detail coefficients ordered on quadtrees. Not only does this lead to potential parallel implementation schemes, but it permits adaptivity to the local edge information in the image. In particular, this method changes filter parameters depending on scale, local signal-to-noise ratio (SNR), and orientation. Because the wavelet detail coefficients are a manifestation of the multiscale edge information in an image, this algorithm may be viewed as an "edge-adaptive" multiscale restoration approach.

Fast and efficient mode and quantizer selection in the rate distortion sense for H.263

Abstract: In this paper, a fast and efficient method for selecting the encoding modes and the quantizers for the ITU H.263 standard is presented. H.263 is a very low bit rate video coder which produces satisfactory results at bit rates around 24 kbits/second for low motion quarter common intermediate format (QCIF) color sequences such as 'mother and daughter.' Two major target applications for H.263 are video telephony using public switched telephone network lines and video telephony over wireless channels. In both cases, the channel bandwidth is very small, hence the efficiency of the video coder needs to be as high as possible. The presented algorithm addresses this problem by finding the smallest frame distortion for a given frame bit budget. The presented scheme is based on Lagrangian relaxation and dynamic programming (DP). It employs a fast evaluation of the operational rate distortion curve in the DCT domain and a fast iterative search which is based on a Bezier function.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Iterative algorithm for improving the resolution of video sequences

Abstract: This paper introduces an iterative and temporally recursive technique to improve the spatial resolution of a video sequence. Such iterative techniques have a number of advantages, among which are the ability to incorporate prior information into the restoration and the removal of the need for taking inverses of matrices. At each iteration, a residual (error) term is added to the current estimate of the high resolution frame. This residual term is based on a model which mathematically describes the relationship between the low resolution images and their corresponding high resolution images. This model incorporates the motion between the frames, and in the most general case, takes into account occlusions and newly uncovered areas. Experimental results are presented which demonstrate the capabilities of the proposed approach. Keywords: video enhancement , interpolation, motion compensation , subsampling

Method and system for estimating motion within a video sequence

Abstract: The present invention provides a method and system for estimating the motion within a video sequence. The invention provides very accurate estimates of both the displacement vector field, as well as, the boundaries of moving objects. The system comprises a preprocessor (102), a spatially adaptive pixel motion estimator (104), a motion boundary estimator (106), and a motion analyzer (108). The preprocessor (102) provides a first estimate of the displacement vector field, and the spatially adaptive pixel motion estimator (104) provides a first estimate of object boundaries. The motion boundary estimator (106) and the motion analyzer (108) improve the accuracy of the first estimates.

Iterative restoration of fast-moving objects in dynamic image sequences

Abstract: If a point on an object passes over two or more photoreceptors during image acquisition, a blur will occur. Under these conditions, an object or scene is said to move fast relative to the camera's ability to capture the motion. In this work, we consider the iterative restoration of images blurred by distinct, fast moving objects in the frames of a (video) image sequence. Even in the simplest case of fast object motion, the degradation is spatially variant with respect to the image scene. Rather than segmenting the image into regions where the degradation can be considered space invariant, we allow the blur to vary at each pixel and perform iterative restoration. Our approach requires complete knowledge of the blur point spread function (PSF) to restore the scene. The blur of fast moving object in a single frame is under specified. With the appropriate assumptions, an estimate of the blur PSF can be specified to within a constant scaling factor using motion information provided by a displacement vector field (DVF). A robust iterative restoration approach is followed which allows for the incorporation of prior knowledge of the scene structure into the algorithm to facilitate the restoration of difficult scenes. A bilinear approximation to the continuous PSF derived from the motion estimate is proposed to obtain results for real and synthetic sequences. We found this approach suitable for restoring motion degradations in a wide range of digital video applications. The results of this work reinforced the well known flexibility of the iterative approach to restoration and its application as an off-line image sequence restoration method.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A video compression scheme with optimal bit allocation between displacement vector field and displaced frame difference

Abstract: We address the fundamental problem of optimally splitting a video sequence into two sources of information, the displaced frame difference (DFD) and the displacement vector field (DVF). We first consider the case of a lossless motion compensated video coder (MCVC) and derive a general dynamic programming (DP) formulation which results in an optimal tradeoff between the DVF and the DFD. We then consider the more important case of a lossy MCVC and present an algorithm which solves optimally the bit allocation between the rate and the distortion. This algorithm is based on Lagrangian relaxation and the DP approach introduced for the lossless MCVC. We then present an H.263-based MCVC which uses the proposed optimal bit allocation scheme and compare its results to H.263. As expected, the proposed coder is superior in the rate-distortion sense.

Multichannel image identification and restoration using the expectation-maximization algorithm

Abstract: Previous work has demonstrated the effectiveness of the expectation-maximization algorithm to restore noisy and blurred single- channel images and simultaneously identify its blur. In addition, a gen- eral framework for processing multichannel images using single-channel techniques has been developed. The authors combine and extend the two approaches to the simultaneous blur identification and restoration of multichannel images. Explicit equations for that purpose are developed for the general case when cross-channel degradations are present. An important difference from the single-channel problem is that the cross power spectra are complex quantities, which further complicates the analysis of the algorithm. The proposed algorithm is very effective at restoring multichannel images, as is demonstrated experimentally.

An optimal segmentation encoding scheme in the rate distortion sense

Abstract: In this paper, we present two fast and efficient methods for the lossy encoding of object boundaries which are given as 8-connect chain codes. We approximate the boundary by a polygon and consider the problem of finding the polygon which leads to the smallest distortion for a given number of bits. We introduce two different approaches to find the best polygonal approximation when an additive distortion measure is employed. The first approach is based on Lagrangian relaxation and a shortest path algorithm. This scheme results in solutions which belong to the convex hull of the operational rate-distortion curve. The second algorithm is based on a tree pruning scheme and finds all optimal solutions. We present results of the proposed algorithms using an object from the "Miss America" sequence.

Method and apparatus for regenerating a dense motion vector field

Abstract: The present invention provides a method (300) and apparatus (100) for regenerating a dense motion vector field, which describes the motion between two temporally adjacent frames of a video sequence, utilizing a previous dense motion vector field. In this method, a spatial DVF and a temporal DVF are determined (302 and 304) and summed to provide a DVF prediction (306). This method and apparatus enables a dense motion vector field to be used in the encoding and decoding process of a video sequence. This is very important since a dense motion vector field provides a much higher quality prediction of the current frame as compared to the standard block matching motion estimation techniques. The problem to date with utilizing a dense motion vector field is that the information contained in a dense motion field is too large to transmit. The present invention eliminates the need to transmit any motion information.

Optimal lossy segmentation encoding scheme

Abstract: In this paper, we present a fast and optimal method for the lossy encoding of object boundaries which are given as 8-connect chain codes. We approximate the boundary by a polygon and consider the problem of finding the polygon which can be encoded with the smallest number of bits for a given maximum distortion. To this end, we derive a fast and optimal scheme which is based on a shortest path algorithm for a weighted directed acyclic graph. We further investigate the dual problem of finding the polygonal approximation which leads to the smallest maximum distortion for a given bit rate. We present an iterative scheme which employs the above mentioned shortest path algorithm and prove that it converges to the optimal solution. We then extend the proposed algorithm to the encoding of multiple object boundaries and introduce a vertex encoding scheme which is a combination of an 8-connect chain code and a run-length code. We present results of the proposed algorithm using objects from the 'Miss America' sequence.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Method and system for improved motion compensation

Abstract: The present invention provides a method (600) and system (100) for predicting a differential vector field. The method and system enable the detection and encoding of an area where motion compensating the past image frame to the current image frame, fails. Based on the DFD signal, the present invention detects regions where the motion compensation has failed (102). The boundaries of these regions are encoded and sent to the decoder (104). The intensity values contained in this region, by the current intensity frame, are also encoded and sent to the decoder. Based on the decoded region boundaries, the decoder decodes the intensity values and places them into the correct regions.

Method and apparatus for detecting occlusion

Abstract: The present invention provides a method and apparatus for detecting occluded areas in a video frame. A previous displacement vector field, DVF, is motion compensated (402) and used to provide an occlusion test parameter (404) which is compared to an optimal threshold (408) to detect occluded areas. The optimal threshold is calculated based on the previous DVF and a predetermined threshold (406).

Detection and encoding of model failures in very low bit rate video coding

Abstract: One of the challenging problems for most existing video codecs is the detection and encoding of the information pertaining to model failure areas, i.e., areas where the compensation of the motion is insufficient. The insufficient motion may result from several different reasons, such as uncovered background by moving objects, complex motion, etc. The existing approaches to detection and encoding of model failures are closely tied to the encoding scheme they are built in, particularly to the specific motion estimation algorithm used; therefore, generalization of these algorithms to other coding techniques is not possible. On the other hand, the efficient encoding of the position and the intensity field information in these areas is also very crucial to the performance of the very low bit rate codecs. The existing approaches fail to meet the target bit rates and satisfactory image quality. In this paper, a new method to detect the model failure areas is described. In this method, the model failure areas are detected based on a motion compensated prediction of the current frame independently of the motion estimation algorithm. Thus the proposed method can be used with any type of coding scheme. In addition efficient and robust encoding of the boundary and the intensity information is described. The simulation results demonstrate that with the described method, the requirements of very low bit rate coding can be satisfactorily met.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Detection and encoding of occluded areas in very low bit rate video coding

Abstract: One of the challenging problems that most existing video codecs face today is the encoding of the information pertaining to the occluded areas, i.e., the areas which are covered or uncovered by moving objects. The existing techniques necessitate the transmission of the position information of the occluded areas following the detection process, which can constitute a large overhead in bandwidth consumption. In addition, these detection techniques fail under noisy conditions. On the other hand, no effort was made to incorporate the spatio-temporal correlation that exists between motion fields of consecutive frames. A new method to detect the occluded areas is described. In this method, the decoder is given additional intelligence to extract the position information of the occluded areas, thus reducing the bandwidth needed to transmit the occlusion information significantly. According to the proposed method, the temporal correlation of the motion fields is exploited. The proposed method is robust under noisy conditions and provides a computationally simple solution.

Restoration of severely blurred high range images using compound models

Abstract: We examine the use of compound Gauss Markov random fields (CGMRF) to restore severely blurred high range images. For this deblurring problem, the convergence of the simulated annealing (SA) and iterative conditional mode (ICM) algorithms has not been established. We propose two new iterative restoration algorithms which extend the classical SA and ICM approaches. Their convergence is established and they are tested on real and synthetic images.

Optimal quad-tree-based motion estimator

Abstract: In this paper we propose an optimal quad-tree (QT)-based motion estimator for video compression. It is optimal in the sense that for a given bit budget for encoding the displacement vector field (DVF) and the QT segmentation, the scheme finds a DVF and a QT segmentation which minimizes the energy of the resulting displaced frame difference (DFD). We find the optimal QT decomposition and the optimal DVF jointly using the Lagrangian multiplier method and a multilevel dynamic program. The resulting DVF is spatially inhomogeneous since large blocks are used in areas with simple motion and small blocks in areas with complex motion. We present results with the proposed QT-based motion estimator which show that for the same DFD energy the proposed estimator uses about 30% fewer bits than the commonly used block matching algorithm.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Resolution enhancement of color video

Abstract: In this paper, an approach to improve the spatial resolution of color video is presented. Such high resolution images are desired, for example, in video printing. Previous work has shown that the most important step in achieving high quality results is the accuracy of the motion field. It is well known that motion estimation is an ill-posed problem. However, in processing color video, additional information contained in the color channels may be used to improve the accuracy of the motion field over the motion field obtained with the use of only one channel. In turn, this improvement in the motion field will be shown through several experimental results to significantly improve the estimation of a high resolution image sequence from a corresponding observed low resolution sequence.

Recursive map displacement field estimation and its applications

Abstract: We briefly describe some of our work on the use of stochastic models to describe the displacement vector field (DVF) in an image sequence. Specifically, autoregressive models are used which describe the abrupt transitions in the DVF with the use of a line process, but also result in spatio-temporally recursive structures. The use of such models in developing maximum a posteriori estimators for the DVF and the line process is subsequently described. Finally, the extension and application of the resulting estimator to the problems of object tracking, video compression and restoration of video sequences is reviewed.

Iterative regularized error concealment algorithm

Abstract: In this paper, we propose an iterative regularized error concealment algorithm. The coded image can be degraded due to channel errors, network congestion, and switching system problems. We may have therefore seriously degraded images due to information loss. When the structure of the image and video codec is hierarchical, the degradation may be worse because of the inter-dependence of the coded information. In order to solve the error concealment problem of compressed images, we use an iterative regularized algorithm. We analyze the necessity of an oriented high pass operator we introduce and the requirement of changing the initial condition when all the quantized DCT coefficients in a block are lost. Several experimental results are presented.

Spatially adaptive filtering for video encoding

Abstract: The present invention provides a method (200) and an apparatus (100) for spatially adaptive filtering for video encoding. The apparatus filters a video sequence prior the encoding process. The apparatus comprises a noise variance determiner (102), a local variance determiner (104), a noise visibility function determiner (106), a Gaussian kernel determiner (108), and a convolver (110). The apparatus removes noise directly from a Displaced Frame Difference, DFD, signal. This novel approach removes noise and miscellaneous high frequency components from the DFD signal without the introduction of the filtering artifacts characteristic of current techniques. By reducing the miscellaneous high frequency components, the present invention is capable of reducing the amount of information that must be encoded by the video encoder without substantially degrading the decoded video sequence.

Parameter estimation in regularized reconstruction of BDCT compressed images for reducing blocking artifacts

Abstract: High compression ratios for both still images and sequences of images are usually achieved by discarding information represented by block discrete cosine transform (BDCT) coefficients which is considered unimportant. This compression procedure yields images that exhibit annoying block artifacts. In this paper we examine the reconstruction of BDCT compressed images which results in the removal of the blocking artifact. The method we propose for the reconstruction of such images, is based on a hierarchical Bayesian approach. With such an approach image and degradation models are required. In addition, unknown hyperparameters, usually the noise and image variances, have to be estimated in advanced or simultaneously with the reconstructed image. We show how to introduce knowledge about these parameters into the reconstruction procedure. The proposed algorithm is tested experimentally.

Method and device for encoding/decoding a displaced frame difference

Abstract: The present invention provides a method (100, 700), device (400, 800) and microprocessor (500) for performing a computationally efficient iterative expansion of a displaced frame difference, DFD, image over a predetermined dictionary of modulated Gaussian functions for the purposes of video compression. The iterative expansion described in this invention decomposes the DFD image into a set of coefficients which represent the perceptually important areas of a video frame in a compact way. The resulting method, device and microprocessor serve to provide a means for very low bit rate coding/decoding of a video sequence.

Exploitation of Spatio-Temporal Inter-Correlation Among Motion, Segmentation and Intensity Fields for Very Low Bit Rate Coding of Video

Abstract: During the last decade there has been a dramatic increase in the number of the applications requiring video compression techniques. These applications, which range from high definition television (HDTV) and digital cable to video conferencing and picture phones, have varying bandwidth requirements. The challenging general problem is the representation of full motion video at very low bit rates (VLBR), possibly as low as 8 kbits/sec. The research that proposes workable solutions to the problem will directly impact the future developments and possible applications in the area, as well as, one of the functionalities of a new standard (MPEG-4). Potential applications include videophone, multi-media electronic mail, remote sensing, electronic newspapers, interactive multi-media databases, multi-media videotex, video games, interactive computer imagery, multi-media annotation, surveillance, telemedicine and communication aids for the hearing impaired.

Recent trends in image restoration and enhancement techniques

Abstract: Many digital consumer products which utilize digital images and video are on the horizon today. Items such as personal communication systems (PCS) which utilize video may provide a great deal of opportunity for the application of restoration ideas to coded images. It is important to obtain a perspective on the future of image restoration which considers the driving importance of consumer applications. Specific restoration techniques are described and various experimental restoration results with video and still images are demonstrated.

A very low bit-rate video codec with optimal trade-off among DVF, DFD and segmentation

Abstract: In this paper we present a theory for the optimal bit allocation among quad-tree (QT) segmentation, displacement vector field (DVF) and displaced frame difference (DFD) The theory is applicable to variable block size motion compensated video coders (VBSMCVC), where the variable block sizes are encoded using the QT structure, the DVF is encoded by first order differential pulse code modulation (DPCM), the DFD is encoded by a block based scheme and an additive distortion measure is employed We consider the case of a lossless VBSMCVC first, for which we develop the optimal bit allocation algorithm using Dynamic Programming (DP) We then consider a lossy VBSMCVC, for which we use La-grangian relaxation and show how an iterative scheme, which employees the DP-based solution, can be used to find the optimal solution We finally present a VBSMCVC, which is based on the proposed theory, which employees a DCT-based DFD encoding scheme We compare the proposed coder with H263 The results show that it outperforms H263 by about 25% in terms of bit rate for the same quality reconstructed image

Regularized estimation of occluded displacement vector fields

Abstract: Occluded regions and motion boundaries introduce displacement vector field (DVF) discontinuities that must be reconciled to accurately estimate image flow. In this work, the robust regularized estimation of the DVF is considered in the presence of these discontinuities. A robust convex estimation criterion is presented that preserves motion boundaries and allows for a globally optimal estimate of the DVF. A new class of robust convex measures is introduced for edge preserving regularization and an occlusion weighted gradient is proposed as mechanism for managing DVF discontinuities due to occlusion. Results using synthetic image sequences are presented.