We describe a model of visual processing in which feedback connections from a higher- to a lower- order visual cortical area carry predictions of lower-level neural activities, whereas the feedforward connections carry the residual errors between the predictions and the actual lower-level activities. When exposed to natural images, a hierarchical network of model neurons implementing such a model developed simple-cell-like receptive fields. A subset of neurons responsible for carrying the residual errors showed endstopping and other extra-classical receptive-field effects. These results suggest that rather than being exclusively feedforward phenomena, nonclassical surround effects in the visual cortex may also result from cortico-cortical feedback as a consequence of the visual system using an efficient hierarchical strategy for encoding natural images.

Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.

First published in 1995, Wavelets and Subband Coding offered a unified view of the exciting field of wavelets and their discrete-time cousins, filter banks, or subband coding. The book developed the theory in both continuous and discrete time, and presented important applications. During the past decade, it filled a useful need in explaining a new view of signal processing based on flexible time-frequency analysis and its applications. Since 2007, the authors now retain the copyright and allow open access to the book.

Wavelets and Subband Coding

Most of the signal processing that we will study in this course involves local operations on a signal, namely transforming the signal by applying linear combinations of values in the neighborhood of each sample point. You are familiar with such operations from Calculus, namely, taking derivatives and you are also familiar with this from optics namely blurring a signal. We will be looking at sampled signals only. Let's start with a few basic examples. Local difference Suppose we have a 1D image and we take the local difference of intensities, DI(x) = 1 2 (I(x + 1) − I(x − 1)) which give a discrete approximation to a partial derivative. (We compute this for each x in the image.) What is the effect of such a transformation? One key idea is that such a derivative would be useful for marking positions where the intensity changes. Such a change is called an edge. It is important to detect edges in images because they often mark locations at which object properties change. These can include changes in illumination along a surface due to a shadow boundary, or a material (pigment) change, or a change in depth as when one object ends and another begins. The computational problem of finding intensity edges in images is called edge detection. We could look for positions at which DI(x) has a large negative or positive value. Large positive values indicate an edge that goes from low to high intensity, and large negative values indicate an edge that goes from high to low intensity. Example Suppose the image consists of a single (slightly sloped) edge:

http://ieeexplore.ieee.org/iel5/5/31307/01456462.pdf

Linear systems

Wavelet-based statistical signal processing techniques such as denoising and detection typically model the wavelet coefficients as independent or jointly Gaussian. These models are unrealistic for many real-world signals. We develop a new framework for statistical signal processing based on wavelet-domain hidden Markov models (HMMs) that concisely models the statistical dependencies and non-Gaussian statistics encountered in real-world signals. Wavelet-domain HMMs are designed with the intrinsic properties of the wavelet transform in mind and provide powerful, yet tractable, probabilistic signal models. Efficient expectation maximization algorithms are developed for fitting the HMMs to observational signal data. The new framework is suitable for a wide range of applications, including signal estimation, detection, classification, prediction, and even synthesis. To demonstrate the utility of wavelet-domain HMMs, we develop novel algorithms for signal denoising, classification, and detection.

/pdf/wavelet-based-statistical-signal-processing-using-hidden-229ikww4n7.pdf

Wavelet-based statistical signal processing using hidden Markov models

We describe a learning-based method for low-level vision problems—estimating scenes from images. We generate a synthetic world of scenes and their corresponding rendered images, modeling their relationships with a Markov network. Bayesian belief propagation allows us to efficiently find a local maximum of the posterior probability for the scene, given an image. We call this approach VISTA—Vision by Image/Scene TrAining.

We apply VISTA to the “super-resolution” problem (estimating high frequency details from a low-resolution image), showing good results. To illustrate the potential breadth of the technique, we also apply it in two other problem domains, both simplified. We learn to distinguish shading from reflectance variations in a single image under particular lighting conditions. For the motion estimation problem in a “blobs world”, we show figure/ground discrimination, solution of the aperture problem, and filling-in arising from application of the same probabilistic machinery.

Learning Low-Level Vision

Recently, a framework for multiscale stochastic modeling was introduced based on coarse-to-fine scale-recursive dynamics defined on trees. This model class has some attractive characteristics which lead to extremely efficient, statistically optimal signal and image processing algorithms. The authors show that this model class is also quite rich. In particular, they describe how 1-D Markov processes and 2-D Markov random fields (MRFs) can be represented within this framework. The recursive structure of 1-D Markov processes makes them simple to analyze, and generally leads to computationally efficient algorithms for statistical inference. On the other hand, 2-D MRFs are well known to be very difficult to analyze due to their noncausal structure, and thus their use typically leads to computationally intensive algorithms for smoothing and parameter identification. In contrast, their multiscale representations are based on scale-recursive models and thus lead naturally to scale-recursive algorithms, which can be substantially more efficient computationally than those associated with MRF models. In 1-D, the multiscale representation is a generalization of the midpoint deflection construction of Brownian motion. The representation of 2-D MRFs is based on a further generalization to a "midline" deflection construction. The exact representations of 2-D MRFs are used to motivate a class of multiscale approximate MRF models based on one-dimensional wavelet transforms. They demonstrate the use of these latter models in the context of texture representation and, in particular, they show how they can be used as approximations for or alternatives to well-known MRF texture models. >

Multiscale representations of Markov random fields

A new approach to regularization methods for image processing is introduced and developed using as a vehicle the problem of computing dense optical flow fields in an image sequence. The solution of the new problem formulation is computed with an efficient multiscale algorithm. Experiments on several image sequences demonstrate the substantial computational savings that can be achieved due to the fact that the algorithm is noniterative and in fact has a per pixel computational complexity that is independent of image size. The new approach also has a number of other important advantages. Specifically, multiresolution flow field estimates are available, allowing great flexibility in dealing with the tradeoff between resolution and accuracy. Multiscale error covariance information is also available, which is of considerable use in assessing the accuracy of the estimates. In particular, these error statistics can be used as the basis for a rational procedure for determining the spatially-varying optimal reconstruction resolution. Furthermore, if there are compelling reasons to insist upon a standard smoothness constraint, the new algorithm provides an excellent initialization for the iterative algorithms associated with the smoothness constraint problem formulation. Finally, the usefulness of the approach should extend to a wide variety of ill-posed inverse problems in which variational techniques seeking a "smooth" solution are generally used. >

Efficient multiscale regularization with applications to the computation of optical flow

A propagation model that describes the temporal characteristics of singly scattered radiation in a homogeneous scattering and absorbing medium is presented. The model generalizes previous results in the area and is used to analyze the angular spectrum of singly scattered energy as well as the impulse-response durations and path losses of short-range non-line-of-sight optical communication systems. It is shown that the angular response starts to drop off significantly at an off-axis angle equal to the receiver half-field of view. It is also shown that lower path losses correspond to longer impulse responses so that a lower available bandwidth is indicated. These results are based on numerical examples motivated by the operation of non-line-of-sight communications links in the middle-ultraviolet wave band.

Non-line-of-sight single-scatter propagation model

A class of multiscale stochastic models based on scale-recursive dynamics on trees has previously been introduced. Theoretical and experimental results have shown that these models provide an extremely rich framework for representing both processes which are intrinsically multiscale, e.g., 1/f processes, as well as 1D Markov processes and 2D Markov random fields. Moreover, efficient optimal estimation algorithms have been developed for these models by exploiting their scale-recursive structure. The authors exploit this structure in order to develop a computationally efficient and parallelizable algorithm for likelihood calculation. They illustrate one possible application to texture discrimination and demonstrate that likelihood-based methods using the algorithm achieve performance comparable to that of Gaussian Markov random field based techniques, which in general are prohibitively complex computationally. >

Likelihood calculation for a class of multiscale stochastic models, with application to texture discrimination

A framework for multiscale stochastic modeling was introduced (K.C. Chou et al., 1989) based on coarse-to-fine scale-recursive dynamics defined on trees. This model class has some attractive characteristics which lead to extremely efficient, statistically optimal signal and image processing algorithms. In the present work, the authors describe how 1-D Markov processes and 2-D Markov random fields (MRFs) can be represented within this framework. In addition, they propose a framework for reduced-order multiscale modeling of Gaussian MRFs. >

M.R. Luettgen

Papers

Multiscale representations of Markov random fields

Efficient multiscale regularization with applications to the computation of optical flow

Non-line-of-sight single-scatter propagation model

Likelihood calculation for a class of multiscale stochastic models, with application to texture discrimination

Multiscale representations of Markov random fields