MUCKE aims to mine a large volume of images, to structure them conceptually and to use this conceptual structuring in order to improve large-scale image retrieval. The last decade witnessed important progress concerning low-level image representations. However, there are a number problems which need to be solved in order to unleash the full potential of image mining in applications. The central problem with low-level representations is the mismatch between them and the human interpretation of image content. This problem can be instantiated, for instance, by the incapability of existing descriptors to capture spatial relationships between the concepts represented or by their incapability to convey an explanation of why two images are similar in a content-based image retrieval framework. We start by assessing existing local descriptors for image classification and by proposing to use co-occurrence matrices to better capture spatial relationships in images. The main focus in MUCKE is on cleaning large scale Web image corpora and on proposing image representations which are closer to the human interpretation of images. Consequently, we introduce methods which tackle these two problems and compare results to state of the art methods. Note: some aspects of this deliverable are withheld at this time as they are pending review. Please contact the authors for a preview.

http://ieeexplore.ieee.org/iel2/4524/12820/00592460.pdf

Image Processing

We develop an efficient general-purpose blind/no-reference image quality assessment (IQA) algorithm using a natural scene statistics (NSS) model of discrete cosine transform (DCT) coefficients. The algorithm is computationally appealing, given the availability of platforms optimized for DCT computation. The approach relies on a simple Bayesian inference model to predict image quality scores given certain extracted features. The features are based on an NSS model of the image DCT coefficients. The estimated parameters of the model are utilized to form features that are indicative of perceptual quality. These features are used in a simple Bayesian inference approach to predict quality scores. The resulting algorithm, which we name BLIINDS-II, requires minimal training and adopts a simple probabilistic model for score prediction. Given the extracted features from a test image, the quality score that maximizes the probability of the empirically determined inference model is chosen as the predicted quality score of that image. When tested on the LIVE IQA database, BLIINDS-II is shown to correlate highly with human judgments of quality, at a level that is competitive with the popular SSIM index.

/pdf/blind-image-quality-assessment-a-natural-scene-statistics-2at3p0z79b.pdf

Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain

Images of outdoor scenes captured in bad weather suffer from poor contrast. Under bad weather conditions, the light reaching a camera is severely scattered by the atmosphere. The resulting decay in contrast varies across the scene and is exponential in the depths of scene points. Therefore, traditional space invariant image processing techniques are not sufficient to remove weather effects from images. We present a physics-based model that describes the appearances of scenes in uniform bad weather conditions. Changes in intensities of scene points under different weather conditions provide simple constraints to detect depth discontinuities in the scene and also to compute scene structure. Then, a fast algorithm to restore scene contrast is presented. In contrast to previous techniques, our weather removal algorithm does not require any a priori scene structure, distributions of scene reflectances, or detailed knowledge about the particular weather condition. All the methods described in this paper are effective under a wide range of weather conditions including haze, mist, fog, and conditions arising due to other aerosols. Further, our methods can be applied to gray scale, RGB color, multispectral and even IR images. We also extend our techniques to restore contrast of scenes with moving objects, captured using a video camera.

/pdf/contrast-restoration-of-weather-degraded-images-orn61jh9qu.pdf

Contrast restoration of weather degraded images

Current vision systems are designed to perform in clear weather. Needless to say, in any outdoor application, there is no escape from “bad” weather. Ultimately, computer vision systems must include mechanisms that enable them to function (even if somewhat less reliably) in the presence of haze, fog, rain, hail and snow.

We begin by studying the visual manifestations of different weather conditions. For this, we draw on what is already known about atmospheric optics, and identify effects caused by bad weather that can be turned to our advantage. Since the atmosphere modulates the information carried from a scene point to the observer, it can be viewed as a mechanism of visual information coding. We exploit two fundamental scattering models and develop methods for recovering pertinent scene properties, such as three-dimensional structure, from one or two images taken under poor weather conditions.

Next, we model the chromatic effects of the atmospheric scattering and verify it for fog and haze. Based on this chromatic model we derive several geometric constraints on scene color changes caused by varying atmospheric conditions. Finally, using these constraints we develop algorithms for computing fog or haze color, depth segmentation, extracting three-dimensional structure, and recovering “clear day” scene colors, from two or more images taken under different but unknown weather conditions.

/pdf/vision-and-the-atmosphere-29oh2z55ig.pdf

Vision and the Atmosphere

This paper presents a fast deblurring method that produces a deblurring result from a single image of moderate size in a few seconds. We accelerate both latent image estimation and kernel estimation in an iterative deblurring process by introducing a novel prediction step and working with image derivatives rather than pixel values. In the prediction step, we use simple image processing techniques to predict strong edges from an estimated latent image, which will be solely used for kernel estimation. With this approach, a computationally efficient Gaussian prior becomes sufficient for deconvolution to estimate the latent image, as small deconvolution artifacts can be suppressed in the prediction. For kernel estimation, we formulate the optimization function using image derivatives, and accelerate the numerical process by reducing the number of Fourier transforms needed for a conjugate gradient method. We also show that the formulation results in a smaller condition number of the numerical system than the use of pixel values, which gives faster convergence. Experimental results demonstrate that our method runs an order of magnitude faster than previous work, while the deblurring quality is comparable. GPU implementation facilitates further speed-up, making our method fast enough for practical use.

Fast motion deblurring

Subjective evaluation by human observers is usually used to analyze and select an edge detector parametric setup when real-world images are considered. We propose a statistical objective performance analysis and detector parameter selection, using detection results produced by different detector parameters. Using the correspondence between the different detection results, an estimated best edge map, utilized as an estimated ground truth (EGT), is obtained. This is done using both a receiver operating characteristics (ROC) analysis and a Chi-square test, and considers the trade off between information and noisiness in the detection results. The best edge detector parameter set (PS) is then selected by the same statistical approach, using the EGT. Results are demonstrated for several edge detection techniques, and compared to published subjective evaluation results. The method developed here suggests a general tool to assist in practical implementations of parametric edge detectors where an automatic process is required.

A method for objective edge detection evaluation and detector parameter selection

Identification of blur parameters from motion blurred images

We deal with the problem of restoration of images blurred by relative motion between the camera and the object of interest. This problem is common when the imaging system is in moving vehicles or held by human hands, and in robot vision. For correct restoration of the degraded image, it is useful to know the point-spread function (PSF) of the blurring system. We propose a straightforward method to restore motion-blurred images given only the blurred image itself. The method first identifies the PSF of the blur and then uses it to restore the blurred image. The blur identification here is based on the concept that image characteristics along the direction of motion are affected mostly by the blur and are different from the characteristics in other directions. By filtering the blurred image, we emphasize the PSF correlation properties at the expense of those of the original image. Experimental results for image restoration are presented for both synthetic and real motion blur.

/pdf/direct-method-for-restoration-of-motion-blurred-images-f374zp35ll.pdf

Direct method for restoration of motion-blurred images

Restoration for actual atmospherically blurred images is per- formed using an atmospheric Wiener filter that corrects simultaneously for both turbulence and aerosol blur by enhancing the image spectrum primarily at those high frequencies least affected by the jitter or random- ness in a turbulence modulation transfer function (MTF). The correction is based on weather-predicted rather than measured atmospheric MTFs. Both turbulence and aerosol MTFs are predicted using meteorological parameters measured with standard weather stations at the time and location where the image was recorded. A variety of weather conditions and seasons are considered. Past results have shown good correlation between measured and predicted atmospheric MTFs. Here, the pre- dicted MTFs are implemented in actual image restoration and quantita- tive analysis of the MTF improvement is presented. Corrections are shown also for turbulence blur alone, for aerosol blur alone, and for both together. © 1997 Society of Photo-Optical Instrumentation Engineers. (S0091-3286(97)01511-0)

Restoration of atmospherically blurred images according to weather-predicted atmospheric modulation transfer functions

A difficult problem in imaging systems is degradation of images caused by motion. This problem is common when the imaging system is in moving vehicles such as tanks or planes and even when the camera is held by human hands. For correct restoration of the degraded image we need to know the point spread function (PSF) of the blurring system. In this paper we propose a method to identify important parameters with which to characterize the PSF of the blur, given only the blurred image itself. A first step of this method has been suggested in a former paper where only the blur extent parameter was considered. The identification method here is based on the concept that image characteristics along the direction of motion are different than the characteristics in other directions. Depending on the PSF shape, the homogeneity and the smoothness of the blurred image in the motion direction are higher than in other directions. Furthermore, in the motion direction correlation exists between the pixels forming the blur of the original unblurred objects. The method proposed here identifies the direction and the extent of the PSF of the blur and evaluates its shape which depends on the type of motion during the exposure. Correct identification of the PSF parameters permits fast high resolution restoration of the blurred image.

Yitzhak Yitzhaky

Papers

A method for objective edge detection evaluation and detector parameter selection

Identification of blur parameters from motion blurred images

Direct method for restoration of motion-blurred images

Restoration of atmospherically blurred images according to weather-predicted atmospheric modulation transfer functions

Identification of blur parameters from motion-blurred images