Salient areas in natural scenes are generally regarded as areas which the human eye will typically focus on, and finding these areas is the key step in object detection. In computer vision, many models have been proposed to simulate the behavior of eyes such as SaliencyToolBox (STB), Neuromorphic Vision Toolkit (NVT), and others, but they demand high computational cost and computing useful results mostly relies on their choice of parameters. Although some region-based approaches were proposed to reduce the computational complexity of feature maps, these approaches still were not able to work in real time. Recently, a simple and fast approach called spectral residual (SR) was proposed, which uses the SR of the amplitude spectrum to calculate the image's saliency map. However, in our previous work, we pointed out that it is the phase spectrum, not the amplitude spectrum, of an image's Fourier transform that is key to calculating the location of salient areas, and proposed the phase spectrum of Fourier transform (PFT) model. In this paper, we present a quaternion representation of an image which is composed of intensity, color, and motion features. Based on the principle of PFT, a novel multiresolution spatiotemporal saliency detection model called phase spectrum of quaternion Fourier transform (PQFT) is proposed in this paper to calculate the spatiotemporal saliency map of an image by its quaternion representation. Distinct from other models, the added motion dimension allows the phase spectrum to represent spatiotemporal saliency in order to perform attention selection not only for images but also for videos. In addition, the PQFT model can compute the saliency map of an image under various resolutions from coarse to fine. Therefore, the hierarchical selectivity (HS) framework based on the PQFT model is introduced here to construct the tree structure representation of an image. With the help of HS, a model called multiresolution wavelet domain foveation (MWDF) is proposed in this paper to improve coding efficiency in image and video compression. Extensive tests of videos, natural images, and psychological patterns show that the proposed PQFT model is more effective in saliency detection and can predict eye fixations better than other state-of-the-art models in previous literature. Moreover, our model requires low computational cost and, therefore, can work in real time. Additional experiments on image and video compression show that the HS-MWDF model can achieve higher compression rate than the traditional model.

A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression

Salient areas in natural scenes are generally regarded as the candidates of attention focus in human eyes, which is the key stage in object detection. In computer vision, many models have been proposed to simulate the behavior of eyes such as SaliencyToolBox (STB), neuromorphic vision toolkit (NVT) and etc., but they demand high computational cost and their remarkable results mostly rely on the choice of parameters. Recently a simple and fast approach based on Fourier transform called spectral residual (SR) was proposed, which used SR of the amplitude spectrum to obtain the saliency map. The results are good, but the reason is questionable.

Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform

A novel switching median filter incorporating with a powerful impulse noise detection method, called the boundary discriminative noise detection (BDND), is proposed in this paper for effectively denoising extremely corrupted images. To determine whether the current pixel is corrupted, the proposed BDND algorithm first classifies the pixels of a localized window, centering on the current pixel, into three groups-lower intensity impulse noise, uncorrupted pixels, and higher intensity impulse noise. The center pixel will then be considered as "uncorrupted," provided that it belongs to the "uncorrupted" pixel group, or "corrupted." For that, two boundaries that discriminate these three groups require to be accurately determined for yielding a very high noise detection accuracy-in our case, achieving zero miss-detection rate while maintaining a fairly low false-alarm rate, even up to 70% noise corruption. Four noise models are considered for performance evaluation. Extensive simulation results conducted on both monochrome and color images under a wide range (from 10% to 90%) of noise corruption clearly show that our proposed switching median filter substantially outperforms all existing median-based filters, in terms of suppressing impulse noise while preserving image details, and yet, the proposed BDND is algorithmically simple, suitable for real-time implementation and application.

A switching median filter with boundary discriminative noise detection for extremely corrupted images

We address the issue of visual saliency from three perspectives. First, we consider saliency detection as a frequency domain analysis problem. Second, we achieve this by employing the concept of nonsaliency. Third, we simultaneously consider the detection of salient regions of different size. The paper proposes a new bottom-up paradigm for detecting visual saliency, characterized by a scale-space analysis of the amplitude spectrum of natural images. We show that the convolution of the image amplitude spectrum with a low-pass Gaussian kernel of an appropriate scale is equivalent to an image saliency detector. The saliency map is obtained by reconstructing the 2D signal using the original phase and the amplitude spectrum, filtered at a scale selected by minimizing saliency map entropy. A Hypercomplex Fourier Transform performs the analysis in the frequency domain. Using available databases, we demonstrate experimentally that the proposed model can predict human fixation data. We also introduce a new image database and use it to show that the saliency detector can highlight both small and large salient regions, as well as inhibit repeated distractors in cluttered images. In addition, we show that it is able to predict salient regions on which people focus their attention.

Visual Saliency Based on Scale-Space Analysis in the Frequency Domain

This letter focuses on quality assessment of fusion of multispectral (MS) images with high-resolution panchromatic (Pan) observations. A new quality index suitable for MS imagery having four spectral bands is defined from the theory of hypercomplex numbers, or quaternions. Both spectral and radiometric distortion measurements are encapsulated in a unique measurement, simultaneously accounting for local mean bias, changes in contrast, and loss of correlation of individual bands, together with spectral distortion. Results are presented and discussed on very high-resolution QuickBird data, through comparisons between state-of-the-art and advanced MS+Pan merge algorithms.

A global quality measurement of pan-sharpened multispectral imagery

Fourier transforms are a fundamental tool in signal and image processing, yet, until recently, there was no definition of a Fourier transform applicable to color images in a holistic manner. In this paper, hypercomplex numbers, specifically quaternions, are used to define a Fourier transform applicable to color images. The properties of the transform are developed, and it is shown that the transform may be computed using two standard complex fast Fourier transforms. The resulting spectrum is explained in terms of familiar phase and modulus concepts, and a new concept of hypercomplex axis. A method for visualizing the spectrum using color graphics is also presented. Finally, a convolution operational formula in the spectral domain is discussed

Hypercomplex Fourier Transforms of Color Images

Hypercomplex Fourier transforms based on quaternions have been proposed by several groups for use in image processing, particularly of color images. So far, however, there has not been a coherent explanation of what the spectral domain coefficients produced by a hypercomplex Fourier transform represent and this paper attempts to present such an explanation for the first time making use of the polar form of a quaternion and a separation of a quaternion spectral coefficient into components parallel and perpendicular to the hypercomplex exponentials in the transform (the basis functions).

Hypercomplex Fourier transforms of color images

Preface. Acknowledgements. 1. The present state and the future of colour image processing. Part One: Colour. 2. Colour vision W. McIlhagga. 3. Colour science M. Ronnier Luo. 4. Colour spaces H. Palus. Part Two: Image Acquisition. 5. Colour video systems and signals R.E.N. Horne. 6. Image sources C. Connolly. 7. Practical system considerations C. Connolly, H. Palus. Part Three: Processing. 8. Noise removal and contrast enhancement J.M. Gauch. 9. Segmentation and edge detection J.M. Gauch. 10. Vector filtering K.N. Plataniotis, A.N. Venetsanopoulos. 11. Morphological operations M.L. Comer, E.J. Delp. 12. Frequency domain methods S.J. Sangwine, A.L. Thornton. 13. Compression M. Domanski, M. Bartkowiak. Part Four: Applications. 14. Colour management for the textile industry P.A. Rhodes. 15. Colour management for the graphic arts J. Morovic. 16. Medical imaging case study B.F. Jones, P. Plassmann. 17. Industrial colour inspection case studies C. Connolly. References. Index.

The colour image processing handbook

The 2D quaternion, or hypercomplex, Fourier transform is introduced. This transform makes possible the handling of colour images in the frequency domain in a holistic manner, without separate handling of the colour components, and it thus makes possible very wide generalisation of monochrome frequency domain techniques to colour images.

Fourier transforms of colour images using quaternion or hypercomplex, numbers

Correlation techniques have been applied to almost every area of signal processing over the past century, yet their use has, in general, been limited to scalar signals. While there have been implementations in multichannel applications, these can be characterized as a combination of single channel processes. True vector correlation techniques, with global and interchannel measures, have only recently been demonstrated and are still in their infancy by comparison. This paper describes our work on vector correlation based on the use of hypercomplex Fourier transforms and presents, for the first time, a unified theory behind the information contained in the peak of a vector correlation response. By using example applications for color images, we also demonstrate some of the practical implications, together with our latest results.

Stephen J. Sangwine

Papers

Hypercomplex Fourier Transforms of Color Images

Hypercomplex Fourier transforms of color images

The colour image processing handbook

Fourier transforms of colour images using quaternion or hypercomplex, numbers

Hypercomplex correlation techniques for vector images