There is an analogy between single-chip color cameras and the human visual system in that these two systems acquire only one limited wavelength sensitivity band per spatial location. We have exploited this analogy, defining a model that characterizes a one-color per spatial position image as a coding into luminance and chrominance of the corresponding three colors per spatial position image. Luminance is defined with full spatial resolution while chrominance contains subsampled opponent colors. Moreover, luminance and chrominance follow a particular arrangement in the Fourier domain, allowing for demosaicing by spatial frequency filtering. This model shows that visual artifacts after demosaicing are due to aliasing between luminance and chrominance and could be solved using a preprocessing filter. This approach also gives new insights for the representation of single-color per spatial location images and enables formal and controllable procedures to design demosaicing algorithms that perform well compared to concurrent approaches, as demonstrated by experiments.

/pdf/linear-demosaicing-inspired-by-the-human-visual-system-3a4iil5xvg.pdf

Linear demosaicing inspired by the human visual system

This paper presents a graphical model for learning and recognizing human actions. Specifically, we propose to encode actions in a weighted directed graph, referred to as action graph, where nodes of the graph represent salient postures that are used to characterize the actions and are shared by all actions. The weight between two nodes measures the transitional probability between the two postures represented by the two nodes. An action is encoded as one or multiple paths in the action graph. The salient postures are modeled using Gaussian mixture models (GMMs). Both the salient postures and action graph are automatically learned from training samples through unsupervised clustering and expectation and maximization (EM) algorithm. The proposed action graph not only performs effective and robust recognition of actions, but it can also be expanded efficiently with new actions. An algorithm is also proposed for adding a new action to a trained action graph without compromising the existing action graph. Extensive experiments on widely used and challenging data sets have verified the performance of the proposed methods, its tolerance to noise and viewpoints, its robustness across different subjects and data sets, as well as the effectiveness of the algorithm for learning new actions.

/pdf/expandable-data-driven-graphical-modeling-of-human-actions-3v6luww9ti.pdf

Expandable Data-Driven Graphical Modeling of Human Actions Based on Salient Postures

Many consumer digital color cameras use a single light sensitive sensor and a color filter array (CFA) with each pixel element recording intensity information of one color component. The captured data is interpolated into a full color image, which is then compressed in many applications. Carrying out color interpolation before compression introduces redundancy in the data. In this paper we discuss methods and issues involved in the compression of CFA data before full color interpretation. The compression methods described operate on the same number of pixels as the sensor data. To obtain improved image quality, median filtering is applied as post-processing. Furthermore, to assure low complexity, the CFA data is compressed by JPEG. Simulations have demonstrated that substantial improvement in image quality is achievable using these new schemes.

New efficient methods of image compression in digital cameras with color filter array

In this paper investigations are conducted to simplify and refine a vision-model-based video quality metric without compromising its prediction accuracy. Unlike other vision-model-based quality metrics, the proposed metric is parameterized using subjective quality assessment data recently provided by the Video Quality Experts Group. The quality metric is able to generate a perceptual distortion map for each and every video frame. A perceptual blocking distortion metric (PBDM) is introduced which utilizes this simplified quality metric. The PBDM is formulated based on the observation that blocking artifacts are noticeable only in certain regions of a picture. A method to segment blocking dominant regions is devised, and perceptual distortions in these regions are summed up to form an objective measure of blocking artifacts. Subjective and objective tests are conducted and the performance of the PBDM is assessed by a number of measures such as the Spearman rank-order correlation, the Pearson correlation, and the average absolute error The results show a strong correlation between the objective blocking ratings and the mean opinion scores on blocking artifacts.

Vision-model-based impairment metric to evaluate blocking artifacts in digital video

Following the trend of increased integration in complementary metal oxide semiconductor (CMOS) image sensors, we have explored the potential of implementing light filters by using patterned metal layers placed on top of each pixel’s photodetector. To demonstrate wavelength selectivity, we designed and prototyped integrated color pixels in a standard 0.18-µm CMOS technology. Transmittance of several one-dimensional (1D) and two-dimensional (2D) patterned metal layers was measured under various illumination conditions and found to exhibit wavelength selectivity in the visible range. We performed (a) wave optics simulations to predict the spectral responsivity of an uncovered reference pixel and (b) numerical electromagnetic simulations with a 2D finite-difference time-domain method to predict transmittances through 1D patterned metal layers. We found good agreement in both cases. Finally, we used simulations to predict the transmittance for more elaborate designs.

/pdf/integrated-color-pixels-in-0-18-microm-complementary-metal-2xf8v541a5.pdf

Integrated color pixels in 0.18-microm complementary metal oxide semiconductor technology.

This paper presents a low-complexity wavelet transform that utilizes point-symmetric extension at the image tile boundaries. The proposed approach is considered as an alternative to the well-known symmetric signal extension in dealing with the boundary artifacts that manifest when tiles of images are lossy compressed. The new solution developed for odd-length filters preserves the perfect reconstruction property of the filter banks and deals efficiently with blocking artifacts without requiring any post processing technique or additional information from the neighboring tiles. It is shown that point-symmetric extension at the tile boundaries does not need to be applied explicitly, but instead the equivalent boundary filters can be derived. The lifting-based implementation of the filters provides a very simple way of changing filter parameters at the boundaries that suits both hardware and software platforms.

A wavelet transform with point-symmetric extension at tile boundaries

The intention of zooming in a video camera is to obtain more detailed visual information than the full frame view. The optical zooming mechanisms provide this additional information, however, when optically zoomed, the camera no longer detects the full frame view, which was detected before zooming. Digital zooming can detect the full view at all times, however, fails to provide additional visual information when zoomed. In applications such as visual surveillance and manufacturing quality measurement, it is required to detect the full frame view at all times, the camera should be capable of providing additional visual information when zoomed to the region of interest (ROI). This paper presents a video camera system incorporating an autonomous pan, tilt and zoom functionality based on ROI decided upon motion detection capability, which provides more visual information in zoomed modes while detecting the full view at all times. The intelligent ROI decision can be made not only based on motion but also shape, color, sizes etc. of targeted objects for tracking. Image sharpening and noise reduction methods are also provided within the camera system. Standard testing results indicate that this camera system can resolve up to 800 TV lines and produces comparable color quality to other CCD based CCTV surveillance cameras, especially at good lighting conditions.

Digital zoom camera with image sharpening and suppression

This paper presents two algorithms for removing the crosstalk effect in CMOS sensor based color-imaging systems. The algorithms work on the Bayer raw data and have low computational complexity. Experimental results on Macbeth color chart and real images demonstrates that both algorithms can effectively eliminate the crosstalk effect and produce better quality images with conventional color interpolation and correction algorithms designed for CCD image sensors. Complexity of the algorithms is also analyzed.

/pdf/cmos-sensor-cross-talk-compensation-for-digital-cameras-484jzhkijt.pdf

CMOS sensor cross-talk compensation for digital cameras

This paper presents a way to model the cross-talk effect in CMOS image sensors. Two algorithms are derived from the model; both of them work on the Bayer raw data and have low computational complexity. Experiments on Macbeth color chart and real images have shown the effectiveness of the modeling to eliminate the cross-talk effect and produce better quality images with traditional color interpolation and correction algorithms designed for CCD image sensors.

/pdf/modelling-of-color-cross-talk-in-cmos-image-sensors-3phbgqs5vt.pdf

Modelling of color cross-talk in CMOS image sensors

This paper presents a color interpolation algorithm for a single sensor color camera The proposed algorithm is especially designed to solve the problem of pixel crosstalk among the pixels of different color channels Interchannel cross-talk gives rise to blocking effects on the interpolated green plane, and also spreading of false colors into detailed structures The proposed algorithm separates the green channel into two planes, one highly correlated with the red channel and the other with the blue channel These separate planes are used for red and blue channel interpolation Experiments conducted on McBeth color chart and natural images have shown that the proposed algorithm can eliminate or suppress blocking and color artifacts to produce better quality images

/pdf/method-of-color-interpolation-in-a-single-sensor-color-5g6ronoyj7.pdf

I. Kharitonenko

Papers

A wavelet transform with point-symmetric extension at tile boundaries

Digital zoom camera with image sharpening and suppression

CMOS sensor cross-talk compensation for digital cameras

Modelling of color cross-talk in CMOS image sensors

Method of color interpolation in a single sensor color camera using green channel separation