This paper reviews the basic background knowledge necessary to design effective no-reference (NR) quality estimators (QEs) for images and video. We describe a three-stage framework for NR QE that encompasses the range of potential use scenarios for the NR QE and allows knowledge of the human visual system to be incorporated throughout. We survey the measurement stage of the framework, considering methods that rely on bitstream, pixels, or both. By exploring both the accuracy requirements of potential uses as well as evaluation criteria to stress-test a QE, we set the stage for our community to make substantial future improvements to the challenging problem of NR quality estimation.

No-reference image and video quality estimation: Applications and human-motivated design

Single-sensor digital color cameras use a process called color demosaicking to produce full color images from the data captured by a color filter array (CFA). The quality of demosaicked images is degraded due to the sensor noise introduced during the image acquisition process. The conventional solution to combating CFA sensor noise is demosaicking first, followed by a separate denoising processing. This strategy will generate many noise-caused color artifacts in the demosaicking process, which are hard to remove in the denoising process. Few denoising schemes that work directly on the CFA images have been presented because of the difficulties arisen from the red, green and blue interlaced mosaic pattern, yet a well designed ldquodenoising first and demosaicking laterrdquo scheme can have advantages such as less noise-caused color artifacts and cost-effective implementation. This paper presents a principle component analysis (PCA) based spatially-adaptive denoising algorithm, which works directly on the CFA data using a supporting window to analyze the local image statistics. By exploiting the spatial and spectral correlations existed in the CFA image, the proposed method can effectively suppress noise while preserving color edges and details. Experiments using both simulated and real CFA images indicate that the proposed scheme outperforms many existing approaches, including those sophisticated demosaicking and denoising schemes, in terms of both objective measurement and visual evaluation.

/pdf/pca-based-spatially-adaptive-denoising-of-cfa-images-for-2hw1fi7565.pdf

PCA-Based Spatially Adaptive Denoising of CFA Images for Single-Sensor Digital Cameras

A novel no-reference metric that can automatically quantify ringing annoyance in compressed images is presented. In the first step a recently proposed ringing region detection method extracts the regions which are likely to be impaired by ringing artifacts. To quantify ringing annoyance in these detected regions, the visibility of ringing artifacts is estimated, and is compared to the activity of the corresponding local background. The local annoyance score calculated for each individual ringing region is averaged over all ringing regions to yield a ringing annoyance score for the whole image. A psychovisual experiment is carried out to measure ringing annoyance subjectively and to validate the proposed metric. The performance of our metric is compared to existing alternatives in literature and shows to be highly consistent with subjective data.

/pdf/a-no-reference-metric-for-perceived-ringing-artifacts-in-35uh7ughj1.pdf

A No-Reference Metric for Perceived Ringing Artifacts in Images

This paper presents the design of a wireless capsule endoscope system. The proposed system is mainly composed of a CMOS image sensor, a RF transceiver and a low-power controlling and processing application specific integrated circuit (ASIC). Several design challenges involving system power reduction, system miniaturization and wireless wake-up method are resolved by employing optimized system architecture, integration of an area and power efficient image compression module, a power management unit (PMU) and a novel wireless wake-up subsystem with zero standby current in the ASIC design. The ASIC has been fabricated in 0.18-mum CMOS technology with a die area of 3.4 mm * 3.3 mm. The digital baseband can work under a power supply down to 0.95 V with a power dissipation of 1.3 mW. The prototype capsule based on the ASIC and a data recorder has been developed. Test result shows that proposed system architecture with local image compression lead to an average of 45% energy reduction for transmitting an image frame.

A Wireless Capsule Endoscope System With Low-Power Controlling and Processing ASIC

This paper proposes an architecture of the wireless endoscopy system for the diagnoses of whole human digestive tract and real-time endoscopic image monitoring. The low-power digital IC design inside the wireless endoscopic capsule is discussed in detail. A very large scale integration (VLSI) architecture of three-stage clock management is applied, which can save 46% power inside the capsule compared with the design without such a low-power design. A stoppable ring crystal oscillator with minimal overhead is used in the sleep mode, which results in about 60-muW system power dissipation in sleep mode. A new image compression algorithm based on Bayer image format and its corresponding VLSI architecture are both proposed for low-power, high-data volume. Thus, 8 frames per second with 320*288 pixels can be transmitted with 2 Mb/s. The digital IC design also assures that the capsule has many flexible and useful functions for clinical application. The digital circuits were verified on field-programmable gate arrays and have been implemented in 0.18-mum CMOS process with 6.2 mW

A Low-Power Digital IC Design Inside the Wireless Endoscopic Capsule

Many consumer digital color cameras use a single light sensitive sensor and a color filter array (CFA) with each pixel element recording intensity information of one color component. The captured data is interpolated into a full color image, which is then compressed in many applications. Carrying out color interpolation before compression introduces redundancy in the data. In this paper we discuss methods and issues involved in the compression of CFA data before full color interpretation. The compression methods described operate on the same number of pixels as the sensor data. To obtain improved image quality, median filtering is applied as post-processing. Furthermore, to assure low complexity, the CFA data is compressed by JPEG. Simulations have demonstrated that substantial improvement in image quality is achievable using these new schemes.

New efficient methods of image compression in digital cameras with color filter array

In this work we describe a study on a limited number of artifacts in MPEG-2 compressed video with the primary aim to analyze how annoying these compression artifacts are. More specifically, the objectives were: (1) to determine how the subjective annoyances of individual artifacts contribute to the overall annoyance, (2) to obtain the subjective ranks of the artifacts, and (3) to determine what relationships exist between annoyance values and annoyance ranks. To this end, a psychophysical experiment was carried out in which observers provided us with their subjective assessment of video sequences. The results showed that at low compression bit-rates, the blocking artifacts were the most annoying, whereas at higher compression bit-rates the ringing artifacts were the most annoying. The blocking artifact had the highest mean annoyance rank across all videos and compression bit-rates. Mean annoyance values were ordered differently from mean annoyance ranks due to very high annoyance values associated with some artifacts. Overall annoyance was related to the total squared error by a Logistic function. Individual artifact annoyance was related to overall annoyance by a weighted Minkowski metric.

Annoyance of individual artifacts in MPEG-2 compressed video and their relation to overall annoyance

In this study, we examined the subjective attributes of color preference and perceived naturalness of digital videos when the chroma of the color content was varied. More specifically, the objectives were: (1) to determine how the scaling of color chroma affected the mean color preference rating and the mean naturalness rating across subjects and (2) to determine how preference and naturalness were related. To this end, psychophysical experiments were carried out in which naive subjects provided us with their subjective assessments of these two attributes. Data from the experiments indicated that for both preference and naturalness, the mean opinion scores (MOS) increased to a maximum and decreased as the mean chroma (MC) of the videos was varied. Naturalness MOS (NMOS) tended to reach a maximum at lower MC than preference MOS (PMOS). The maximums for both PMOS and NMOS occurred at relatively close MC levels for most of the videos. The individual functions relating MOS to chroma were modeled by simple Gaussian functions relatively well. However, the mean, amplitude and standard deviation varied with video. It was possible to reduce the total number of parameters from 36 to 15 parameters by making use of relationships between the model parameters. In this model the chroma corresponding to maximum preference is about 1.1 times the chroma corresponding to maximum naturalness.

Color preference and perceived color naturalness of digital videos

In this work, we studied how video compression and color scaling interact to affect the overall video quality and the
color quality attributes. We examined the three subjective attributes: perceived color preference, perceived color
naturalness, and overall annoyance, as digital videos were subjected to compression and chroma scaling. Our objectives
were: (1) to determine how the color chroma scaling of compressed digital videos affected the mean color preference
and naturalness and overall annoyance ratings across subjects and (2) to determine how preference, naturalness, and
annoyance were related. Psychophysical experiments were carried out in which naive subjects made numerical
judgments of these three attributes. Preference and naturalness scores increased to a maximum and decreased as the
mean chroma of the videos increased. As compression increased, both preference and naturalness scores decreased and
they varied less with mean chroma. Naturalness scores tended to reach a maximum at lower mean chroma than
preference scores. Annoyance scores decreased to a minimum and then increased as mean chroma increased. The mean
chroma at which annoyance was minimum was less than the mean chroma at which naturalness and preference were
maximum. Preference, naturalness, and annoyance scores for individual videos, were approximated relatively well by
Gaussian functions of mean chroma. Preference and naturalness scores decreased linearly as a function of the logarithm
of the total squared error, while annoyance scores increased as an S-shaped function of the logarithm of the total squared
error. A three-parameter model is shown to provide a good description of how each attribute depends on chroma and
compression for individual videos. Model parameters vary with video content.

Color Preference, Color Naturalness, and Annoyance of Compressed and Color Scaled Digital Videos

Many video processing algorithms have been developed in recent years. However, visual performance validation of these algorithms is often overlooked. We discuss the need for a standardized suite of free test videos appropriately selected according to a set of usage rules. These videos are to be stored in a publicly accessible online library that will facilitate effective collaboration and interaction among research scientists and development engineers.

Chin Chye Koh

Papers

New efficient methods of image compression in digital cameras with color filter array

Annoyance of individual artifacts in MPEG-2 compressed video and their relation to overall annoyance

Color preference and perceived color naturalness of digital videos

Color Preference, Color Naturalness, and Annoyance of Compressed and Color Scaled Digital Videos

Test Video Selection for Video Processing Research and Development