Showing papers by "Mohamed-Chaker Larabi published in 2013"

Qualinet White Paper on Definitions of Quality of Experience

Abstract: This White Paper is a contribution of the European Network on Quality of Experience in Multimedia Systems and Services, Qualinet (COST Action IC 1003, see www.qualinet.eu), to the scientific discussion about the term "Quality of Experience" (QoE) and its underlying concepts. It resulted from the need to agree on a working definition for this term which facilitates the communication of ideas within a multidisciplinary group, where a joint interest around multimedia communication systems exists, however approached from different perspectives. Thus, the concepts and ideas cited in this paper mainly refer to the Quality of Experience of multimedia communication systems, but may be helpful also for other areas where QoE is an issue. The Network of Excellence (NoE) Qualinet aims at extending the notion of network-centric Quality of Service (QoS) in multimedia systems, by relying on the concept of Quality of Experience (QoE). The main scientific objective is the development of methodologies for subjective and objective quality metrics taking into account current and new trends in multimedia communication systems as witnessed by the appearance of new types of content and interactions. A substantial scientific impact on fragmented efforts carried out in this field will be achieved by coordinating the research of European experts under the catalytic COST umbrella. The White Paper has been compiled on the basis of a first open call for ideas which was launched for the February 2012 Qualinet Meeting held in Prague, Czech Republic. The ideas were presented as short statements during that meeting, reflecting the ideas of the persons listed under the headline "Contributors" in the previous section. During the Prague meeting, the ideas have been further discussed and consolidated in the form of a general structure of the present document. An open call for authors was issued at that meeting, to which the persons listed as "Authors" in the previous section have announced their willingness to contribute in the preparation of individual sections. For each section, a coordinating author has been assigned which coordinated the writing of that section, and which is underlined in the author list preceding each section. The individual sections were then integrated and aligned by an editing group (listed as "Editors" in the previous section), and the entire document was iterated with the entire group of authors. Furthermore, the draft text was discussed with the participants of the Dagstuhl Seminar 12181 "Quality of Experience: From User Perception to Instrumental Metrics" which was held in Schlos Dagstuhl, Germany, May 1-4 2012, and a number of changes were proposed, resulting in the present document. As a result of the writing process and the large number of contributors, authors and editors, the document will not reflect the opinion of each individual person at all points. Still, we hope that it is found to be useful for everybody working in the field of Quality of Experience of multimedia communication systems, and most probably also beyond that field.

A perceptual metric for stereoscopic image quality assessment based on the binocular energy

Abstract: Stereoscopic imaging is becoming very popular and its deployment by means of photography, television, cinema. . .is rapidly increasing. Obviously, the access to this type of images imposes the use of compression and transmission that may generate artifacts of different natures. Consequently, it is important to have appropriate tools to measure the quality of stereoscopic content. Several studies tried to extend well-known metrics, such as the PSNR or SSIM, to 3D. However, the results are not as good as for 2D images and it becomes important to have metrics dealing with 3D perception. In this work, we propose a full reference metric for quality assessment of stereoscopic images based on the binocular fusion process characterizing the 3D human perception. The main idea consists of the development of a model allowing to reproduce the binocular signal generated by simple and complex cells, and to estimate the associated binocular energy. The difference of binocular energy has shown a high correlation with the human judgement for different impairments and is used to build the Binocular Energy Quality Metric (BEQM). Extensive experiments demonstrated the performance of the BEQM with regards to literature.

Perceptual metrics for static and dynamic triangle meshes

Abstract: Almost all mesh processing procedures cause some more or less visible changes in the appearance of objects represented by polygonal meshes In many cases, such as mesh watermarking, simplification or lossy compression, the objective is to make the change in appearance negligible, or as small as possible, given some other constraints Measuring the amount of distortion requires taking into account the final purpose of the data In many applications, the final consumer of the data is a human observer, and therefore the perceptibility of the introduced appearance change by a human observer should be the criterion that is taken into account when designing and configuring the processing algorithms In this review, we discuss the existing comparison metrics for static and dynamic (animated) triangle meshes We describe the concepts used in perception-oriented metrics used for 2D image comparison, and we show how these concepts are employed in existing 3D mesh metrics We describe the character of subjective data used for evaluation of mesh metrics and provide comparison results identifying the advantages and drawbacks of each method Finally, we also discuss employing the perception-correlated metrics in perception-oriented mesh processing algorithms

A survey of perceptual image processing methods

Abstract: Perceptual approaches have been widely used in many areas of visual information processing. This paper presents an overview of perceptual based approaches for image enhancement, segmentation and coding. The paper also provides a brief review of image quality assessment (IQA) methods, which are used to evaluate the performance of visual information processing techniques. The intent of this paper is not to review all the relevant works that have appeared in the literature, but rather to focus on few topics that have been extensively researched and developed over the past few decades. The goal is to present a perspective as broad as possible on this actively evolving domain due to relevant advances in vision research and signal processing. Therefore, for each topic, we identify the main contributions of perceptual approaches and their limitations, and outline how perceptual vision has influenced current state-of-the-art techniques in image enhancement, segmentation, coding and visual information quality assessment.

Investigation of visual fatigue/discomfort generated by S3D video using eye-tracking data

Abstract: Stereoscopic 3D is undoubtedly one of the most attractive content. It has been deployed intensively during the last decade through movies and games. Among the advantages of 3D are the strong involvement of viewers and the increased feeling of presence. However, the sanitary e ects that can be generated by 3D are still not precisely known. For example, visual fatigue and visual discomfort are among symptoms that an observer may feel. In this paper, we propose an investigation of visual fatigue generated by 3D video watching, with the help of eye-tracking. From one side, a questionnaire, with the most frequent symptoms linked with 3D, is used in order to measure their variation over time. From the other side, visual characteristics such as pupil diameter, eye movements ( xations and saccades) and eye blinking have been explored thanks to data provided by the eye-tracker. The statistical analysis showed an important link between blinking duration and number of saccades with visual fatigue while pupil diameter and xations are not precise enough and are highly dependent on content. Finally, time and content play an important role in the growth of visual fatigue due to 3D watching.

Acceptable bit-rates for human face identification from CCTV imagery

Abstract: The objective of this investigation is to produce recommendations for acceptable bit-rates of CCTV footage of people onboard London buses. The majority of CCTV recorders on buses use a proprietary format based on the H.264/AVC video coding standard, exploiting both spatial and temporal redundancy. Low bit-rates are favored in the CCTV industry but they compromise the image usefulness of the recorded imagery. In this context usefulness is defined by the presence of enough facial information remaining in the compressed image to allow a specialist to identify a person. The investigation includes four steps: 1) Collection of representative video footage. 2) The grouping of video scenes based on content attributes. 3) Psychophysical investigations to identify key scenes, which are most affected by compression. 4) Testing of recording systems using the key scenes and further psychophysical investigations. The results are highly dependent upon scene content. For example, very dark and very bright scenes were the most challenging to compress, requiring higher bit-rates to maintain useful information. The acceptable bit-rates are also found to be dependent upon the specific CCTV system used to compress the footage, presenting challenges in drawing conclusions about universal ‘average’ bit-rates.

Visual coherence metric for evaluation of color image restoration

Abstract: Digital image inpainting technique could provide substantial support for future restoration of images. However, much less effort has been devoted to the development of image inpainting evaluation. In this paper, we proposed a novel objective metric for assessing the quality of color image inpainting which takes into account some constraints and characteristics related to the specific objectives of inpainting approaches. The used characteristics are the visual coherence of the recovered regions and the visual saliency describing the visual importance of the area. A series of psychophysical experiments have been conducted to evaluate the performance of the proposed image quality index. The results show that the proposed image quality metric is well adapted for image quality inpainting evaluation.

Binocular Energy Estimation Based on Properties of the Human Visual System

Abstract: 3D applications are very popular nowadays. They allow to bring new sensations (e.g., cinema, gaming, etc) and new ways for analyzing data (e.g., video-surveillance, pattern recognition, etc). The large availability of 3D data ia better understanding of human 3D perception in order to improve the quality of 3D visual data, increase the visual comfort and avoid visual fatigue and visual illness. In this paper, we explore the binocular perception through its various indices. The focus is put on the binocular energy and its evolution with regard to image impairments. Two types of cells are explored, that is, simple and complex cells responsible of the sensory fusion in the visual cortex. A model is proposed for these cells in order to estimate the binocular energy from color stereo-pairs. The integration of stereoscopic constraints such as unicity, coherence and occlusion allows to refine the proposed model described previously by taking into account the occluded and the non-occluded information. A deep experimentation demonstrates the efficiency of the described modeling. The estimated binocular energy presents a correlation with the impairment level caused by compression or noise.

Inpainted image quality assessment

Abstract: A new Image Quality Metric (IQM) dedicated to image in-painting evaluation is proposed in this paper. This objective measure is computed by combining some structural and salient features related to the specific goals of inpainting purposes. The main perceptual characteristics used in this approach are the visual coherence and the visual saliency. The proposed Inpainted Image Quality Assessment is evaluated trough some experimental results. It is shown that the proposed metric is consistent with the perceptual evaluation of image inpainting results.

Quality assessment of still images

Abstract: In this chapter, a description of evaluation methods to quantify the quality of impaired still images is proposed. The presentation starts with an overview of the mainly subjective methods recommended by both the International Telecommunication Union (ITU) and International Organization for Standardization (ISO) and widely used by Video Quality Experts Group (VQEG). Then, the algorithmic measures are investigated. In this context, low-complexity metrics such as Peak Signal to Noise Ratio (PSNR) and Mean Squared Error (MSE) are first presented to finally reach perceptual metrics. The general scheme of these latter is based on the Human Visual System (HVS) and exploits many properties such as the luminance adaptation, the spatial frequency sensitivity, the contrast and the masking effects. The performance evaluation of the objective quality metrics follows a methodology that is described.

Attentional mechanisms driven adaptive quantization and selective bit allocation scheme for H.264/AVC

Abstract: Rate control algorithm adopted in H.264/AVC reference software shows several shortcomings that have been highlighted by different studies. For instance, in the baseline profile, the frame target bit-rate estimation assumes similar characteristics for all frames and the quantization parameter determination uses the Mean Absolute Difference for complexity estimation. Consequently, an inefficient bit allocation is performed leading to important quality variation of decoded sequences. A saliency-based rate-control is proposed in this paper to achieve bit-rate saving and improve perceived quality. The saliency map of each frame, simulating the human visual attention by a bottom-up approach, is used at the frame level to adjust the quantization parameter and at the macroblock level to guide the bit allocation process. Simulation results show that the proposed attentional model is well correlated to human behavior. When compared to JM15.0 reference software, at the frame level, the saliency map exploitation achieves bit-rate savings of up to 26%. At the MB level and under the same quality constraint, bit-rate improvement is up to 42% and buffer level variation is reduced by up to 71%.

Perceptual evaluation of digital image completion quality

Abstract: Recently, image completion or inpainting has become a hot research topic because of its extensive applications in various real-world applications Many papers have been published on this topic, but there are very few studies devoted to objective image inpainting evaluation The intend of this work is to propose a solution based on Image Quality Metrics (IQMs) The idea is to adapt some IQMs by taking into account some specificities and constraints related to image inpainting objectives The main characteristics used in this approach are the visual coherence of the recovered regions and the visual saliency In this paper the notion of Inpainted Image Quality Assessment is introduced and discussed trough some experimental results The obtained results demonstrate that the proposed image quality metric is well adapted to image inpainting evaluation

Image Super-Resolution, a State-of-the-Art Review and Evaluation

Abstract: Image super-resolution is a popular technique for increasing the resolution of a given image. Its most common application is to provide better visual effect after resizing a digital image for display or printing. In recent years, due to consumer multimedia products being in vogue, imaging and display device become ubiquitous, and image super-resolution is becoming more and more important. There are mainly three categories of approaches for this problem: interpolation-based methods, reconstruction-based methods, and learning-based methods.

Biologically inspired approaches for visual information processing and analysis

Abstract: Recent developments in digital technologies that make use of large scale image and video data, such as the internet, multimedia systems, digital TV, digital cinema, and mobile/wireless devices, have increased the need for efficient processing, storage, and transmission of visual information. It is now widely acknowledged that such technologies can further benefit by incorporating concepts from human vision and corresponding implementation of biologically inspired models. Indeed, in many applications, the human observer plays a crucial role in the decisionmaking tasks such as diagnosis, recognition, and evaluation based on visual assessment of images. Therefore, biologically inspired image/video processing and analysis approaches have been receiving considerable interest in recent years. This special issue presents some of the recent advances in image and video processing techniques inspired by the human visual system (HVS), with applications to multimedia, visual pattern recognition, surveillance, visual information coding and security, and other related subjects. The first paper by Beghdadi, Larabi, Bouzerdoum, and Iftekharuddin provides an overview of the state-of-the-art techniques in perceptual image processing, with particular emphasis on image enhancement, segmentation, and coding. The remaining articles are grouped into four categories, according to the topics addressed in the papers. The first group deals with perceptual video and image coding. The first paper entitled “Perceptual video coding based on MB classification and rate-distortion optimization,” by Guillotel, Aribuki, Olivier and Urban, exploits the perceptual image quality of macroblock in rate-distortion optimization to improve the perceptual quality of coded video sequences. The main idea is to introduce the perceptual distortion metric in the encoding loop in order to adapt the macroblock quantization parameters. A learning strategy is then applied to a number of sequences to provide a set of representative distortion–quantization curves for all macroblock types. The second paper by Hrarti, Saadane and Larabi, entitled “Attentional mechanisms driven adaptive quantization and selective bit allocation scheme for H.264/AVC,” proposes a saliency based

Coding and rendering enhancement of multi-view video by color correction

Abstract: Multi-View Video (MVV) is captured from different viewpoints with multiple cameras, where substantial inconsistencies of illumination and color are often observed between the different views. These color mismatches can reduce significantly the compression efficiency and the performance of image-based rendering algorithms. Therefore, we propose a preprocessing method based on the Histogram Matching (HM) algorithm for correcting these color discrepancies in MVV. To consider occlusion problem in MVV, we perform the HM algorithm only on a set of common regions across views. These regions are defined using an invariant feature detector (SIFT), followed by RANSAC (Random Sample Consensus) algorithm to increase the matching robustness. In addition, to maintain temporal correlation, HM algorithm is applied on a temporal sliding-window. The choice of the color reference is also addressed in this paper and is carried out automatically in a more adaptive way. Experimental results show that the proposed method increases coding efficiency with gains of up to 1.1 dB and 2.2 dB for the luminance and chrominance components, respectively. Moreover, once correction is performed, the color of both real and rendered views is harmonized and looks very consistent as a whole.