Visual discomfort has been the subject of considerable research in relation to stereoscopic and autostereoscopic displays. In this paper, the importance of various causes and aspects of visual discomfort is clarified. When disparity values do not surpass a limit of 1°, which still provides sufficient range to allow satisfactory depth perception in stereoscopic television, classical determinants such as excessive binocular parallax and accommodation-vergence conflict appear to be of minor importance. Visual discomfort, however, may still occur within this limit and we believe the following factors to be the most pertinent in contributing to this: (1) temporally changing demand of accommodation-vergence linkage, e.g., by fast motion in depth; (2) three-dimensional artifacts resulting from insufficient depth information in the incoming data signal yielding spatial and temporal inconsistencies; and (3) unnatural blur. In order to ad- equately characterize and understand visual discomfort, multiple types of measurements, both objective and subjective, are required. © 2009 Society for Imaging Science and Technology. DOI: 10.2352/J.ImagingSci.Technol.2009.53.3.030201

Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review

Changing network conditions pose severe problems to video streaming in the Internet. HTTP adaptive streaming (HAS) is a technology employed by numerous video services that relieves these issues by adapting the video to the current network conditions. It enables service providers to improve resource utilization and Quality of Experience (QoE) by incorporating information from different layers in order to deliver and adapt a video in its best possible quality. Thereby, it allows taking into account end user device capabilities, available video quality levels, current network conditions, and current server load. For end users, the major benefits of HAS compared to classical HTTP video streaming are reduced interruptions of the video playback and higher bandwidth utilization, which both generally result in a higher QoE. Adaptation is possible by changing the frame rate, resolution, or quantization of the video, which can be done with various adaptation strategies and related client- and server-side actions. The technical development of HAS, existing open standardized solutions, but also proprietary solutions are reviewed in this paper as fundamental to derive the QoE influence factors that emerge as a result of adaptation. The main contribution is a comprehensive survey of QoE related works from human computer interaction and networking domains, which are structured according to the QoE impact of video adaptation. To be more precise, subjective studies that cover QoE aspects of adaptation dimensions and strategies are revisited. As a result, QoE influence factors of HAS and corresponding QoE models are identified, but also open issues and conflicting results are discussed. Furthermore, technical influence factors, which are often ignored in the context of HAS, affect perceptual QoE influence factors and are consequently analyzed. This survey gives the reader an overview of the current state of the art and recent developments. At the same time, it targets networking researchers who develop new solutions for HTTP video streaming or assess video streaming from a user centric point of view. Therefore, this paper is a major step toward truly improving HAS.

A Survey on Quality of Experience of HTTP Adaptive Streaming

Working memory is important for online language processing during conversation. We use it to maintain relevant information, to inhibit or ignore irrelevant information, and to attend to conversation selectively. Working memory helps us to keep track of and actively participate in conversation, including taking turns and following the gist. This paper examines the Ease of Language Understanding model (i.e., the ELU model, Ronnberg, 2003; Ronnberg et al., 2008) in light of new behavioral and neural findings concerning the role of working memory capacity (WMC) in uni-modal and bimodal language processing. The new ELU model is a meaning prediction system that depends on phonological and semantic interactions in rapid implicit and slower explicit processing mechanisms that both depend on WMC albeit in different ways. It is based on findings that address the relationship between WMC and (a) early attention processes in listening to speech, (b) signal processing in hearing aids and its effects on short-term memory, (c) inhibition of speech maskers and its effect on episodic long-term memory, (d) the effects of hearing impairment on episodic and semantic long-term memory, and finally, (e) listening effort. New predictions and clinical implications are outlined. Comparisons with other WMC and speech perception models are made.

/pdf/the-ease-of-language-understanding-elu-model-theoretical-12jopklc1i.pdf

The Ease of Language Understanding (ELU) model: Theoretical, empirical, and clinical advances

With the increasing demand for video-based applications, the reliable prediction of video quality has increased in importance. Numerous video quality assessment methods and metrics have been proposed over the past years with varying computational complexity and accuracy. In this paper, we introduce a classification scheme for full-reference and reduced-reference media-layer objective video quality assessment methods. Our classification scheme first classifies a method according to whether natural visual characteristics or perceptual (human visual system) characteristics are considered. We further subclassify natural visual characteristics methods into methods based on natural visual statistics or natural visual features. We subclassify perceptual characteristics methods into frequency or pixel-domain methods. According to our classification scheme, we comprehensively review and compare the media-layer objective video quality models for both standard resolution and high definition video. We find that the natural visual statistics based MultiScale-Structural SIMilarity index (MS-SSIM), the natural visual feature based Video Quality Metric (VQM), and the perceptual spatio-temporal frequency-domain based MOtion-based Video Integrity Evaluation (MOVIE) index give the best performance for the LIVE Video Quality Database.

Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison

Foundations of modern auditory theory

Among the key topics of discussion and research on three-dimensional television (3D-TV), visual comfort is certainly one of the most critical. This is because it is well known that some viewers experience visual discomfort when looking at stereoscopic displays. It is important to properly address the issue of visual comfort to avoid possible delays in the deployment of 3D-TV. Here we present a concise overview of the main topics relevant to comfort in viewing stereoscopic television and survey the key factors influencing visual comfort. Potential end users of 3D-TV, content creators, program providers, broadcasters, display manufacturers and researchers will find this overview useful.

Stereoscopic 3D-TV: Visual Comfort

It is well known that some viewers experience visual discomfort when looking at stereoscopic displays One of the factors that can give rise to visual discomfort is the presence of large horizontal disparities The relationship between excessive horizontal disparity and visual comfort has been well documented for the case in which disparity magnitude does not change across space and time, eg for objects in still images Much less is known about the case in which
disparity magnitude varies over time, eg, objects moving in depth at some velocity In this study, we investigated the relationship between binocular disparity, object motion and visual comfort using computer-generated stereoscopic video sequences Specifically, viewers were asked to rate the visual comfort of stereoscopic sequences that had objects moving periodically back and forth in depth These sequences varied with respect to the number, size, position in depth, and velocity of movement of the objects in the scene The results indicate that change in disparity magnitude over time might be more important in determining visual comfort than the absolute magnitude of the disparity per se The results also suggest that rapid switches between crossed and uncrossed disparities might negatively affect visual comfort

Effect of disparity and motion on visual comfort of stereoscopic images

With the constant evolution of video technology and the deployment of new video services, content providers and broadcasters always face the challenge of delivering an adequate video quality which meets end-users expectations. The development of reliable quality testing and quality monitoring tools that can be used by broadcasters ultimately requires reliable objective video quality metrics. In turn, the validation of these objective models requires reliable subjective assessment, the most accurate representation of the quality perceived by end-users. Many different subjective assessment methodologies exist, and each has its advantages and drawbacks. One important element in a subjective testing methodology is the choice of the rating scale. In this paper, we make a direct comparison between four scales, which are either included in existing international standards or proposed to be used in future standardization activities related to video quality. We examine the subjective data from the points of view of response behavior from participants, similarity and variability of subjective scores. We discuss these results within the context of the subjective quality assessment of high-definition video compressed and transmitted over error-prone networks. Our experimental data show no overall statistical differences between the different scales. Results also show that the single-stimulus presentation provides highly repeatable results even if different scales or groups of participants are used.

Study of Rating Scales for Subjective Quality Assessment of High-Definition Video

The purpose of this study is to propose a quality metric of video encoded with variable frame rates and quantization parameters suitable for mobile video broadcasting applications. As a first step, experiments are conducted to assess the subjective quality of video sequences encoded with variable frame rates and quantization parameters. Resulting experimental data show that for the purpose of video rate control, optimization using the classical PSNR does not match up to that of subjective quality data. The second step bridges this gap between PSNR and subjective quality data by constructing a new quality metric that accounts for both encoding parameters (quantization and frame rate), and intrinsic video sequence characteristics (motion speed). The average correlation coefficient for five video sequences tested is as high as 0.93 with the proposed metric, in contrast with the PSNR's 0.70

Video Quality Metric for Bit Rate Control via Joint Adjustment of Quantization and Frame Rate

Depth image based rendering (DIBR) is a method for converting 2D material to stereoscopic 3D. With DIBR, information contained in a gray-level (luminance intensity) depth map is used to shift pixels in the 2D image to generate a new image as if it were captured from a new viewpoint. The larger the shift (binocular parallax), the larger is the perceived depth of the generated stereoscopic pair. However, a major problem with DIBR is that the shifted pixels now occupy new positions and leave areas that they originally occupied "empty." These disoccluded regions have to be filled properly, otherwise they can degrade image quality. In this study we investigated different methods for filling these disoccluded regions: (a) Filling regions with a constant color, (b) filling regions with horizontal linear interpolation of values on the hole border, (c) solving the Laplace equation on the hole boundary and propagate the values inside the region, (d) horizontal extrapolation with depth information taken into account, (e) variational inpainting with depth information taken into account, and (f) preprocessing of the depth map to prevent disoccluded regions from appearing. The methods differed in the time required for computing and filling, and the appearance of the filled-in regions. We assessed the subjective image quality outcome for several stereoscopic test images in which the left-eye view was the source and the right-eye view was a rendered view, in line with suggestions in the literature for the asymmetrical coding of stereoscopic images.

Filippo Speranza

Papers

Stereoscopic 3D-TV: Visual Comfort

Effect of disparity and motion on visual comfort of stereoscopic images

Study of Rating Scales for Subjective Quality Assessment of High-Definition Video

Video Quality Metric for Bit Rate Control via Joint Adjustment of Quantization and Frame Rate

Stereoscopic imaging: filling disoccluded areas in depth image-based rendering