Musicophilia. Tales of Music and the Brain.

Three-dimensional video content has attracted much attention in both the cinema and television industries, because 3D is considered to be the next key feature that can significantly enhance the visual experience of viewers. However, one of the major challenges is the difficulty in providing high quality images that are comfortable to view and that also meet signal transmission requirements over a limited bandwidth for display on television screens. The different processing steps that are necessary in a 3D-TV delivery chain can all introduce artifacts that may create problems in terms of human visual perception. In this paper, we highlight the importance of considering 3D visual attention when addressing 3D human factors issues. We provide a review of the field of 3D visual attention, discuss the challenges in both the understanding and modeling of 3D visual attention, and provide guidance to researchers in this field. Finally, we identify perceptual issues generated during the various steps in a typical 3D-TV broadcasting delivery chain, review them and explain how consideration of 3D visual attention modeling can help improve the overall 3D viewing experience.

/pdf/the-importance-of-visual-attention-in-improving-the-3d-tv-i10891qjzd.pdf

The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives

A video compression framework based on spatio-temporal resolution adaptation (ViSTRA) is proposed, which dynamically resamples the input video spatially and temporally during encoding, based on a quantisation-resolution decision, and reconstructs the full resolution video at the decoder. Temporal upsampling is performed using frame repetition, whereas a convolutional neural network super-resolution model is employed for spatial resolution upsampling. ViSTRA has been integrated into the high efficiency video coding reference software (HM 16.14). Experimental results verified via an international challenge show significant improvements, with BD-rate gains of 15% based on PSNR and an average MOS difference of 0.5 based on subjective visual quality tests.

/pdf/video-compression-based-on-spatio-temporal-resolution-42kwsat56n.pdf

Video Compression Based on Spatio-Temporal Resolution Adaptation

Deaf and hard-of-hearing (DHH) individuals encounter difficulties when engaged in group conversations with hearing individuals, due to factors such as simultaneous utterances from multiple speakers and speakers whom may be potentially out of view. We interviewed and co-designed with eight DHH participants to address the following challenges: 1) associating utterances with speakers, 2) ordering utterances from different speakers, 3) displaying optimal content length, and 4) visualizing utterances from out-of-view speakers. We evaluated multiple designs for each of the four challenges through a user study with twelve DHH participants. Our study results showed that participants significantly preferred speechbubble visualizations over traditional captions. These design preferences guided our development of SpeechBubbles, a real-time speech recognition interface prototype on an augmented reality head-mounted display. From our evaluations, we further demonstrated that DHH participants preferred our prototype over traditional captions for group conversations.

SpeechBubbles: Enhancing Captioning Experiences for Deaf and Hard-of-Hearing People in Group Conversations

This paper introduces a new high-definition video quality database, referred to as BVI-HD, which contains 32 reference and 384 distorted video sequences plus subjective scores. The reference material in this database was carefully selected to optimize the coverage range and distribution uniformity of five low-level video features, while the included 12 distortions, using both original high efficiency video coding (HEVC) and HEVC with synthesis mode, represent state-of-the-art approaches to compression. The range of quantization parameters included in the database for HEVC compression was determined by a subjective study, the results of which indicate that a wider range of QP values should be used than the current recommendation. The subjective opinion scores for all 384 distorted videos were collected from a total of 86 subjects, using a double stimulus test methodology. Based on these results, we compare the subjective quality between HEVC and synthesised content, and evaluate the performance of nine state-of-the-art, full-reference objective quality metrics. This database has now been made available online, representing a valuable resource to those concerned with compression performance evaluation and objective video quality assessment.

/pdf/bvi-hd-a-video-quality-database-for-hevc-compressed-and-2wtpn7x6mv.pdf

BVI-HD: A Video Quality Database for HEVC Compressed and Texture Synthesized Content

Subtitles (closed captions) on television are typically placed at the bottom-centre of the screen. However, placing subtitles in varying positions, according to the underlying video content (`dynamic subtitles'), has the potential to make the overall viewing experience less disjointed and more immersive. This paper describes the testing of such subtitles with hearing-impaired users, and a new analysis of previously collected eye-tracking data. The qualitative data demonstrates that dynamic subtitles can lead to an improved User Experience, although not for all types of subtitle user. The eye-tracking data was analysed to compare the gaze patterns of subtitle users with a baseline of those for people viewing without subtitles. It was found that gaze patterns of people watching dynamic subtitles were closer to the baseline than those of people watching with traditional subtitles. Finally, some of the factors that need to be considered when authoring dynamic subtitles are discussed.

/pdf/dynamic-subtitles-the-user-experience-1ljhdbo7lt.pdf

Dynamic Subtitles: The User Experience

The term `object-based media' has become commonly used to describe the representation of media content by a set of individual assets together with metadata describing their relationships and associations. At the point of consumption these objects can be assembled to create an overall user experience. The precise combination of objects can be flexible, and responsive to user, environmental and platform specific factors. An object-based approach like this can serve end users more effectively; by optimising the experience to best suit their access requirements, the characteristics of their playback platform or personal preferences. This paper presents an overview of the key characteristics and potential benefits of Object-Based Broadcasting and a case study of an object-based audio documentary, which can change duration in response to implicit or explicit input from a listener. Further examples will be given to illustrate the capabilities of a future object-based broadcasting system.

Object-based broadcasting - curation, responsiveness and user experience

Currently there exists no agreed-upon user experience guidelines regarding subtitling (closed captions) in immersive 360-degree video experiences. It is not clear how subtitles might be acceptably displayed within this context, namely to support the balance between comprehension, freedom to look around the scene, and immersion. This work-in-progress describes four subtitle behaviours that we have designed and implemented in order to perform user-testing. We describe our rationale for each behaviour and discuss our initial hypotheses surrounding a full empirical investigation.

Subtitles in 360-degree Video

The frame and field rates that have been used for television since the 1930s cause problems for motion portrayal, which are increasingly evident on the large, high-resolution television displays that are now common. This paper reports on a program of experimental work that successfully demonstrated the advantages of higher frame rate capture and display as a means of improving the quality of television systems of all spatial resolutions. Additional benefits were identified from the use of high frame-rate capture for the production of programs to be viewed using conventional televisions. Ways to mitigate some of the production and distribution issues that high frame-rate television implies are suggested.

http://ieeexplore.ieee.org/iel7/7261654/7270605/07270625.pdf

High Frame-Rate Television

When creating digital artefacts, it is important to ensure that the product being made is accessible to as much of the population as is possible. Many guidelines and supporting tools exist to assist reaching this goal. However, little is known about developers' understanding of accessible practice and the methods that are used to implement this. We present findings from an accessibility design workshop that was carried out with a mixture of 197 developers and digital technology students. We discuss perceptions of accessibility, techniques that are used when designing accessible products, and what areas of accessibility development participants believed were important. We show that there are gaps in the knowledge needed to develop accessible products despite the effort to promote accessible design. Our participants are themselves aware of where these gaps are and have suggested a number of areas where tools, techniques and guidance would improve their practice.

/pdf/developing-accessible-services-understanding-current-2b79idw4f0.pdf

Mike Armstrong

Papers

Dynamic Subtitles: The User Experience

Object-based broadcasting - curation, responsiveness and user experience

Subtitles in 360-degree Video

High Frame-Rate Television

Developing Accessible Services: Understanding Current Knowledge and Areas for Future Support