Photoplethysmography and its application in clinical physiological measurement

Nowadays, 360° video/image has been increasingly popular and drawn great attention. The spherical viewing range of 360° video/image accounts for huge data, which pose the challenges to 360° video/image processing in solving the bottleneck of storage, transmission, etc. Accordingly, the recent years have witnessed the explosive emergence of works on 360° video/image processing. In this article, we review the state-of-the-art works on 360° video/image processing from the aspects of perception, assessment and compression. First, this article reviews both datasets and visual attention modelling approaches for 360° video/image. Second, we survey the related works on both subjective and objective visual quality assessment (VQA) of 360° video/image. Third, we overview the compression approaches for 360° video/image, which either utilize the spherical characteristics or visual attention models. Finally, we summarize this overview article and outlook the future research trends on 360° video/image processing.

State-of-the-Art in 360° Video/Image Processing: Perception, Assessment and Compression

HTTP streaming has become a cost-effective means for multimedia delivery nowadays. For adaptivity to networks and terminals, a provider should generate multiple representations of an original video as well as the related metadata. Recently, there have been various adaptation methods to support adaptive HTTP streaming. In this paper, we investigate typical adaptation methods in the context of live video streaming. We first discuss the trade-off among typical adaptation methods. The evaluation and comparison are then carried out not only in terms of bitrate and buffer behaviors but also in terms of the perceptual impact on end users. It is found that the perceptual impact depends not only on adaptation method but also on the content itself. We also show that the preparation of representation sets may affect the behaviors of some adaptation methods.

An Evaluation of Bitrate Adaptation Methods for HTTP Live Streaming

The increasing popularity of video (i.e., audio-visual) applications or services over both wired and wireless links has prompted recent growing interests in the investigations of quality of experience (QoE) in online video transmission. Conventional video quality metrics, such as peak-signal-to-noise-ratio and quality of service, only focus on the reception quality from the systematic perspective. As a result, they cannot represent the true visual experience of an individual user. Instead, the QoE introduces a user experience-driven strategy which puts special emphasis on the contextual and human factors in addition to the transmission system. This advantage has raised the popularity and widespread usage of QoE in video transmission. In this paper, we present an overview of selected issues pertaining to QoE and its recent applications in video transmission, with consideration of the compelling features of QoE (i.e., context and human factors). The selected issues include QoE modeling with influence factors in the end-to-end chain of video transmission, QoE assessment (including subjective test and objective QoE monitoring) and QoE management of video transmission over different types of networks. Through the literature review, we observe that the context and human factors in QoE-aware video transmission have attracted significant attentions since the past two to three years. A vast number of high quality works were published in this area, and will be highlighted in this survey. In addition to a thorough summary of recent progresses, we also present an outlook of future developments on QoE assessment and management in video transmission, especially focusing on the context and human factors that have not been addressed yet and the technical challenges that have not been completely solved so far. We believe that our overview and findings can provide a timely perspective on the related issues and the future research directions in QoE-oriented services over video communications.

QoE in Video Transmission: A User Experience-Driven Strategy

Omnidirectional or 360° video is increasingly being used, mostly due to the latest advancements in immersive Virtual Reality (VR) technology. However, its wide adoption is hindered by the higher bandwidth and lower latency requirements than associated with traditional video content delivery. Diverse researchers propose and design solutions that help support an immersive visual experience of 360° video, primarily when delivered over a dynamic network environment. This paper presents the state-of-the-art on adaptive 360° video delivery solutions considering end-to-end video streaming in general and then specifically of 360° video delivery. Current and emerging solutions for adaptive 360° video streaming, including viewport-independent, viewport-dependent, and tile-based schemes are presented. Next, solutions for network-assisted unicast and multicast streaming of 360° video content are discussed. Different research challenges for both on-demand and live 360° video streaming are also analyzed. Several proposed standards and technologies and top international research projects are then presented. We demonstrate the ongoing standardization efforts for 360° media services that ensure interoperability and immersive media deployment on a massive scale. Finally, the paper concludes with a discussion about future research opportunities enabled by 360° video.

A Survey on Adaptive 360° Video Streaming: Solutions, Challenges and Opportunities

In this paper, we propose a new adaptation approach for viewport-adaptive streaming of 360-degree videos over the Internet. The proposed approach is able to systematically decide versions of tiles according to user head movements and network conditions by taking into account not only viewport estimation errors, but also users head movements in each segment duration. The experimental results show that the proposed approach can effectively adapt 360-degree videos to both varying network conditions and user’s head movements. Compared to existing approaches, the proposed approach can improve the average viewport quality by up to 3.8 dB and reduce the standard deviation of the viewport quality by up to 1.1 dB. Also, the impacts of the segment duration and the buffer size are investigated. It is found that long segment durations and large buffer sizes have significant impacts on the performances of tile selection methods.

An Optimal Tile-Based Approach for Viewport-Adaptive 360-Degree Video Streaming

Live and on-demand video streaming has recently contributed a significant part of the Internet traffic. In order to guarantee the best possible user experience, multimedia service is required to support large fluctuations in different network conditions. One of the cost effective means for multimedia delivery nowadays is HTTP streaming. The key challenge of this technology is how to change adaptively video bitrates according to network variation and client buffer while bestowing upon users the best possible video quality. In this paper, we propose a novel rate-adaptation method on client side, which allows clients to estimate buffer level in the future, and thus achieving a balance between the need for high video quality and buffer stability. The experiments show that our method performs well under varying network conditions.

Buffer-based bitrate adaptation for adaptive HTTP streaming

HTTP streaming, which is currently based on the pull-based HTTP/1.1 protocol, has a tradeoff problem between overhead and adaptivity. We propose an adaptation method for adaptive streaming over the new HTTP/2 protocol, leveraging its server push feature. The method is based on a cost function that takes into account the number of pushed segments and the client buffer level. Experiment results show that the proposed method can improve the balance between the number of requests and buffer stability compared to existing methods.

Adaptation method for video streaming over HTTP/2

In this paper, we propose a new adaptation approach for viewport-adaptive streaming of 360-degree videos over the Internet. The proposed approach is able to systematically decide quality levels of tiles according to user head movements and network conditions by taking into account not only prediction errors but also user head movements in each adaptation interval. Experimental results show that the proposed approach can effectively adapt 360-degree videos to both varying network conditions and user head movements. Compared to existing approaches, the proposed approach can improve the average viewport quality by up to 3.9dB and reduce the standard deviation of the viewport quality by up to 50%.

Duc V. Nguyen

Papers

An Optimal Tile-Based Approach for Viewport-Adaptive 360-Degree Video Streaming

Buffer-based bitrate adaptation for adaptive HTTP streaming

Adaptation method for video streaming over HTTP/2

A New Adaptation Approach for Viewport-adaptive 360-degree Video Streaming

A client-based adaptation framework for 360-degree video streaming