360-degree video provides an immersive experience to end-users through Virtual Reality (VR) Head-Mounted-Displays (HMDs). However, it is not trivial to understand the Quality of Experience (QoE) of 360-degree video since user experience is influenced by various factors that affect QoE when watching a 360-degree video in VR. This manuscript presents a machine learning-based QoE prediction of 360-degree video in VR, considering the two key QoE aspects: perceptual quality and cybersickness. In addition, we proposed two new QoE-affecting factors: user's familiarity with VR and user's interest in 360-degree video for the QoE evaluation. To aim this, we first conduct a subjective experiment on 96 video samples and collect datasets from 29 users for perceptual quality and cybersickness. We design a new Logistic Regression (LR) based model for QoE prediction in terms of perceptual quality. The prediction accuracy of the proposed model is compared against well-known supervised machine-learning algorithms such as k-Nearest Neighbors (kNN), Support Vector Machine (SVM), and Decision Tree (DT) with respect to accuracy rate, recall, f1-score, precision, and mean absolute error (MAE). LR performs well with 86% accuracy, which is in close agreement with subjective opinion. The prediction accuracy of the proposed model is then compared with existing QoE models in terms of perceptual quality. Finally, we build a Neural Network-based model for the QoE prediction in terms of cybersickness. The proposed model performs well against the state of the art QoE prediction methods in terms of cybersickness.

Subjective QoE of 360-Degree Virtual Reality Videos and Machine Learning Predictions

A review of scenario analysis methods in planning and operation of modern power systems: Methodologies, applications, and challenges

The groundbreaking evolution in mobile and wireless communication networks design in recent years, in combination with the advancement of mobile terminal equipment capabilities, has led in an exponential growth of mobile internet technologies, and arose an ever-growing demand for innovative multimedia services. The highly demanding in terms of network resources over-the-top media services, as well as the emergence of new and complex mobile multimedia services such as video gaming, ultra-high-definition video, and extended reality, requires the enhancement of end-users’ perceived quality of experience (QoE). QoE has garnered much research interest in recent years, and has emerged as a key component in the evaluation of network services and operations. As a result, a QoE-aware network planning approach is getting increasingly favored, and novel design challenges, such as how to quantify and measure QoE, have arisen. In this regard, a paradigm shift in network implementations is being envisioned, in which the focus will be on machine learning (ML) methodologies for developing QoE prediction models, directly related to end-user’s personalized experience. In this survey, an analysis on application-oriented, ML-based QoE prediction models for the goal of QoE management for multimedia services is presented. In addition, an examination of the state-of-the-art ML-based QoE predictive models and some of the innovative techniques and challenges related to multimedia services quality assessment with focus on extended reality and video gaming applications are outlined.

https://ieeexplore.ieee.org/ielx7/6287639/9668973/09706206.pdf

A Survey on Multimedia Services QoE Assessment and Machine Learning-Based Prediction

The groundbreaking evolution in mobile and wireless communication networks design in recent years, in combination with the advancement of mobile terminal equipment capabilities, has led in an exponential growth of mobile internet technologies, and arose an ever-growing demand for innovative multimedia services. The highly demanding in terms of network resources over-the-top media services, as well as the emergence of new and complex mobile multimedia services such as video gaming, ultra-high-definition video, and extended reality, requires the enhancement of end-users’ perceived quality of experience (QoE). QoE has garnered much research interest in recent years, and has emerged as a key component in the evaluation of network services and operations. As a result, a QoE-aware network planning approach is getting increasingly favored, and novel design challenges, such as how to quantify and measure QoE, have arisen. In this regard, a paradigm shift in network implementations is being envisioned, in which the focus will be on machine learning (ML) methodologies for developing QoE prediction models, directly related to end-user’s personalized experience. In this survey, an analysis on application-oriented, ML-based QoE prediction models for the goal of QoE management for multimedia services is presented. In addition, an examination of the state-of-the-art ML-based QoE predictive models and some of the innovative techniques and challenges related to multimedia services quality assessment with focus on extended reality and video gaming applications are outlined. 

/pdf/a-survey-on-multimedia-services-qoe-assessment-and-machine-6gkneqzg.pdf

When haptic signals are integrated with existing audio-visual dominated multimedia applications, multi-modal services that can satisfy an individual's immersive experience have emerged. In order to support multi-modal services, cross-modal communications come into being. However, facing collaborative transmission and comprehensive processing requirements of audio, visual, and haptic signals as well as their influences for user's immersive experience, the research of cross-modal communications is still in its infancy and needs to tackle many technical challenges. Due to great successes and powerful abilities of artificial intelligence (AI), it can be expected to underpin cross-modal communications. In this article, we try to deal with issues in cross-modal communications by using AI technology. Specifically, we first adopt the federated learning paradigm to solve sparse data collection and privacy protection problems in the immersive experience description of multi-modal services. Then, we resort to the reinforcement learning paradigm to construct a joint optimization framework of caching, communication, and computation, realizing collaborative transmission of audio, visual, and haptic streams. Finally, we attempt to use the transfer learning paradigm to extract, transfer, and fuse knowledge, semantics, and characteristics from different modalities, recovering corrupted signals and promoting rendering effects at the receiver. Experimental results validate the effectiveness of the AI-enabled cross-modal communications strategies.

AI-Enabled Cross-Modal Communications

Quality of experience (QoE) serves as a direct evaluation of users’ experiences in mobile video transmission and thus essential for network management, such as network optimization. In this paper, we propose a deep learning-based QoE prediction approach with a large-scale QoE dataset for mobile video transmission. Specifically, we develop a mobile phone application for collecting user QoE data when viewing videos transmitted over the mobile internet in a practical environment. Then, we construct a large-scale dataset by collecting over 80000 piece of data with four kinds of subjective scores and 89 network parameters. Each QoE metric is related to only some of the 89 network parameters. Therefore, we apply the feature selection method to find the feature parameters related to user scores. Additionally, the boxplot method is used to clean the raw data by removing outliers. Finally, a deep neural network (DNN) is developed to learn the relationships between the network parameters and the subjective QoE scores. The proposed DNN can also be seen as a data-driven objective QoE prediction approach for mobile video transmission, which can be used to predict the user QoE scores. The experimental results show that the proposed approach can effectively remove most features irrelevant to QoE prediction. Moreover, the performance of QoE prediction by the proposed model outperforms other state-of-the-art approaches.

Zhishen Meng

Papers

Learning QoE of Mobile Video Transmission With Deep Neural Network: A Data-Driven Approach