Change detection is a basic task of remote sensing image processing. The research objective is to identify the change information of interest and filter out the irrelevant change information as interference factors. Recently, the rise in deep learning has provided new tools for change detection, which have yielded impressive results. However, the available methods focus mainly on the difference information between multitemporal remote sensing images and lack robustness to pseudochange information. To overcome the lack of resistance in current methods to pseudochanges, in this article, we propose a new method, namely, dual attentive fully convolutional Siamese networks, for change detection in high-resolution images. Through the dual attention mechanism, long-range dependencies are captured to obtain more discriminant feature representations to enhance the recognition performance of the model. Moreover, the imbalanced sample is a serious problem in change detection, i.e., unchanged samples are much more abundant than changed samples, which is one of the main reasons for pseudochanges. We propose the weighted double-margin contrastive loss to address this problem by punishing attention to unchanged feature pairs and increasing attention to changed feature pairs. The experimental results of our method on the change detection dataset and the building change detection dataset demonstrate that compared with other baseline methods, the proposed method realizes maximum improvements of 2.9% and 4.2%, respectively, in the F 1 score. Our PyTorch implementation is available at https://github.com/lehaifeng/DASNet .

DASNet: Dual Attentive Fully Convolutional Siamese Networks for Change Detection in High-Resolution Satellite Images

Nowadays, 360° video/image has been increasingly popular and drawn great attention. The spherical viewing range of 360° video/image accounts for huge data, which pose the challenges to 360° video/image processing in solving the bottleneck of storage, transmission, etc. Accordingly, the recent years have witnessed the explosive emergence of works on 360° video/image processing. In this article, we review the state-of-the-art works on 360° video/image processing from the aspects of perception, assessment and compression. First, this article reviews both datasets and visual attention modelling approaches for 360° video/image. Second, we survey the related works on both subjective and objective visual quality assessment (VQA) of 360° video/image. Third, we overview the compression approaches for 360° video/image, which either utilize the spherical characteristics or visual attention models. Finally, we summarize this overview article and outlook the future research trends on 360° video/image processing.

State-of-the-Art in 360° Video/Image Processing: Perception, Assessment and Compression

/pdf/artificial-intelligence-for-the-metaverse-a-survey-3tfftt9g.pdf

Artificial Intelligence for the Metaverse: A Survey

Nowadays, 360° video/image has been increasingly popular and drawn great attention. The spherical viewing range of 360° video/image accounts for huge data, which pose the challenges to 360° video/image processing in solving the bottleneck of storage, transmission, etc. Accordingly, the recent years have witnessed the explosive emergence of works on 360° video/image processing. In this paper, we review the state-of-the-art works on 360° video/image processing from the aspects of perception, assessment and compression. First, this paper reviews both datasets and visual attention modelling approaches for 360° video/image. Second, we survey the related works on both subjective and objective visual quality assessment (VQA) of 360° video/image. Third, we overview the compression approaches for 360° video/image, which either utilize the spherical characteristics or visual attention models. Finally, we summarize this overview paper and outlook the future research trends on 360° video/image processing.

State-of-the-art in 360{\deg} Video/Image Processing: Perception, Assessment and Compression

Due to rapid growth in Virtual Reality (VR) technology, the industry of VR is expected to grow around $26.89 billion by 2022. However, with its extensive growth and immersive inclusion in human life, health-related issues are reported including, but not limited to nauseated feeling, vomiting, dizziness and cold sweats. These issues introduce a well-known side effect termed as motion sickness in VR users. Consequently, motion sickness limits the VR community in the full adaptation of this immersive technology. Since there is no lack of literature investigating motion sickness caused by VR, yet researches on the effect of VR on human's physiology is still in its infancy. This study presents novel findings, by comparing different factors such as gender, motion sickness experience, 3D games experience and VR experience. Furthermore, it reports the impact of concerning factors in a within-subjects design (46 participants participated in an experiment) under different virtual environment genres. The key findings of this article report that there is a significant difference in the amount of motion sickness when shifting from pleasant to the horror genre of the environment and having a strong dependence on gender. Moreover, the type of virtual environment is an essential factor that has a notable effect on the user's blood pressure, blood sugar and heart rate. However, past experiences with motion sickness and 3D games show no significant impact on the user's level of motion sickness.

/pdf/motion-sickness-in-virtual-reality-an-empirical-evaluation-4vokl0adu5.pdf

Motion Sickness in Virtual Reality: An Empirical Evaluation

Virtual reality (VR), a new type of simulation and interaction technology, has aroused widespread attention and research interest. It is necessary to evaluate the VR quality and provide a standard for the rapidly developing technology. To the best of our knowledge, a few researchers have built benchmark databases and designed related algorithms, which has hindered the further development of the VR technology. In this paper, a free available data set (VRQ-TJU) for VR quality assessment is proposed with subjective scores for each sample data. The validity for the designed database has been proved based on the traditional multimedia quality assessment metrics. In addition, an end-to-end 3-D convolutional neural network is introduced to predict the VR video quality without a referenced VR video. This method can extract spatiotemporal features and does not require using hand-crafted features. At the same time, a new score fusion strategy is designed based on the characteristics of the VR video projection process. Taking the pre-processed VR video patches as input, the network captures local spatiotemporal features and gets the score of every patch. Then, the new quality score fusion strategy is applied to get the final score. Such approach shows advanced performance on this database.

3D Panoramic Virtual Reality Video Quality Assessment Based on 3D Convolutional Neural Networks

Panoramic video and stereoscopic panoramic video are essential carriers of virtual reality content, so it is very crucial to establish their quality assessment models for the standardization of virtual reality industry. However, it is very challenging to evaluate the quality of the panoramic video at present. One reason is that the spatial information of the panoramic video is warped due to the projection process, and the conventional video quality assessment (VQA) method is difficult to deal with this problem. Another reason is that the traditional VQA method is problematic to capture the complex global time information in the panoramic video. In response to the above questions, this paper presents an end-to-end neural network model to evaluate the quality of panoramic video and stereoscopic panoramic video. Compared to other panoramic video quality assessment methods, our proposed method combines spherical convolutional neural networks (CNN) and non-local neural networks, which can effectively extract complex spatiotemporal information of the panoramic video. We evaluate the method in two databases, VRQ-TJU and VR-VQA48. Experiments show the effectiveness of different modules in our method, and our method outperforms state-of-the-art other related methods.

/pdf/panoramic-video-quality-assessment-based-on-non-local-5epzqigbt1.pdf

Panoramic Video Quality Assessment Based on Non-Local Spherical CNN

The invention relates to an immersive virtual reality quality evaluation method based on human visual perception process. The method comprisesrandomly extracting the distorted virtual reality data, calculating the natural statistical characteristics of the left frame Vl and the right frame Vr of the extracted distorted VR map respectively, and extracting the feature vector by adopting an equalizedGaussian distribution fitting method; performing comfort analysis oriented to virtual reality and computingreference VR disparity map and distorted VR disparity map, computingglobal correlation coefficient and mean square error MSE to generate comfort oriented eigenvectors. For the immersive analysis of the virtual reality scene, the depth map fusion calculation is performed on the left and rightviewpoint data of the random frame, and the structural similarity map SC-SSIM in the multi-scale background is obtained, and the gradient map is obtained on the basis of the structure; the physical distortion and comfort are used to evaluate the quality of visual acceptance experience in virtual reality; the physical distortion and immersion are used to evaluate the perceptual experience qualityof visual information in virtual reality, and the final evaluation results are obtained.

Tianlin Liu

Papers

3D Panoramic Virtual Reality Video Quality Assessment Based on 3D Convolutional Neural Networks

Panoramic Video Quality Assessment Based on Non-Local Spherical CNN

Quality evaluation method of immersive virtual reality based on human visual perception process