With the wide applications of colored point cloud (CPC) in many fields, many attentions have been paid to CPC's distortions caused by its compression and reconstruction. How to effectively evaluate the visual quality of CPC has become an urgent issue to be resolved. In this paper, a Point cloud projection and Multi-scale feature fusion network based Blind Visual Quality Assessment method (denoted as PM-BVQA) is proposed for CPC. CPC in 3D space is first projected into 2D color projection map and geometric projection map, then a multi-scale feature fusion network is designed to blindly evaluate the visual quality of CPC. The proposed PM-BVQA method includes three modules, that is, joint color-geometric feature extractor, two-stage multi-scale feature fusion, and spatial pooling module. Considering the multi-channel characteristics of human visual system (HVS), unimodal features of different scales are obtained by joint color-geometric feature extractor from the color and geometric projection maps. The fusion of the unimodal color and geometric features is carried out to capture the cross-modal complementary information between these two types of information. By integrating cross-modal fused features at different scales, the complementary relationships between different channels of HVS are simulated. The spatial pooling module takes into account the attention mechanism of HVS and realizes the weighted summation of local regional quality to obtain the final global quality score of CPC. A subjective CPC database with coding distortion is used to verify the effectiveness of the proposed method, and the experimental results show that the proposed blind quality assessment method is more consistent with the subjective visual perception than the existing quality assessment methods.

https://dl.acm.org/doi/pdf/10.1145/3474085.3475645

Point Cloud Projection and Multi-Scale Feature Fusion Network Based Blind Quality Assessment for Colored Point Clouds

Over the past decade, three-dimensional (3D) graphics have become highly detailed to mimic the real world, exploding their size and complexity. Certain applications and device constraints necessitate their simplification and/or lossy compression, which can degrade their visual quality. Thus, to ensure the best Quality of Experience, it is important to evaluate the visual quality to accurately drive the compression and find the right compromise between visual quality and data size. In this work, we focus on subjective and objective quality assessment of textured 3D meshes. We first establish a large-scale dataset, which includes 55 source models quantitatively characterized in terms of geometric, color, and semantic complexity, and corrupted by combinations of five types of compression-based distortions applied on the geometry, texture mapping, and texture image of the meshes. This dataset contains over 343k distorted stimuli. We propose an approach to select a challenging subset of 3,000 stimuli for which we collected 148,929 quality judgments from over 4,500 participants in a large-scale crowdsourced subjective experiment. Leveraging our subject-rated dataset, a learning-based quality metric for 3D graphics was proposed. Our metric demonstrates state-of-the-art results on our dataset of textured meshes and on a dataset of distorted meshes with vertex colors. Finally, we present an application of our metric and dataset to explore the influence of distortion interactions and content characteristics on the perceived quality of compressed textured meshes.

https://dl.acm.org/doi/pdf/10.1145/3592786

Textured Mesh Quality Assessment: Large-scale Dataset and Deep Learning-based Quality Metric

The current researches on 3D point cloud quality evaluation focus on the full reference methods, but the blind quality evaluation has more advantages in actual applications. Therefore, a Blind Quality Evaluator for Colored Point Cloud (CPC) based on Visual Perception (BQE-CVP) is proposed. Firstly, considering the visual masking effect of CPC's geometric information, CPC is segmented into different regions and distributed with different weights to describe the influence of visual masking effect in CPC quality assessment. At the same time, a geometric combination feature vector is defined. Secondly, considering the visual masking effect of CPC's color information, the CPC is projected to the plane and the corresponding visual perception features are extracted. Then, the CPC's color combination feature vector is extracted in the 3D domain to better reflect the color distortion. Thirdly, the joint feature of CPC is extracted as the supplement of the geometric and color features. Finally, all the extracted features are constituted as a feature vector to predict the quality of CPC. Experimental results on three databases (CPCD2.0, IRPC and SJTU-PCQA) show that the proposed metric is better than relevant existing methods.

BQE-CVP: Blind Quality Evaluator for Colored Point Cloud Based on Visual Perception

From the perspective of optimizing the visual experience of the omnidirectional system, this article explores the terminal image quality of the high dynamic range omnidirectional image (HOI) system for the first time. The HOI system has higher contrast and wider field of view perception, but its complex imaging process poses a challenge to quality measurement. In this article, we propose a novel quality measurement method for HOI (QM-HOI) system based on distortion perception and brain information-processing mechanism. Because HOI system may encounter single distortion or mixed distortion, the proposed method consists of two modules, i.e., local progressive distortion model (LPDM) and global mixed distortion model (GMDM). LPDM is designed to mainly extract local features based on the different perception effects that tone mapping (TM) operators impart to coding distortion in different luminance regions. GMDM is designed to jointly consider the structure distortion caused by coding and the color distortion of TM. The extracted features are formed into feature vectors to predict the visual quality of the distorted HOIs. A new HOI subjective evaluation dataset with multidistortion is established, called NBU-HOID, specialized for the design and testing of the proposed method. Experimental results show that the objective measurement results of the proposed method is better than the state-of-the-art methods, and more in line with the human perception. We will also provide the NBU-HOID dataset free of charge shortly afterward.

Quality Measurement for High Dynamic Range Omnidirectional Image Systems

3D point clouds have found a wide variety of applications in multimedia processing, remote sensing, and scientific computing. Although most point cloud processing systems are developed to improve viewer experiences, little work has been dedicated to perceptual quality assessment of 3D point clouds. In this work, we build a new 3D point cloud database, namely the Waterloo Point Cloud (WPC) database. In contrast to existing datasets consisting of small-scale and low-quality source content of constrained viewing angles, the WPC database contains 20 high quality, realistic, and omni-directional source point clouds and 740 diversely distorted point clouds. We carry out a subjective quality assessment experiment over the database in a controlled lab environment. Our statistical analysis suggests that existing objective point cloud quality assessment (PCQA) models only achieve limited success in predicting subjective quality ratings. We propose a novel objective PCQA model based on an attention mechanism and a variant of information content-weighted structural similarity, which significantly outperforms existing PCQA models. The database has been made publicly available at https://github.com/qdushl/Waterloo-Point-Cloud-Database. 

/pdf/perceptual-quality-assessment-of-colored-3d-point-clouds-2rcbbjw9.pdf

Perceptual Quality Assessment of Colored 3D Point Clouds

https://ietresearch.onlinelibrary.wiley.com/doi/pdf/10.1049/ipr2.12211

CPC-GSCT: Visual quality assessment for coloured point cloud based on geometric segmentation and colour transformation

Towards A Colored Point Cloud Quality Assessment Method Using Colored Texture And Curvature Projection

Color point clouds can provide users more realistic visual information and better immersive experience than traditional imaging techniques. How to evaluate the visual quality of color point clouds accurately is an important issue to be solved urgently. In this work, we propose a novel full reference metric, called as Visual Quality Assessment of Color Point Clouds (VQA-CPC). Starting from the geometry and texture of color point cloud, the proposed metric calculates the distances from color point cloud’s points to their geometric centroid and the distances from the texture coordinates of the points to texture centroid. Then, a measuring distortion strategy based on distortion measurement is designed and used to extract the features of color point cloud. Finally, the extracted geometric features and texture features are used to construct the feature vector and predict quality of the distorted color point cloud. Moreover, we construct a color point cloud database, called as NBU-PCD1.0, for verifying the effectiveness of the proposed metric. Experimental results show that the proposed VQA-CPC metric is better than the existing point cloud metrics.

Zhouyan He

Papers

CPC-GSCT: Visual quality assessment for coloured point cloud based on geometric segmentation and colour transformation

Towards A Colored Point Cloud Quality Assessment Method Using Colored Texture And Curvature Projection

BQE-CVP: Blind Quality Evaluator for Colored Point Cloud Based on Visual Perception

VQA-CPC: a novel visual quality assessment metric of color point clouds

Blind Tone Mapped Image Quality Assessment with Image Segmentation and Visual Perception