Point cloud based 3D visual representation is becoming popular due to its ability to exhibit the real world in a more comprehensive and immersive way. However, under a limited network bandwidth, it is very challenging to communicate this kind of media due to its huge data volume. Therefore, the MPEG have launched the standardization for point cloud compression (PCC), and proposed three model categories, i.e., TMC1, TMC2, and TMC3. Because the 3D geometry compression methods of TMC1 and TMC3 are similar, TMC1 and TMC3 are further merged into a new platform namely TMC13. In this paper, we first introduce some basic technologies that are usually used in 3D point cloud compression, then review the encoder architectures of these test models in detail, and finally analyze their rate distortion performance as well as complexity quantitatively for different cases (i.e., lossless geometry and lossless color, lossless geometry and lossy color, lossy geometry and lossy color) by using 16 benchmark 3D point clouds that are recommended by MPEG. Experimental results demonstrate that the coding efficiency of TMC2 is the best on average (especially for lossy geometry and lossy color compression) for dense point clouds while TMC13 achieves the optimal coding performance for sparse and noisy point clouds with lower time complexity.

A Comprehensive Study and Comparison of Core Technologies for MPEG 3-D Point Cloud Compression

In rate-distortion optimization, the encoder settings are determined by maximizing a reconstruction quality measure subject to a constraint on the bit rate. One of the main challenges of this approach is to define a quality measure that can be computed with low computational cost and which correlates well with perceptual quality. While several quality measures that fulfil these two criteria have been developed for images and video, no such one exists for 3D point clouds. We address this limitation for the video-based point cloud compression (V-PCC) standard by proposing a linear perceptual quality model whose variables are the V-PCC geometry and color quantization parameters and whose coefficients can easily be computed from two features extracted from the original 3D point cloud. Subjective quality tests with 400 compressed 3D point clouds show that the proposed model correlates well with the mean opinion score, outperforming state-of-the-art full reference objective measures in terms of Spearman rank-order and Pearsons linear correlation coefficient. Moreover, we show that for the same target bit rate, ratedistortion optimization based on the proposed model offers higher perceptual quality than rate-distortion optimization based on exhaustive search with a point-to-point objective quality metric.

Reduced Reference Perceptual Quality Model and Application to Rate Control for 3D Point Cloud Compression.

We propose using stationary Gaussian processes (GPs) to model the statistics of the signal on points in a point cloud, which can be considered samples of a GP at the positions of the points. Furthermore, we propose using Gaussian process transforms (GPTs), which are Karhunen–Loève transforms of the GP, as the basis of transform coding of the signal. Focusing on colored 3D point clouds, we propose a transform coder that breaks the point cloud into blocks, transforms the blocks using GPTs, and entropy codes the quantized coefficients. The GPT for each block is derived from both the covariance function of the GP and the locations of the points in the block, which are separately encoded. The covariance function of the GP is parameterized, and its parameters are sent as side information. The quantized coefficients are sorted by the eigenvalues of the GPTs, binned, and encoded using an arithmetic coder with bin-dependent Laplacian models, whose parameters are also sent as side information. Results indicate that transform coding of 3D point cloud colors using the proposed GPT and entropy coding achieves superior compression performance on most of our data sets.

Transform Coding for Point Clouds Using a Gaussian Process Model

In this paper, a Backward Attentive Fusing Network with Local Aggregation Classifier (BAF-LAC) is proposed to improve the performance of 3D point cloud semantic segmentation. It consists of a Backward Attentive Fusing Encoder-Decoder (BAF-ED) to learn semantic features and a Local Aggregation Classifier (LAC) to maintain the context-awareness of points. BAF-ED narrows the semantic gap between the encoder and the decoder via fusing multi-layer encoder features with the decoder features. High-level encoder features are transformed into an attention map to modulate low-level encoder features backward. LAC adaptively enhances the intermediate features in point-wise MLPs via aggregating the features of neighboring points into the center point. It takes the place of commonly used post-processing techniques and retains context consistency into the classifier. Equipped with these modules, BAF-LAC can extract discriminative semantic features and predict smoother results. Extensive experiments on Semantic3D, SemanticKITTI, and S3DIS demonstrate that the proposed method can achieve competitive results against the state-of-the-art methods.

Backward Attentive Fusing Network With Local Aggregation Classifier for 3D Point Cloud Semantic Segmentation

As 3D scanning devices and depth sensors advance, dynamic point clouds have attracted increasing attention as a format for 3D objects in motion, with applications in various fields such as immersive telepresence, navigation for autonomous driving and gaming. Nevertheless, the tremendous amount of data in dynamic point clouds significantly burden transmission and storage. To this end, we propose a complete compression framework for attributes of 3D dynamic point clouds, focusing on optimal inter-coding. Firstly, we derive the optimal inter-prediction and predictive transform coding assuming the Gaussian Markov Random Field model with respect to a spatio-temporal graph underlying the attributes of dynamic point clouds. The optimal predictive transform proves to be the Generalized Graph Fourier Transform in terms of spatio-temporal decorrelation. Secondly, we propose refined motion estimation via efficient registration prior to inter-prediction, which searches the temporal correspondence between adjacent frames of irregular point clouds. Finally, we present a complete framework based on the optimal inter-coding and our previously proposed intra-coding, where we determine the optimal coding mode from rate-distortion optimization with the proposed offline-trained $\lambda$-Q model. Experimental results show that we achieve around 17% bit rate reduction on average over competitive dynamic point cloud compression methods.

Predictive Generalized Graph Fourier Transform for Attribute Compression of Dynamic Point Clouds.

3D point clouds associated with attributes are considered as a promising paradigm for immersive communication. However, the corresponding compression schemes for this media are still in the infant stage. Moreover, in contrast to conventional image/video compression, it is a more challenging task to compress 3D point cloud data, arising from the irregular structure. In this paper, we propose a novel and effective compression scheme for the attributes of voxelized 3D point clouds. In the first stage, an input voxelized 3D point cloud is divided into blocks of equal size. Then, to deal with the irregular structure of 3D point clouds, a geometry-guided sparse representation (GSR) is proposed to eliminate the redundancy within each block, which is formulated as an $\ell _{0}$ -norm regularized optimization problem. Also, an inter-block prediction scheme is applied to remove the redundancy between blocks. Finally, by quantitatively analyzing the characteristics of the resulting transform coefficients by GSR, an effective entropy coding strategy that is tailored to our GSR is developed to generate the bitstream. Experimental results over various benchmark datasets show that the proposed compression scheme is able to achieve better rate-distortion performance and visual quality, compared with state-of-the-art methods.

3D Point Cloud Attribute Compression Using Geometry-Guided Sparse Representation

3D point clouds associated with attributes are considered as a promising data representation for immersive communication. The large amount of data, however, poses great challenges to the subsequent transmission and storage processes. In this letter, we propose a new compression scheme for the color attribute of static voxelized 3D point clouds. Specifically, we first partition the colors of a 3D point cloud into clusters by applying k-d tree to the geometry information, which are then successively encoded. To eliminate the redundancy, we propose a novel prediction module, namely graph prediction, in which a small number of representative points selected from previously encoded clusters are used to predict the points to be encoded by exploring the underlying graph structure constructed from the geometry information. Furthermore, the prediction residuals are transformed with the graph transform, and the resulting transform coefficients are finally uniformly quantified and entropy encoded. Experimental results show that the proposed compression scheme is able to achieve better rate-distortion performance at a lower computational cost when compared with state-of-the-art methods.

3D Point Cloud Attribute Compression via Graph Prediction

Discrete cosine transform is an ideal method to decorrelate the signal. In this paper, we proposed a novel entropy coding based on 1D-DCT to compress the 3D point clouds. Different from the former method which is based on Laplacian distribution, we encoding the coefficients by count the nonzero coefficients. We count the number, index, value of the nonzero coefficients of each block and then use arithmetic encoder to compress the coefficient. Experimental results on a number of point clouds show our method is more efficient than the former method based on DCT.

Compression of 3D point clouds using 1D discrete cosine transform

The invention discloses a three-dimensional point cloud compression method adopting graphic prediction, belonging to the video coding field. The method comprises the following steps: adopting a KD tree to carry out adaptive block division on the input three-dimensional point cloud; using a KNN algorithm to calculate the K-adjacent points of each point in the coding unit; constructing a graph of each unit block and calculating a graph translation operator; de-averaging the block of each coding unit, adopting a K-means algorithm to adaptively sample the coding unit, and predicting the unsampledpoints by solving the optimization problem; using the block mean prediction algorithm based on the KD tree to predict and encode the mean value of each coding unit block; finally, performing entropycoding on all quantized parameters and residuals by an arithmetic encoder. The method adopts graphic prediction, can effectively compress the huge three-dimensional point cloud data, and greatly improves the transmission and storage efficiency of the three-dimensional point cloud.

A three-dimensional point cloud compression method adopting graphic prediction

The invention discloses a sparse representation three-dimensional point cloud compression method adopting geometric guidance, and belongs to the field of video coding. The method comprises the steps of partitioning an input three-dimensional point cloud through an octree; obtaining an original redundant dictionary by adopting a graphic transformation method; down-sampling the original redundant dictionary by using the geometrical information of the point cloud in the block; carrying out mean value removal on each unit block, and then carrying out sparse representation on the color information subjected to mean value removal on a down-sampling dictionary; performing predictive coding on the mean value of each coding unit block by utilizing a block mean value prediction algorithm based on the octree; encoding the quantized sparse coefficient by adopting a Run-Level method; and finally, carrying out entropy coding on all coded parameters by using an arithmetic coder. According to the method, by utilizing the sparse representation, the huge three-dimensional point cloud data can be efficiently compressed, and the transmission and storage efficiency of the three-dimensional point cloud is greatly improved.

Shuai Gu

Papers

3D Point Cloud Attribute Compression Using Geometry-Guided Sparse Representation

3D Point Cloud Attribute Compression via Graph Prediction

Compression of 3D point clouds using 1D discrete cosine transform

A three-dimensional point cloud compression method adopting graphic prediction

Sparse representation three-dimensional point cloud compression method adopting geometric guidance