P2MAT-NET: Learning medial axis transform from sparse point clouds

We propose two H.264 bit rate prediction and selection applications based on features extracted for our blind video quality assessment (VQA) algorithm, Video BLIINDS. We describe the blind VQA algorithm briefly, and we show that is correlates highly with human judgments of quality. We then demonstrate two applications: an H.264 bit rate predictor and an H.264 bit rate selector.

https://hal.archives-ouvertes.fr/hal-01207875/document

Blind Prediction of Natural Video Quality and H.264 Applications

As the rapid development of big data and the artificial intelligence technology, users prefer uploading more and more local files to the cloud server to reduce the pressure of local storage, but when users upload more and more duplicate files , not only wasting the network bandwidth, but also bringing much more inconvenience to the server management, especially images and videos. To solve the problems above, we design a multi-parameter video quality assessment model based on 3D convolutional neural network in the video deduplication system, we use a method similar to analytic hierarchy process to comprehensively evaluate the impact of packet loss rate, codec, frame rate, bit rate, resolution on video quality, and build a two-stream 3D convolutional neural network from the spatial flow and timing flow to capture the details of video distortion, set the coding layer to remove redundant distortion information. Finally, the LIVE and CSIQ data sets are used for experimental verification, we compare the performance of the proposed scheme with the V-BLIINDS scheme and VIDEO scheme under different packet loss rates. We also use the part of data set to simulate the interaction process between the client and the server, then test the time cost of the scheme. On the whole, the scheme proposed in this paper has a high quality assessment efficiency.

A Multi-parameter Video Quality Assessment Model Based on 3D Convolutional Neural Network on the Cloud

Implicit representations of geometry, such as occupancy fields or signed distance fields (SDF), have recently re-gained popularity in encoding 3D solid shape in a functional form. In this work, we introduce medial fields: a field function derived from the medial axis transform (MAT) that makes available information about the underlying 3D geometry that is immediately useful for a number of downstream tasks. In particular, the medial field encodes the local thickness of a 3D shape, and enables O(1) projection of a query point onto the medial axis. To construct the medial field we require nothing but the SDF of the shape itself, thus allowing its straightforward incorporation in any application that relies on signed distance fields. Working in unison with the O(1) surface projection supported by the SDF, the medial field opens the door for an entirely new set of efficient, shape-aware operations on implicit representations. We present three such applications, including a modification to sphere tracing that renders implicit representations with better convergence properties, a fast construction method for memory-efficient rigid-body collision proxies, and an efficient approximation of ambient occlusion that remains stable with respect to viewpoint variations.

Deep Medial Fields.

GAN inversion and editing via StyleGAN maps an input image into the embedding spaces ($\mathcal{W}$, $\mathcal{W^+}$, and $\mathcal{F}$) to simultaneously maintain image fidelity and meaningful manipulation. From latent space $\mathcal{W}$ to extended latent space $\mathcal{W^+}$ to feature space $\mathcal{F}$ in StyleGAN, the editability of GAN inversion decreases while its reconstruction quality increases. Recent GAN inversion methods typically explore $\mathcal{W^+}$ and $\mathcal{F}$ rather than $\mathcal{W}$ to improve reconstruction fidelity while maintaining editability. As $\mathcal{W^+}$ and $\mathcal{F}$ are derived from $\mathcal{W}$ that is essentially the foundation latent space of StyleGAN, these GAN inversion methods focusing on $\mathcal{W^+}$ and $\mathcal{F}$ spaces could be improved by stepping back to $\mathcal{W}$. In this work, we propose to first obtain the precise latent code in foundation latent space $\mathcal{W}$. We introduce contrastive learning to align $\mathcal{W}$ and the image space for precise latent code discovery. %The obtaining process is by using contrastive learning to align $\mathcal{W}$ and the image space. Then, we leverage a cross-attention encoder to transform the obtained latent code in $\mathcal{W}$ into $\mathcal{W^+}$ and $\mathcal{F}$, accordingly. Our experiments show that our exploration of the foundation latent space $\mathcal{W}$ improves the representation ability of latent codes in $\mathcal{W^+}$ and features in $\mathcal{F}$, which yields state-of-the-art reconstruction fidelity and editability results on the standard benchmarks. Project page: https://kumapowerliu.github.io/CLCAE.

Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint

3D deep learning performance depends on object representation and local feature extraction. In this work, we present MAT-Net, a neural network which captures local and global features from the Medial Axis Transform (MAT). Different from K-Nearest-Neighbor method which extracts local features by a fixed number of neighbors, our MAT-Net exploits effective modules Group-MAT and Edge-Net to process topological structure. Experimental results illustrate that MAT-Net demonstrates competitive or better performance on 3D shape recognition than state-of-the-art methods, and prove that MAT representation has excellent capacity in 3D deep learning, even in the case of low resolution.

/pdf/mat-net-medial-axis-transform-network-for-3d-object-4eomjecikq.pdf

MAT-Net: Medial Axis Transform Network for 3D Object Recognition.

The differentiation rule and driving mechanisms of collapse disasters in various regions are unclear, and the results from existing methods of research are not sufficiently scientific. To reveal the nature of collapse disasters, this study utilized data from the 1:50,000 geological disaster investigation results database, 1:50,000 topographic data, and TM images. Topography, human activity intensity, rock mass structure, hydrological conditions, vegetation status, and meteorological conditions were used as indicators in the DEA model to analyze their validity and to explore the differentiation law and driving mechanisms of the highway slope along the YaLu river, a location of frequent geological disasters in Changbai County. In the analysis process, each index was quantitatively graded, i.e., the number of disaster points corresponding to each index was used as an input index, and the number of disaster points and the scale and stability of disaster points corresponding to the graded quantitative index were used as the output indexes. The results of the analysis of the study area indicate that there are significant differences in geological disasters due to different regional characteristics. We carried out three evaluations and performed spatial superposition analysis of the indicators corresponding to the effective values and the regional collapse points. The driving factors of collapse disasters can be divided into three categories, namely the impact of human activities, rainfall, and gravity stress. The GIS analysis and mapping found that the collapse points located to the south of the Grand Canyon of Changbai County were primarily affected by rainfall. Additionally, the areas affected by activity intensity are mostly concentrated in county towns with concentrated populations and road slopes.

/pdf/differentiation-rule-and-driving-mechanisms-of-collapse-2v9xmboq.pdf

Differentiation Rule and Driving Mechanisms of Collapse Disasters in Changbai County

Each part of a nonuniform distorted video (NUDV) has a unique distortion degree. When NUDV blocks are used as inputs, traditional machine-learning-based video quality assessment (VQA) methods frequently do not work effectively. Because these methods directly assign the label of the entire video to blocks, causing the unreliability of labels. We creatively propose video bag, a collection of video blocks, to deal with this unreliability. We develop a novel multiple instance learning (MIL) based model, VQA-MIL, which dynamically adjusts the weights by a block-wise attention module and enriches the features of video bags by a MI Pooling layer. Furthermore, we apply the mixup data-augmentation strategy to address the lack of human labels in common video datasets. We test our method on LIVE and CSIQ, and on a relatively large-scale dataset, named NUDV-KT, that we have collected. Results show that our method outperforms popular state-of-the-art no-reference VQA methods on NUDVs.

No-Reference Nonuniform Distorted Video Quality Assessment Based on Deep Multiple Instance Learning

Human motion transfer aims to transfer motions from a target dynamic person to a source static one for motion synthesis. An accurate matching between the source person and the target motion in both large and subtle motion changes is vital for improving the transferred motion quality. In this paper, we propose Human MotionFormer, a hierarchical ViT framework that leverages global and local perceptions to capture large and subtle motion matching, respectively. It consists of two ViT encoders to extract input features (i.e., a target motion image and a source human image) and a ViT decoder with several cascaded blocks for feature matching and motion transfer. In each block, we set the target motion feature as Query and the source person as Key and Value, calculating the cross-attention maps to conduct a global feature matching. Further, we introduce a convolutional layer to improve the local perception after the global cross-attention computations. This matching process is implemented in both warping and generation branches to guide the motion transfer. During training, we propose a mutual learning loss to enable the co-supervision between warping and generation branches for better motion representations. Experiments show that our Human MotionFormer sets the new state-of-the-art performance both qualitatively and quantitatively. Project page: \url{https://github.com/KumapowerLIU/Human-MotionFormer}

Human MotionFormer: Transferring Human Motions with Vision Transformers

In recent years, geological disasters have frequently occurred on basarlt highway slopes. Studying the stability of highway slopes in this type of area is of great significance for traffic safety. However, due to the high cost and low efficiency of traditional monitoring and experimental methods for slope engineering, these methods are not conducive to the quick and comprehensive identification of regional slope stability. Due to the high iron content of basalt, iron staining anomalies in the ore prospecting field are reinterpreted from an engineering perspective in this study. Taking the S3K section of a highway in Changbai County, China, as an example, Landsat8 remote sensing (RS) images from 2014, 2016, 2018, 2020, and 2021 are selected, and principal component analysis is used to extract iron staining anomalies in the region. Combined with field investigation and evidence collection, the corresponding rock mass fragmentation is distinguished via iron staining anomalies. Then, according to previous research results, eight indexes including annual rainfall, slope, topographic relief, surface roughness, vegetation index, leaf area index (LAI), root depth of vegetation, and human activity intensity are selected for investigation. The artificial neural network–cellular automata (ANN-CA) model is established, and the rock fragmentation classification data obtained based on iron staining anomalies are used to simulate the area. Next, the calculation formula of slope stability is determined based on the simulation results, and the stability of a high and steep slope in the area is calculated and analyzed. Finally, a comparison with an actual field investigation shows that the effect of the proposed method is good. The research findings reveal that it is feasible to judge the stability of a high and steep slope in a basalt area via the use of iron staining anomalies as an indicator. The findings are tantamount to expanding the application scope of RS in practical engineering.

/pdf/determination-of-the-stability-of-a-high-and-steep-highway-2ul2b2su.pdf

Lihui Qian

Papers

MAT-Net: Medial Axis Transform Network for 3D Object Recognition.

Differentiation Rule and Driving Mechanisms of Collapse Disasters in Changbai County

No-Reference Nonuniform Distorted Video Quality Assessment Based on Deep Multiple Instance Learning

Human MotionFormer: Transferring Human Motions with Vision Transformers

Determination of the Stability of a High and Steep Highway Slope in a Basalt Area Based on Iron Staining Anomalies