Showing papers by "Shihui Ying published in 2020"

Kullback-Leibler Divergence Metric Learning.

Abstract: The Kullback-Leibler divergence (KLD), which is widely used to measure the similarity between two distributions, plays an important role in many applications. In this article, we address the KLD metric-learning task, which aims at learning the best KLD-type metric from the distributions of datasets. Concretely, first, we extend the conventional KLD by introducing a linear mapping and obtain the best KLD to well express the similarity of data distributions by optimizing such a linear mapping. It improves the expressivity of data distribution, which means it makes the distributions in the same class close and those in different classes far away. Then, the KLD metric learning is modeled by a minimization problem on the manifold of all positive-definite matrices. To deal with this optimization task, we develop an intrinsic steepest descent method, which preserves the manifold structure of the metric in the iteration. Finally, we apply the proposed method along with ten popular metric-learning approaches on the tasks of 3-D object classification and document classification. The experimental results illustrate that our proposed method outperforms all other methods.

Large-Scale Place Recognition Based on Camera-LiDAR Fused Descriptor.

Abstract: In the field of autonomous driving, carriers are equipped with a variety of sensors, including cameras and LiDARs. However, the camera suffers from problems of illumination and occlusion, and the LiDAR encounters motion distortion, degenerate environment and limited ranging distance. Therefore, fusing the information from these two sensors deserves to be explored. In this paper, we propose a fusion network which robustly captures both the image and point cloud descriptors to solve the place recognition problem. Our contribution can be summarized as: (1) applying the trimmed strategy in the point cloud global feature aggregation to improve the recognition performance, (2) building a compact fusion framework which captures both the robust representation of the image and 3D point cloud, and (3) learning a proper metric to describe the similarity of our fused global feature. The experiments on KITTI and KAIST datasets show that the proposed fused descriptor is more robust and discriminative than the single sensor descriptor.

IExpressNet: Facial Expression Recognition with Incremental Classes

Abstract: Existing methods on facial expression recognition (FER) are mainly trained in the setting when all expression classes are fixed in advance. However, in real applications, expression classes are becoming increasingly fine-grained and incremental. To deal with sequential expression classes, we can fine-tune or re-train these models, but this often results in poor performance or large computing resources consumption. To address these problems, we develop an Incremental Facial Expression Recognition Network (IExpressNet), which can learn a competitive multi-class classifier at any time with a lower requirement of computing resources. Specifically, IExpressNet consists of two novel components. First, we construct an exemplar set by dynamically selecting representative samples from old expression classes. Then, the exemplar set and new expression classes samples constitute the training set. Second, we design a novel center-expression-distilled loss. As for facial expression in the wild, center-expression-distilled loss enhances the discriminative power of the deeply learned features and prevents catastrophic forgetting. Extensive experiments are conducted on two large-scale FER datasets in the wild, RAF-DB and AffectNet. The results demonstrate the superiority of the proposed method as compared to state-of-the-art incremental learning approaches.

An Improved Deep Polynomial Network Algorithm for Transcranial Sonography–Based Diagnosis of Parkinson’s Disease

Abstract: Transcranial sonography (TCS) is a valid neuroimaging tool for the diagnosis of Parkinson’s disease (PD). The TCS-based computer-aided diagnosis (CAD) has attracted increasing attention in recent years, in which feature representation and pattern classification are two critical issues. Deep polynomial network (DPN) is a newly proposed deep learning algorithm that has shown its advantage in learning effective feature representation for samples with a small size. In this work, an improved DPN algorithm with enhanced performance on both feature representation and classification is proposed. First, the empirical kernel mapping (EKM) algorithm is embedded into DPN (EKM-DPN) to improve its feature representation. Second, the network pruning strategy is utilized in the EKM-DPN (named P-EKM-DPN). It not only produces robust feature representation, but also addresses the overfitting issues for the subsequent classifiers to some extent. Lastly, the generalization ability is further enhanced by applying the Dropout approach to P-EKM-DPN (D-P-EKM-DPN). The proposed D-P-EKM-DPN algorithm has been evaluated on a TCS dataset with 153 samples. The experimental results indicate that D-P-EKM-DPN outperforms all the compared algorithms and achieves the best classification accuracy, sensitivity, and specificity of 86.95 ± 3.15%, 85.77 ± 7.87%, and 87.16 ± 6.50%, respectively. The proposed D-P-EKN-DPN algorithm has a great potential in TCS-based CAD for PD due to its excellent performance.

Parameter Transfer Deep Neural Network for Single-Modal B-Mode Ultrasound-Based Computer-Aided Diagnosis

Abstract: Elastography ultrasound (EUS) imaging has shown its effectiveness for diagnosis of tumors by providing additional information about tissue stiffness to the conventional B-mode ultrasound (BUS). However, due to the lack of EUS devices and experienced sonologists, EUS is not widely used, especially in rural areas. It is still a challenging task to improve the performance of the single-modal BUS-based computer-aided diagnosis (CAD) for tumors. In this work, we propose a novel transfer learning (TL)–based deep neural network (DNN) algorithm, named CW-PM-DNN, for the BUS-based CAD by transferring diagnosis knowledge from EUS during model training. CW-PM-DNN integrates both the feature-level and classifier-level knowledge transfer into a unified framework. In the feature-level TL, a bichannel DNN is learned by the cross-weight-based multimodal DL (MDL-CW) algorithm to transfer informative features from EUS to BUS. In the classifier-level TL, a projective model (PM)–based classifier is then embedded to the pretrained bichannel DNN to implement the parameter transfer in the classifier model at the second stage. The back-propagation procedure is then applied to optimize the whole CW-PM-DNN to further improve its performance. Experimental results on two bimodal ultrasound tumor datasets demonstrate that the proposed CW-PM-DNN achieves the best classification accuracy, sensitivity, and specificity of 89.02 ± 1.54%, 88.37 ± 4.72%, and 89.63 ± 4.06%, respectively, for the breast ultrasound dataset, and the corresponding values of 80.57 ± 3.41%, 76.67 ± 3.85%, and 83.94 ± 3.95%, respectively, for the prostate ultrasound dataset. The proposed two-stage TL-based CW-PM-DNN algorithm outperforms all the compared algorithms. It is also proved that the performance of the BUS-based CAD can be significantly improved by transferring the knowledge of EUS. It suggests that CW-PM-DNN has the potential for more applications in the field of medical image–based CAD.

Deep Doubly Supervised Transfer Network for Diagnosis of Breast Cancer with Imbalanced Ultrasound Imaging Modalities

Abstract: Elastography ultrasound (EUS) provides additional bio-mechanical information about lesion for B-mode ultrasound (BUS) in the diagnosis of breast cancers. However, joint utilization of both BUS and EUS is not popular due to the lack of EUS devices in rural hospitals, which arouses a novel modality imbalance problem in computer-aided diagnosis (CAD) for breast cancers. Current transfer learning (TL) pay little attention to this special issue of clinical modality imbalance, that is, the source domain (EUS modality) has fewer labeled samples than those in the target domain (BUS modality). Moreover, these TL methods cannot fully use the label information to explore the intrinsic relation between two modalities and then guide the promoted knowledge transfer. To this end, we propose a novel doubly supervised TL network (DDSTN) that integrates the Learning Using Privileged Information (LUPI) paradigm and the Maximum Mean Discrepancy (MMD) criterion into a unified deep TL framework. The proposed algorithm can not only make full use of the shared labels to effectively guide knowledge transfer by LUPI paradigm, but also perform additional supervised transfer between unpaired data. We further introduce the MMD criterion to enhance the knowledge transfer. The experimental results on the breast ultrasound dataset indicate that the proposed DDSTN outperforms all the compared state-of-the-art algorithms for the BUS-based CAD.

An Accelerated and Robust Partial Registration Algorithm for Point Clouds

Abstract: Partial registration for point clouds plays an important role in various fields such as 3D mapping reconstruction, remote sensing, unmanned driving, and cultural heritage protection. Unfortunately, partial registration is challenging due to difficulties such as the low overlap ratio of two point clouds and the perturbation in the orderless and sparse 3D point clouds. Thus, a variety of the 3D shape context descriptors are introduced for finding the optimal matching. However, extracting geometric features and descriptors are time consuming and easily degenerated by noise. To overcome these problems, we introduce a parallel coarse-to-fine partial registration method. Our contributions can be summarized as: Firstly, a robust coarse trimmed method is proposed to estimate the coarse overlap area and the initial transformation via fast bilateral denoising and parallel point feature histogram (PPFH) descriptor aligning. Secondly, an accelerated fine registration procedure is conducted by a parallel trimmed iterative closest point (PTrICP) method. Moreover, most parts of our coarse-to-fine workflow are accelerated under the Graphics Processing Unit (GPU) parallel execution mode for efficiency. Thirdly, we extend our method from the rigid registration to the isotropic scaling registration, which improves its applicability. Experiments have demonstrated that our method is feasible and robust in various situations, including the low overlap ratio, outlier, noise and scaling.

CoBigICP: Robust and Precise Point Set Registration using Correntropy Metrics and Bidirectional Correspondence

Abstract: In this paper, we propose a novel probabilistic variant of iterative closest point (ICP) dubbed as CoBigICP. The method leverages both local geometrical information and global noise characteristics. Locally, the 3D structure of both target and source clouds are incorporated into the objective function through bidirectional correspondence. Globally, error metric of correntropy is introduced as noise model to resist outliers. Importantly, the close resemblance between normal-distributions transform (NDT) and correntropy is revealed. To ease the minimization step, an on-manifold parameterization of the special Euclidean group is proposed. Extensive experiments validate that CoBigICP outperforms several well-known and state-of-the-art methods.

The Insular Subregions Topological Characteristics of Patients With Bipolar Depressive Disorder

Abstract: The insular cortex appears to have a crucial role in emotional processing and cognitive control in bipolar disorder (BD). However, most previous studies focused on the entire insular region of BD, neglecting the topological profile of its subregions. Our study aimed to investigate its subregion topological characteristics using the resting-state functional connectivity (rsFC) in patients with BD on depression episode. The magnetic resonance imaging (MRI) data of 28 depressed BD patients and 28 age- and gender-matched healthy controls (HCs) were acquired. We observed that compared to HCs, depressed patients with BD exhibited significantly decreased rsFC between the right ventral anterior insula (vAI) and the left middle temporal gyrus/the right angular, the right dorsal anterior insula (dAI) and the left precuneus, as well as the right posterior insula and the right lingual gyrus. Furthermore, hyperconnectivity was observed between the left dAI and the left medial frontal gyrus, as well as right dAI and left superior temporal gyrus in BD depression. However, no significant group effect was observed between aberrant FC patterns and clinical variables. These findings revealed the functional connectivity patterns of insular subregions for the depressed BD patients, suggesting the potential neural substrate of insular subregions involved in depressive episode of BD. Hence, these results may provide a neural substrate for the potential treatment target of BD on depression episode.

A Coarse-to-Fine Generalized-ICP Algorithm With Trimmed Strategy

Abstract: In this paper, we introduce a modified Generalized Iterative Closest Point (GICP) algorithm by presenting a coarse-to-fine strategy. Our contributions can be summarized as: Firstly, we use adaptively a plane-to-plane probabilistic matching model by gradually reducing the neighborhood range for given two point sets. It is an inner coarse-to-fine iteration process. Secondly, we use an outer coarse-to-fine strategy to bridge the point-to-point and plane-to-plane registration for refining the matching. Thirdly, we use the trimmed method to gradually eliminate the effects of incorrect correspondences, which improves the robustness of the methods especially for the low overlap cases. Moreover, we also extend our method to the scale registration case. Finally, we conduct extensive experiments to demonstrate that our method is more reliable and robust in various situations, including missing points, noise and different scale factors. Experimental results show that our approach outperforms several state-of-the-art registration methods.

A Local-to-Global Metric Learning Framework From the Geometric Insight

Abstract: Metric plays a key role in the description of similarity between samples. An appropriate metric for data can well represent their distribution and further promote the performance of learning tasks. In this paper, to better describe the heterogeneous distributions of data, we propose a semi-supervised local-to-global metric learning framework from the geometric insight. Our contributions can be summarized as: Firstly, to enlarge the application scope of local metric learning, we introduce the unsupervised information as the regularization term into our smoothly glued nonlinear metric model. Secondly, we propose two different nonlinear semi-supervised metric learning models with two different loss terms, and find that the smooth loss performs better than the hinge loss by comparison results. Thirdly, we have established not only two metric learning models, but also a nonlinear metric learning framework based on local metrics, which includes supervised and semi-supervised as well as linear and nonlinear metric learning. Moreover, we present an intrinsic steepest descent algorithm on the positive definite manifold for implementation of our semi-supervised nonlinear metric learning models with smooth triplet constrain loss. Finally, we compare our approaches with several state-of-the-art methods on a variety of datasets. The results validate that the robustness and accuracy of classification are both improved under our metrics.

Multi-metric Joint Discrimination Network for Few-Shot Classification

Abstract: Metric-based few-shot methods learn to recognize object categories from one or a few examples according to the distances between class features and query samples. The predicting labels of query samples are the same as those of the nearest class features. However, a single metric criterion can not model the distributions of different datasets very well. To get the more suitable metrics for a specified dataset, we propose a Multi-Metric Joint Discrimination Network (MMJDN) in this paper. Firstly, Deep Metric Module (DMM) is introduced to catch the complex relation between each pair of class features and query samples. Secondly, Adaptive Weights Module (AWM) is proposed to generate adaptive weights for different metric criteria. Our method is evaluated on three datasets: miniImageNet, Fewshot-Cifar100 (FC100) and Virus Texture Dataset (Virus15). The experimental results show that MMJDN provides positive performance for few-shot learning compared with some baselines.