Showing papers by "Shihui Ying published in 2022"

A Convolutional Neural Network and Graph Convolutional Network Based Framework for Classification of Breast Histopathological Images

Abstract: The spatial correlation among different tissue components is an essential characteristic for diagnosis of breast cancers based on histopathological images. Graph convolutional network (GCN) can effectively capture this spatial feature representation, and has been successfully applied to the histopathological image based computer-aided diagnosis (CAD). However, the current GCN-based approaches need complicated image preprocessing for graph construction. In this work, we propose a novel CAD framework for classification of breast histopathological images, which integrates both convolutional neural network (CNN) and GCN (named CNN-GCN) into a unified framework, where CNN learns high-level features from histopathological images for further adaptive graph construction, and the generated graph is then fed to GCN to learn the spatial features of histopathological images for the classification task. In particular, a novel clique GCN (cGCN) is proposed to learn more effective graph representation, which can arrange both forward and backward connections between any two graph convolution layers. Moreover, a new group graph convolution is further developed to replace the classical graph convolution of each layer in cGCN, so as to reduce redundant information and implicitly select superior fused feature representation. The proposed clique group GCN (cgGCN) is then embedded in the CNN-GCN framework (named CNN-cgGCN) to promote the learned spatial representation for diagnosis of breast cancers. The experimental results on two public breast histopathological image datasets indicate the effectiveness of the proposed CNN-cgGCN with superior performance to all the compared algorithms.

Rotation-Invariant Point Cloud Representation for 3-D Model Recognition

Abstract: Three-dimensional (3-D) data have many applications in the field of computer vision and a point cloud is one of the most popular modalities. Therefore, how to establish a good representation for a point cloud is a core issue in computer vision, especially for 3-D object recognition tasks. Existing approaches mainly focus on the invariance of representation under the group of permutations. However, for point cloud data, it should also be rotation invariant. To address such invariance, in this article, we introduce a relation of equivalence under the action of rotation group, through which the representation of point cloud is located in a homogeneous space. That is, two point clouds are regarded as equivalent when they are only different from a rotation. Our network is flexibly incorporated into existing frameworks for point clouds, which guarantees the proposed approach to be rotation invariant. Besides, a sufficient analysis on how to parameterize the group SO(3) into a convolutional network, which captures a relation with all rotations in 3-D Euclidean space $\mathbb {R}^{3}$ . We select the optimal rotation as the best representation of point cloud and propose a solution for minimizing the problem on the rotation group SO(3) by using its geometric structure. To validate the rotation invariance, we combine it with two existing deep models and evaluate them on ModelNet40 dataset and its subset ModelNet10. Experimental results indicate that the proposed strategy improves the performance of those existing deep models when the data involve arbitrary rotations.

DWNN: Deep Wavelet Neural Network for Solving Partial Differential Equations

Abstract: In this paper, we propose a deep wavelet neural network (DWNN) model to approximate the natural phenomena that are described by some classical PDEs. Concretely, we introduce wavelets to deep architecture to obtain a fine feature description and extraction. That is, we constructs a wavelet expansion layer based on a family of vanishing momentum wavelets. Second, the Gaussian error function is considered as the activation function owing to its fast convergence rate and zero-centered output. Third, we design the cost function by considering the residual of governing equation, the initial/boundary conditions and an adjustable residual term of observations. The last term is added to deal with the shock wave problems and interface problems, which is conducive to rectify the model. Finally, a variety of numerical experiments are carried out to demonstrate the effectiveness of the proposed approach. The numerical results validate that our proposed method is more accurate than the state-of-the-art approach.

Self-Supervised Bi-Channel Transformer Networks for Computer-Aided Diagnosis

Abstract: Self-supervised learning (SSL) can alleviate the issue of small sample size, which has shown its effectiveness for the computer-aided diagnosis (CAD) models. However, since the conventional SSL methods share the identical backbone in both the pretext and downstream tasks, the pretext network generally cannot be well trained in the pre-training stage, if the pretext task is totally different from the downstream one. In this work, we propose a novel task-driven SSL method, namely Self-Supervised Bi-channel Transformer Networks (SSBTN), to improve the diagnostic accuracy of a CAD model by enhancing SSL flexibility. In SSBTN, we innovatively integrate two different networks for the pretext and downstream tasks, respectively, into a unified framework. Consequently, the pretext task can be flexibly designed based on the data characteristics, and the corresponding designed pretext network thus learns more effective feature representation to be transferred to the downstream network. Furthermore, a transformer-based transfer module is developed to efficiently enhance knowledge transfer by conducting feature alignment between two different networks. The proposed SSBTN is evaluated on two publicly available datasets, namely the full-field digital mammography INbreast dataset and the wireless video capsule CrohnIPI dataset. The experimental results indicate that the proposed SSBTN outperforms all the compared algorithms.

Doubly Supervised Transfer Classifier for Computer-Aided Diagnosis With Imbalanced Modalities

Abstract: Transfer learning (TL) can effectively improve diagnosis accuracy of single-modal-imaging-based computer-aided diagnosis (CAD) by transferring knowledge from other related imaging modalities, which offers a way to alleviate the small-sample-size problem. However, medical imaging data generally have the following characteristics for the TL-based CAD: 1) The source domain generally has limited data, which increases the difficulty to explore transferable information for the target domain; 2) Samples in both domains often have been labeled for training the CAD model, but the existing TL methods cannot make full use of label information to improve knowledge transfer. In this work, we propose a novel doubly supervised transfer classifier (DSTC) algorithm. In particular, DSTC integrates the support vector machine plus (SVM+) classifier and the low-rank representation (LRR) into a unified framework. The former makes full use of the shared labels to guide the knowledge transfer between the paired data, while the latter adopts the block-diagonal low-rank (BLR) to perform supervised TL between the unpaired data. Furthermore, we introduce the Schatten-p norm for BLR to obtain a tighter approximation to the rank function. The proposed DSTC algorithm is evaluated on the Alzheimer’s disease neuroimaging initiative (ADNI) dataset and the bimodal breast ultrasound image (BBUI) dataset. The experimental results verify the effectiveness of the proposed DSTC algorithm.

A Self-Supervised Learning Based Channel Attention MLP-Mixer Network for Motor Imagery Decoding

Abstract: Convolutional Neural Network (CNN) is commonly used for the Electroencephalogram (EEG) based motor-imagery (MI) decoding. However, its performance is generally limited due to the small size sample problem. An alternative way to address such issue is to segment EEG trials into small slices for data augmentation, but this approach usually inevitably loses the valuable long-range dependencies of temporal information in EEG signals. To this end, we propose a novel self-supervised learning (SSL) based channel attention MLP-Mixer network (S-CAMLP-Net) for MI decoding with EEG. Specifically, a new EEG slice prediction task is designed as the pretext task to capture the long-range information of EEG trials in the time domain. In the downstream task, a newly proposed MLP-Mixer is applied to the classification task for signals rather than for images. Moreover, in order to effectively learn the discriminative spatial representations in EEG slices, an attention mechanism is integrated into MLP-Mixer to adaptively estimate the importance of each EEG channel without any prior information. Thus, the proposed S-CAMLP-Net can effectively learn more long-range temporal information and global spatial features of EEG signals. Extensive experiments are conducted on the public MI-2 dataset and the BCI Competition IV Dataset 2A. The experimental results indicate that our proposed S-CAMLP-Net achieves superior classification performance over all the compared algorithms.

Deep Hypergraph Structure Learning

Abstract: Learning on high-order correlation has shown superiority in data representation learning, where hypergraph has been widely used in recent decades. The performance of hypergraph-based representation learning methods, such as hypergraph neural networks, highly depends on the quality of the hypergraph structure. How to generate the hypergraph structure among data is still a challenging task. Missing and noisy data may lead to"bad connections"in the hypergraph structure and destroy the hypergraph-based representation learning process. Therefore, revealing the high-order structure, i.e., the hypergraph behind the observed data, becomes an urgent but important task. To address this issue, we design a general paradigm of deep hypergraph structure learning, namely DeepHGSL, to optimize the hypergraph structure for hypergraph-based representation learning. Concretely, inspired by the information bottleneck principle for the robustness issue, we first extend it to the hypergraph case, named by the hypergraph information bottleneck (HIB) principle. Then, we apply this principle to guide the hypergraph structure learning, where the HIB is introduced to construct the loss function to minimize the noisy information in the hypergraph structure. The hypergraph structure can be optimized and this process can be regarded as enhancing the correct connections and weakening the wrong connections in the training phase. Therefore, the proposed method benefits to extract more robust representations even on a heavily noisy structure. Finally, we evaluate the model on four benchmark datasets for representation learning. The experimental results on both graph- and hypergraph-structured data demonstrate the effectiveness and robustness of our method compared with other state-of-the-art methods.

Symmetric Diffeomorphic Image Registration with Multi-Label Segmentation Masks

Abstract: Image registration aims to align two images through a spatial transformation. It plays a significant role in brain imaging analysis. In this research, we propose a symmetric diffeomorphic image registration model based on multi-label segmentation masks to solve the problems in brain MRI registration. We first introduce the similarity metric of the multi-label masks to the energy function, which improves the alignment of the brain region boundaries and the robustness to the noise. Next, we establish the model on the diffeomorphism group through the relaxation method and the inverse consistent constraint. The algorithm is designed through the local linearization and least-squares method. We then give spatially adaptive parameters to coordinate the descent of the energy function in different regions. The results show that our approach, compared with the mainstream methods, has better accuracy and noise resistance, and the transformations are more smooth and more reasonable.

Pseudo-Data based Self-Supervised Federated Learning for Classification of Histopathological Images

Abstract: Computer-aided diagnosis (CAD) can help pathologists improve diagnostic accuracy together with consistency and repeatability for cancers. However, the CAD models trained with the histopathological images only from a single center (hospital) generally suffer from the generalization problem due to the straining inconsistencies among different centers. In this work, we propose a pseudo-data based self-supervised federated learning (FL) framework, named SSL-FT-BT, to improve both the diagnostic accuracy and generalization of CAD models. Specifically, the pseudo histopathological images are generated from each center, which contains inherent and specific properties corresponding to the real images in this center, but does not include the privacy information. These pseudo images are then shared in the central server for self-supervised learning (SSL). A multi-task SSL is then designed to fully learn both the center-specific information and common inherent representation according to the data characteristics. Moreover, a novel Barlow Twins based FL (FL-BT) algorithm is proposed to improve the local training for the CAD model in each center by conducting contrastive learning, which benefits the optimization of the global model in the FL procedure. The experimental results on three public histopathological image datasets indicate the effectiveness of the proposed SSL-FL-BT on both diagnostic accuracy and generalization.

Intrinsic semi-parametric regression model on Grassmannian manifolds with applications

Abstract: Statistics on manifolds attracts more and more attentions because a variety of observations or responses have manifold structures. In this paper, we address the regression task on Grassmannian manifolds, which is arisen from the statistical shape analysis. Particularly, we develop an intrinsically semi-parametric model instead of the geodesic regression for better approximating shapes. Concretely, first we introduce a nonparametric term on Grassmannian manifolds to further depict the relationships that cannot be well described by the geodesic model on Grassmannian manifolds. Second, we utilize an alternatively iterative strategy to update parametric and nonparametric parts to form a solution. Finally, we testify the effectiveness of our regression model on synthetic data on Gr(n,m) with multiple combinations of n and m, as well as a real data of corpus callosum shapes. The experimental results validate that our proposed model outperforms the conventional geodesic regression model and shows the advantage of approaches driven doubly by model and data.

Structure Evolution on Manifold for Graph Learning

Abstract: Graph has been widely used in various applications, while how to optimize the graph is still an open question. In this paper, we propose a framework to optimize the graph structure via structure evolution on graph manifold. We first define the graph manifold and search the best graph structure on this manifold. Concretely, associated with the data features and the prediction results of a given task, we define a graph energy to measure how the graph fits the graph manifold from an initial graph structure. The graph structure then evolves by minimizing the graph energy. In this process, the graph structure can be evolved on the graph manifold corresponding to the update of the prediction results. Alternatively iterating these two processes, both the graph structure and the prediction results can be updated until converge. It achieves the suitable structure for graph learning without searching all hyperparameters. To evaluate the performance of the proposed method, we have conducted experiments on eight datasets and compared with the recent state-of-the-art methods. Experiment results demonstrate that our method outperforms the state-of-the-art methods in both transductive and inductive settings.

Intrinsic partial linear models for manifold-valued data

Abstract: This paper aims to propose an intrinsic partial linear modelling (IPLM) framework for characterizing the complex relationship between the response manifold-valued data and a set of explanatory variables such as age, education years, or gender. Such manifold value data are widespread in medical imaging, gesture recognition, computer vision, feature tracking, shape modeling, and others. Compared with most nonparametric and parametric models for manifold-valued data, our IPLM as a semi-parametric model contains both parametric and nonparametric components, leading to better adaptability, interpretation, fitting, and robustness. Furthermore, we propose an iterative estimation strategy to estimate unknown components in IPLM and use simulation experiments to display the performance of the proposed estimation methods. Finally, we apply the proposed IPLM to model the association between the brain subcortical region Corpus Callosum (CC) 2D shape and multiple covariates, such as age, gender, or disease diagnosis, show its wide application in estimating the continuous 2D shape trajectories and comparing the difference in different groups.