Showing papers by "Michael K. Ng published in 2019"

Molecular subtyping of cancer: Current status and moving toward clinical applications

Abstract: Cancer is a collection of genetic diseases, with large phenotypic differences and genetic heterogeneity between different types of cancers and even within the same cancer type. Recent advances in genome-wide profiling provide an opportunity to investigate global molecular changes during the development and progression of cancer. Meanwhile, numerous statistical and machine learning algorithms have been designed for the processing and interpretation of high-throughput molecular data. Molecular subtyping studies have allowed the allocation of cancer into homogeneous groups that are considered to harbor similar molecular and clinical characteristics. Furthermore, this has helped researchers to identify both actionable targets for drug design as well as biomarkers for response prediction. In this review, we introduce five frequently applied techniques for generating molecular data, which are microarray, RNA sequencing, quantitative polymerase chain reaction, NanoString and tissue microarray. Commonly used molecular data for cancer subtyping and clinical applications are discussed. Next, we summarize a workflow for molecular subtyping of cancer, including data preprocessing, cluster analysis, supervised classification and subtype characterizations. Finally, we identify and describe four major challenges in the molecular subtyping of cancer that may preclude clinical implementation. We suggest that standardized methods should be established to help identify intrinsic subgroup signatures and build robust classifiers that pave the way toward stratified treatment of cancer patients.

Tensor-Based Low-Dimensional Representation Learning for Multi-View Clustering

Abstract: With the development of data collection techniques, multi-view clustering becomes an emerging research direction to improve the clustering performance. This paper has shown that leveraging multi-view information is able to provide a rich and comprehensive description. One of the core problems is how to sufficiently represent multi-view data in the analysis. In this paper, we introduce a tensor-based representation learning method for multi-view clustering (tRLMvC) that can unify heterogeneous and high-dimensional multi-view feature spaces to a low-dimensional shared latent feature space and improve multi-view clustering performance. To sufficiently capture plenty multi-view information, the tRLMvC represents multi-view data as a third-order tensor, expresses each tensorial data point as a sparse $t$ -linear combination of all data points with $t$ -product, and constructs a self-expressive tensor through reconstruction coefficients. The low-dimensional multi-view data representation in the shared latent feature space can be obtained via Tucker decomposition on the self-expressive tensor. These two parts are iteratively performed so that the interaction between self-expressive tensor learning and its factorization can be enhanced and the new representation can be effectively generated for clustering purpose. We conduct extensive experiments on eight multi-view data sets and compare the proposed model with the state-of-the-art methods. Experimental results have shown that tRLMvC outperforms the baselines in terms of various evaluation metrics.

Color image restoration by saturation-value total variation

Abstract: Color image restoration is one of the important tasks in color image processing. Total variation regularizaton was proposed and employed for the recovery of edges in a grayscale image. In the liter...

Lanczos method for large-scale quaternion singular value decomposition

Abstract: In many color image processing and recognition applications, one of the most important targets is to compute the optimal low-rank approximations to color images, which can be reconstructed with a small number of dominant singular value decomposition (SVD) triplets of quaternion matrices. All existing methods are designed to compute all SVD triplets of quaternion matrices at first and then to select the necessary dominant ones for reconstruction. This way costs quite a lot of operational flops and CPU times to compute many superfluous SVD triplets. In this paper, we propose a Lanczos-based method of computing partial (several dominant) SVD triplets of the large-scale quaternion matrices. The partial bidiagonalization of large-scale quaternion matrices is derived by using the Lanczos iteration, and the reorthogonalization and thick-restart techniques are also utilized in the implementation. An algorithm is presented to compute the partial quaternion singular value decomposition. Numerical examples, including principal component analysis, color face recognition, video compression and color image completion, illustrate that the performance of the developed Lanczos-based method for low-rank quaternion approximation is better than that of the state-of-the-art methods.

Robust Low-Tubal-Rank Tensor Completion via Convex Optimization.

Abstract: This paper considers the problem of recovering multidimensional array, in particular third-order tensor, from a random subset of its arbitrarily corrupted entries. Our study is based on a recently proposed algebraic framework in which the tensor-SVD is introduced to capture the low-tubal-rank structure in tensor. We analyze the performance of a convex program, which minimizes a weighted combination of the tensor nuclear norm, a convex surrogate for the tensor tubal rank, and the tensor l1 norm. We prove that under certain incoherence conditions, this program can recover the tensor exactly with overwhelming probability, provided that its tubal rank is not too large and that the corruptions are reasonably sparse. The number of required observations is order optimal (up to a logarithm factor) when comparing with the degrees of freedom of the low-tubal-rank tensor. Numerical experiments verify our theoretical results and real-world applications demonstrate the effectiveness of our algorithm.

A corrected tensor nuclear norm minimization method for noisy low-rank tensor completion

Abstract: In this paper, we study the problem of low-rank tensor recovery from limited sampling with noisy observations for third-order tensors. A tensor nuclear norm method based on a convex relaxation of t...

Oversampling for imbalanced data via optimal transport

Abstract: The issue of data imbalance occurs in many real-world applications especially in medical diagnosis, where normal cases are usually much more than the abnormal cases. To alleviate this issue, one of the most important approaches is the oversampling method, which seeks to synthesize minority class samples to balance the numbers of different classes. However, existing methods barely consider global geometric information involved in the distribution of minority class samples, and thus may incur distribution mismatching between real and synthetic samples. In this paper, relying on optimal transport (Villani 2008), we propose an oversampling method by exploiting global geometric information of data to make synthetic samples follow a similar distribution to that of minority class samples. Moreover, we introduce a novel regularization based on synthetic samples and shift the distribution of minority class samples according to loss information. Experiments on toy and real-world data sets demonstrate the efficacy of our proposed method in terms of multiple metrics.

Sentiment Lexicon Construction With Hierarchical Supervision Topic Model

Abstract: In this paper, we propose a novel hierarchical supervision topic model to construct a topic-adaptive sentiment lexicon (TaSL) for higher-level classification tasks. It is widely recognized that sentiment lexicon as a useful prior knowledge is crucial in sentiment analysis or opinion mining. However, many existing sentiment lexicons are constructed ignoring the variability of the sentiment polarities of words in different topics or domains. For example, the word “amazing” can refer to causing great surprise or wonder but can also refer to very impressive and excellent. In TaSL, we solve this issue by jointly considering the topics and sentiments of words. Documents are represented by multiple pairs of topics and sentiments, where each pair is characterized by a multinomial distribution over words. Meanwhile, this generating process is supervised under hierarchical supervision information of documents and words. The main advantage of TaSL is that the sentiment polarity of each word in different topics can be sufficiently captured. This model is beneficial to construct a domain-specific sentiment lexicon and then effectively improve the performance of sentiment classification. Extensive experimental results on four publicly available datasets, MR , OMD , semEval13A , and semEval16B were presented to demonstrate the usefulness of the proposed approach. The results have shown that TaSL performs better than the existing manual sentiment lexicon (MPQA), the topic model based domain-specific lexicon (ssLDA), the expanded lexicons(Weka-ED, Weka-STS, NRC, Liu's), and deep neural network based lexicons (nnLexicon, HIT, HSSWE).

Structural Similarity-Based Nonlocal Variational Models for Image Restoration

Abstract: In this paper, we propose and develop a novel nonlocal variational technique based on structural similarity (SS) information for image restoration problems. In the literature, patches extracted from images are compared according to their pixel values, and then nonlocal filtering can be employed for image restoration. The disadvantage of this approach is that intensity-based patch distance may not be effective in image restoration, especially for images containing texture or structural information. The main aim of this paper is to propose using SS between image patches to develop nonlocal regularization models. In particular, two types of nonlocal regularizing functions are studied: an SS-based nonlocal quadratic function (SS-NLH1) and an SS-based nonlocal total variation function (SS-NLTV) for regularization of image restoration problems. Moreover, we employ iterative algorithms to solve these SS-NLH1 and SS-NLTV variational models numerically and discuss the convergence of these algorithms. The experimental results are presented to demonstrate the effectiveness of the proposed models.

Online Heterogeneous Transfer Learning by Knowledge Transition

Abstract: In this article, we study the problem of online heterogeneous transfer learning, where the objective is to make predictions for a target data sequence arriving in an online fashion, and some offline labeled instances from a heterogeneous source domain are provided as auxiliary data. The feature spaces of the source and target domains are completely different, thus the source data cannot be used directly to assist the learning task in the target domain. To address this issue, we take advantage of unlabeled co-occurrence instances as intermediate supplementary data to connect the source and target domains, and perform knowledge transition from the source domain into the target domain. We propose a novel online heterogeneous transfer learning algorithm called Online Heterogeneous Knowledge Transition (OHKT) for this purpose. In OHKT, we first seek to generate pseudo labels for the co-occurrence data based on the labeled source data, and then develop an online learning algorithm to classify the target sequence by leveraging the co-occurrence data with pseudo labels. Experimental results on real-world data sets demonstrate the effectiveness and efficiency of the proposed algorithm.

Crank–nicolson alternative direction implicit method for space-fractional diffusion equations with nonseparable coefficients

Abstract: In this paper, we study the Crank--Nicolson alternative direction implicit (ADI) method for two-dimensional Riesz space-fractional diffusion equations with nonseparable coefficients. Existing ADI m...

A Fast Algorithm for Cosine Transform Based Tensor Singular Value Decomposition.

Abstract: Recently, there has been a lot of research into tensor singular value decomposition (t-SVD) by using discrete Fourier transform (DFT) matrix. The main aims of this paper are to propose and study tensor singular value decomposition based on the discrete cosine transform (DCT) matrix. The advantages of using DCT are that (i) the complex arithmetic is not involved in the cosine transform based tensor singular value decomposition, so the computational cost required can be saved; (ii) the intrinsic reflexive boundary condition along the tubes in the third dimension of tensors is employed, so its performance would be better than that by using the periodic boundary condition in DFT. We demonstrate that the tensor product between two tensors by using DCT can be equivalent to the multiplication between a block Toeplitz-plus-Hankel matrix and a block vector. Numerical examples of low-rank tensor completion are further given to illustrate that the efficiency by using DCT is two times faster than that by using DFT and also the errors of video and multispectral image completion by using DCT are smaller than those by using DFT.

A Semisupervised Classification Approach for Multidomain Networks With Domain Selection

Abstract: Multidomain network classification has attracted significant attention in data integration and machine learning, which can enhance network classification or prediction performance by integrating information from different sources. Despite the previous success, existing multidomain network learning methods usually assume that different views are available for the same set of instances, and thus, they seek a consistent classification result for all domains. However, in many real-world problems, each domain has its specific instance set, and one instance in one domain may correspond to multiple instances in another domain. Moreover, due to the rapid growth of data sources, different domains may not be relevant to each other, which asks for selecting domains relevant to the target/focused domain. A key challenge under this setting is how to achieve accurate prediction by integrating different data representations without losing data information. In this paper, we propose a semisupervised classification approach for a multidomain network based on label propagation, i.e., multidomain classification with domain selection (MCS), which can deal with the cross-domain information and different instance sets in domains. In particular, with sparse weight properties, the proposed MCS can automatically identify those domains relevant to our target domain by assigning them higher weights than the other irrelevant domains. This not only significantly improves a classification accuracy but also helps to obtain optimal network partition for the target domain. From the theoretical viewpoint, we equivalently decompose MCS into two simpler subproblems with analytical solutions, which can be efficiently solved by their computational procedures. Extensive experimental results on both synthetic and real-world data sets empirically demonstrate the advantages of the proposed approach in terms of both prediction performance and domain selection ability.

A fast solver for multidimensional time–space fractional diffusion equation with variable coefficients

Abstract: In this paper, we study a discretization scheme and the corresponding fast solver for multi-dimensional time–space fractional diffusion equation with variable coefficients, in which L 1 formula and shifted Grunwald formula are employed to discretize the temporal and spatial derivatives, respectively. A divide-and-conquer strategy is applied to the large linear system assembling discrete equations of all time levels, which in turn requires to solve a series of multidimensional linear systems related to the spatial discretization. Preconditioned generalized minimal residual method is employed to solve the spatial linear systems resulting from the spatial discretization. The discretization is proven to be unconditionally stable and convergent in the sense of infinity norm for general nonnegative coefficients. Numerical results are reported to show the efficiency of the proposed method.

Robust Tensor Completion Using Transformed Tensor SVD.

Abstract: In this paper, we study robust tensor completion by using transformed tensor singular value decomposition (SVD), which employs unitary transform matrices instead of discrete Fourier transform matrix that is used in the traditional tensor SVD. The main motivation is that a lower tubal rank tensor can be obtained by using other unitary transform matrices than that by using discrete Fourier transform matrix. This would be more effective for robust tensor completion. Experimental results for hyperspectral, video and face datasets have shown that the recovery performance for the robust tensor completion problem by using transformed tensor SVD is better in PSNR than that by using Fourier transform and other robust tensor completion methods.

A variational gamma correction model for image contrast enhancement

Abstract: Image contrast enhancement plays an important role in computer vision and pattern recognition by improving image quality. The main aim of this paper is to propose and develop a variational model for contrast enhancement of color images based on local gamma correction. The proposed variational model contains an energy functional to determine a local gamma function such that the gamma values can be set according to the local information of the input image. A spatial regularization of the gamma function is incorporated into the functional so that the contrast in an image can be modified by using the information of each pixel and its neighboring pixels. Another regularization term is also employed to preserve the ordering of pixel values. Theoretically, the existence and uniqueness of the minimizer of the proposed model are established. A fast algorithm can be developed to solve the resulting minimization model. Experimental results on benchmark images are presented to show that the performance of the proposed model are better than that of the other testing methods.

Low-resolution image categorization via heterogeneous domain adaptation

Abstract: Most of existing image categorizations assume that the given datasets have a good resolution and quality However, the assumption is often violated in real applications In this paper, we study the low-resolution (LR) image categorization By utilizing labeled high-resolution (HR) images as auxiliary information, we formulate the problem as a heterogeneous domain adaptation problem and propose a Discriminative Joint Distribution Adaptation (DJDA) model to solve it The DJDA model projects both LR and HR images into an intermediate subspace with a well-designed objective function, where the distance between classes is expected to be enlarged and the distribution divergence to be reduced As a result, the discriminative knowledge for HR images can be transferred effectively to LR images Experimental results demonstrate the proposed DJDA method produces significantly superior categorization accuracies against state-of-the-art competitors

Image colorization by fusion of color transfers based on DFT and variance features

Abstract: Color transfer methods usually suffer from spatial color coherency problem. In order to address this problem, this paper develops a fused color transfer method for image colorization. Our idea is to design a local student’s t-test to screen the incoherent colors in the preliminary colorization results obtained by a simple color transfer method with DFT and variance features. Furthermore, we propose a variational fusion model to inpaint these incoherent colors and fuse the other useful colors together. We also present an efficient algorithm for solving the fusion model numerically, and show the convergence of the algorithm. Finally, experimental results are reported to demonstrate the effectiveness of the proposed method, and its performance is competitive with those of the other testing methods.

Circulant preconditioners for a kind of spatial fractional diffusion equations

Abstract: In this paper, circulant preconditioners are studied for discretized matrices arising from finite difference schemes for a kind of spatial fractional diffusion equations. The fractional differential operator is comprised of left-sided and right-sided derivatives with order in $(\frac {1}{2},1)$ . The resulting discretized matrices preserve Toeplitz-like structure and hence their matrix-vector multiplications can be computed efficiently by the fast Fourier transform. Theoretically, the spectra of the circulant preconditioned matrices are shown to be clustered around 1 under some conditions. Numerical experiments are presented to demonstrate that the preconditioning technique is very efficient.

Nonnegative Low Rank Matrix Approximation for Nonnegative Matrices

Abstract: This paper describes a new algorithm for computing Nonnegative Low Rank Matrix (NLRM) approximation for nonnegative matrices. Our approach is completely different from classical nonnegative matrix factorization (NMF) which has been studied for more than twenty five years. For a given nonnegative matrix, the usual NMF approach is to determine two nonnegative low rank matrices such that the distance between their product and the given nonnegative matrix is as small as possible. However, the proposed NLRM approach is to determine a nonnegative low rank matrix such that the distance between such matrix and the given nonnegative matrix is as small as possible. There are two advantages. (i) The minimized distance by the proposed NLRM method can be smaller than that by the NMF method, and it implies that the proposed NLRM method can obtain a better low rank matrix approximation. (ii) Our low rank matrix admits a matrix singular value decomposition automatically which provides a significant index based on singular values that can be used to identify important singular basis vectors, while this information cannot be obtained in the classical NMF. The proposed NLRM approximation algorithm was derived using the alternating projection on the low rank matrix manifold and the non-negativity property. Experimental results are presented to demonstrate the above mentioned advantages of the proposed NLRM method compared the NMF method.

Calibration of ϵ−insensitive loss in support vector machines regression

Abstract: Support vector machines regression (SVMR) is an important tool in many machine learning applications. In this paper, we focus on the theoretical understanding of SVMR based on the ϵ − insensitive loss. For fixed ϵ ≥ 0 and general data generating distributions, we show that the minimizer of the expected risk for ϵ − insensitive loss used in SVMR is a set-valued function called conditional ϵ − median. We then establish a calibration inequality of ϵ − insensitive loss under a noise condition on the conditional distributions. This inequality also ensures us to present a nontrivial variance-expectation bound for ϵ − insensitive loss, and which is known to be important in statistical analysis of the regularized learning algorithms. With the help of the calibration inequality and variance-expectation bound, we finally derive an explicit learning rate for SVMR in some L r − space.

Framelet Representation of Tensor Nuclear Norm for Third-Order Tensor Completion

Abstract: The main aim of this paper is to develop a framelet representation of the tensor nuclear norm for third-order tensor completion. In the literature, the tensor nuclear norm can be computed by using tensor singular value decomposition based on the discrete Fourier transform matrix, and tensor completion can be performed by the minimization of the tensor nuclear norm which is the relaxation of the sum of matrix ranks from all Fourier transformed matrix frontal slices. These Fourier transformed matrix frontal slices are obtained by applying the discrete Fourier transform on the tubes of the original tensor. In this paper, we propose to employ the framelet representation of each tube so that a framelet transformed tensor can be constructed. Because of framelet basis redundancy, the representation of each tube is sparsely represented. When the matrix slices of the original tensor are highly correlated, we expect the corresponding sum of matrix ranks from all framelet transformed matrix frontal slices would be small, and the resulting tensor completion can be performed much better. The proposed minimization model is convex and global minimizers can be obtained. Numerical results on several types of multi-dimensional data (videos, multispectral images, and magnetic resonance imaging data) have tested and shown that the proposed method outperformed the other testing methods.

Orthogonal Nonnegative Tucker Decomposition

Abstract: In this paper, we study the nonnegative tensor data and propose an orthogonal nonnegative Tucker decomposition (ONTD). We discuss some properties of ONTD and develop a convex relaxation algorithm of the augmented Lagrangian function to solve the optimization problem. The convergence of the algorithm is given. We employ ONTD on the image data sets from the real world applications including face recognition, image representation, hyperspectral unmixing. Numerical results are shown to illustrate the effectiveness of the proposed algorithm.

Image colorization by using graph bi-Laplacian

Abstract: Image colorization aims to recover the whole color image based on a known grayscale image (luminance or brightness) and some known color pixel values. In this paper, we generalize the graph Laplacian to its second-order variant called graph bi-Laplacian, and then propose an image colorization method by using graph bi-Laplacian. The eigenvalue analysis of graph bi-Laplacian matrix and its corresponding normalized bi-Laplacian matrix is given to show their properties. We apply graph bi-Laplacian approach to image colorization by formulating it as an optimization problem and solving the resulting linear system efficiently. Numerical results show that the proposed method can perform quite well for image colorization problem, and its performance in terms of efficiency and colorization quality for test images can be better than that by the state-of-the-art colorization methods when the randomly given color pixels ratio attains some level.

A C-eigenvalue problem for tensors with applications to higher-order multivariate Markov chains

Abstract: In this paper, we study a new tensor eigenvalue problem, which involves E - and S -eigenvalues as its special cases. Some theoretical results such as existence of an eigenvalue and the number of eigenvalues are given. For an application of the proposed eigenvalue problem, we establish a tensor model for a higher-order multivariate Markov chain. The core issue of this problem is to study a stationary probability distribution of a higher-order multivariate Markov chain. A sufficient condition of the unique stationary positive distribution is given. An algorithm for computing stationary probability distribution is also developed. Numerical examples of applications in stock market modeling, sales demand prediction and biological sequence analysis are given to illustrate the proposed tensor model and the computed stationary probability distribution can provide a better prediction in these Markov chain applications.

A Fast Algorithm for Solving Linear Inverse Problems with Uniform Noise Removal

Abstract: In this paper, we develop a fast algorithm for solving an unconstrained optimization model for uniform noise removal which is an important task in inverse problems. The optimization model consists of an $$\ell _\infty $$ data fitting term and a total variation regularization term. By utilizing the alternating direction method of multipliers (ADMM) for such optimization model, we demonstrate that one of the ADMM subproblems can be formulated by involving a projection onto $$\ell _1$$ ball which can be solved efficiently by iterations. The convergence of the ADMM method can be established under some mild conditions. In practice, the balance between the $$\ell _\infty $$ data fitting term and the total variation regularization term is controlled by a regularization parameter. We present numerical experiments by using the L-curve method of the logarithms of data fitting term and total variation regularization term to select regularization parameters for uniform noise removal. Numerical results for image denoising and deblurring, inverse source, inverse heat conduction problems and second derivative problems have shown the effectiveness of the proposed model.