Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL.

A Survey on Multi-Task Learning

https://www.linkgroup.hu/docs/13PharmTher.pdf

Structure and dynamics of molecular networks: A novel paradigm of drug discovery: A comprehensive review

Multi-view learning overview

Multilabel learning has become a relevant learning paradigm in the past years due to the increasing number of fields where it can be applied and also to the emerging number of techniques that are being developed. This article presents an up-to-date tutorial about multilabel learning that introduces the paradigm and describes the main contributions developed. Evaluation measures, fields of application, trending topics, and resources are also presented.

A Tutorial on Multilabel Learning

In image analysis, the images are often represented by multiple visual features (also known as multiview features), that aim to better interpret them for achieving remarkable performance of the learning. Since the processes of feature extraction on each view are separated, the multiple visual features of images may include overlap, noise, and redundancy. Thus, learning with all the derived views of the data could decrease the effectiveness. To address this, this paper simultaneously conducts a hierarchical feature selection and a multiview multilabel (MVML) learning for multiview image classification, via embedding a proposed a new block-row regularizer into the MVML framework. The block-row regularizer concatenating a Frobenius norm ( ${F}$ -norm) regularizer and an $\boldsymbol {\ell }_{\textbf {2,1}}$ -norm regularizer is designed to conduct a hierarchical feature selection, in which the ${F}$ -norm regularizer is used to conduct a high-level feature selection for selecting the informative views (i.e., discarding the uninformative views) and the $\boldsymbol {\ell }_{\textbf {2,1}}$ -norm regularizer is then used to conduct a low-level feature selection on the informative views. The rationale of the use of a block-row regularizer is to avoid the issue of the over-fitting (via the block-row regularizer), to remove redundant views and to preserve the natural group structures of data (via the ${F}$ -norm regularizer), and to remove noisy features (the $\boldsymbol {\ell }_{\textbf {2,1}}$ -norm regularizer), respectively. We further devise a computationally efficient algorithm to optimize the derived objective function and also theoretically prove the convergence of the proposed optimization method. Finally, the results on real image datasets show that the proposed method outperforms two baseline algorithms and three state-of-the-art algorithms in terms of classification performance.

Block-Row Sparse Multiview Multilabel Learning for Image Classification

In many real-world applications, it is becoming common to have data extracted from multiple diverse sources, known as "multi-view" data. Multi-view learning (MVL) has been widely studied in many applications, but existing MVL methods learn a single task individually. In this paper, we study a new direction of multi-view learning where there are multiple related tasks with multi-view data (i.e. multi-view multi-task learning, or MVMT Learning). In our MVMT learning methods, we learn a linear mapping for each view in each task. In a single task, we use co-regularization to obtain functions that are in-agreement with each other on the unlabeled samples and achieve low classification errors on the labeled samples simultaneously. Cross different tasks, additional regularization functions are utilized to ensure the functions that we learn in each view are similar. We also developed two extensions of the MVMT learning algorithm. One extension handles missing views and the other handles non-uniformly related tasks. Experimental studies on three real-world data sets demonstrate that our MVMT methods significantly outperform the existing state-of-the-art methods.

Inductive multi-task learning with multiple view data

In this paper, we present a novel graph database-mining method called APGM (APproximate Graph Mining) to mine useful patterns from noisy graph database. In our method, we designed a general framework for modeling noisy distribution using a probability matrix and devised an efficient algorithm to identify approximate matched frequent subgraphs. We have used APGM to both synthetic data set and real-world data sets on protein structure pattern identification and structure classification. Our experimental study demonstrates the efficiency and efficacy of the proposed method.

An efficient graph-mining method for complicated and noisy data with real-world applications

Automatic identification of structure fingerprints from a group of diverse protein structures is challenging, especially for proteins whose divergent amino acid sequences may fall into the "twilight-" or "midnight-" zones where pair-wise sequence identities to known sequences fall below 25% and sequence-based functional annotations often fail. Here we report a novel graph database mining method and demonstrate its application to protein structure pattern identification and structure classification. The biologic motivation of our study is to recognize common structure patterns in "immunoevasins", proteins mediating virus evasion of host immune defense. Our experimental study, using both viral and non-viral proteins, demonstrates the efficiency and efficacy of the proposed method. We present a theoretic framework, offer a practical software implementation for incorporating prior domain knowledge, such as substitution matrices as studied here, and devise an efficient algorithm to identify approximate matched frequent subgraphs. By doing so, we significantly expanded the analytical power of sophisticated data mining algorithms in dealing with large volume of complicated and noisy protein structure data. And without loss of generality, choice of appropriate compatibility matrices allows our method to be easily employed in domains where subgraph labels have some uncertainty.

https://kuscholarworks.ku.edu/bitstream/handle/1808/18905/Jia_structural_motif2009.pdf?sequence=1

Towards comprehensive structural motif mining for better fold annotation in the "twilight zone" of sequence dissimilarity.

Despite intense investment growth and technology development, there is an observed bottleneck in drug discovery and development over the past decade. NIH started the Molecular Libraries Initiative (MLI) in 2003 to enlarge the pool for potential drug targets, especially from the “undruggable” part of human genome, and potential drug candidates from much broader types of drug-like small molecules. All results are being made publicly available in a web portal called PubChem. In this paper we construct a network from bioassay data in PubChem, apply network biology concepts to characterize this bioassay network, integrate information from multiple biological databases (e.g. DrugBank, OMIM, and UniHI), and systematically analyze the potential of bioassay targets being new drug targets in the context of complex biological networks. We propose a model to quantitatively prioritize this druggability of bioassay targets, and literature evidence was found to confirm our prioritization of bioassay targets at a roughly 70% accuracy. Our analysis provide some measures of the value of the MLI data as a resource for both basic chemical biology research and future therapeutic discovery.

/pdf/the-bioassay-network-and-its-implications-to-future-coz1cumcm2.pdf

The BioAssay network and its implications to future therapeutic discovery

The NIH Molecular Libraries Initiative (MLI), launched in 2004 with initial goals of identifying chemical probes for characterizing gene function and druggability, has produced PubChem, a chemical genomics knowledgebase for fostering translation of basic research into new therapeutic strategies. This paper assesses progress toward these goals by evaluating MLI target novelty and propensity for undergoing biochemically or therapeutically relevant modulations and the degree of chemical diversity and biogenic bias inherent in the MLI screening set. Our analyses suggest that while MLI target selection has not yet been fully optimized for biochemical diversity, it covers biologically interesting pathway space that complements established drug targets. We find the MLI screening set to be chemically diverse and to have greater biogenic bias than comparable collections of commercially available compounds. Biogenic enhancements such as incorporation of more metabolite-like chemotypes are suggested.

Jintao Zhang

Papers

Inductive multi-task learning with multiple view data

An efficient graph-mining method for complicated and noisy data with real-world applications

Towards comprehensive structural motif mining for better fold annotation in the "twilight zone" of sequence dissimilarity.

The BioAssay network and its implications to future therapeutic discovery

Characterizing the Diversity and Biological Relevance of the MLPCN Assay Manifold and Screening Set