Inferring network interactions using recurrent neural networks and swarm intelligence.
01 Jan 2006-Vol. 1, pp 4241-4244
TL;DR: The results indicate that the proposed hybrid SI-RNN algorithm has a promising potential to infer complex interactions such as gene regulatory networks from time-series gene expression data.
Abstract: We present a novel algorithm combining artificial neural networks and swarm intelligence (SI) methods to infer network interactions The algorithm uses ant colony optimization (ACO) to identify the optimal architecture of a recurrent neural network (RNN), while the weights of the RNN are optimized using particle swarm optimization (PSO) Our goal is to construct an RNN that mimics the true structure of an unknown network and the time-series data that the network generated We applied the proposed hybrid SI-RNN algorithm to infer a simulated genetic network The results indicate that the algorithm has a promising potential to infer complex interactions such as gene regulatory networks from time-series gene expression data I INTRODUCTION inference algorithm based on GAs for the optimization of the influence matrix of gene regulatory network In (13), GAs and ANNs are combined to determine gene interactions in temporal gene expression data In this paper, we propose to apply a hybrid of ANNs and swarm intelligence (SI) methods (12) to infer network interactions from time-series data The architecture and the synaptic weights of a recurrent neural network (RNN) are optimized using ant colony optimization (ACO) and particle swarm optimization (PSO) methods, respectively Unlike previous computational methods, which targeted at one-step- ahead prediction of time-series data (13), our method enables a multi-step-ahead prediction This is achieved through our RNN, which is self-evolutionary The RNN starts with a given initial condition, evolves, and eventually reaches final states The proposed hybrid SI-RNN algorithm selects the architecture of the RNN and weights not only to mimic the response of the unknown network at each time point but also to identify the structure of the network that generated the time-series data This is a challenging task given that there may be many possible structures with responses that closely match the generated data The algorithm evaluates various structures through the cross- validation method to avoid the selection of a wrong structure and to make sure that the correct structure is identified despite the presence of noise and complexity of the unknown network We successfully applied the algorithm to infer simulated network interactions
Citations
More filters
Dissertation•
21 Apr 2015
TL;DR: The PSO is shown to be a very effective algorithm for the training of self-adaptive FFNNs, capable of learning from unscaled data and having a dominant effect on the resultant FFNN accuracy.
Abstract: The phenomenon of overfitting, where a feed-forward neural network (FFNN) over trains on training data at the cost of generalisation accuracy is known to be specific to the training algorithm used. This study investigates overfitting within the context of particle swarm optimised (PSO) FFNNs. Two of the most widely used PSO algorithms are compared in terms of FFNN accuracy and a description of the overfitting behaviour is established. Each of the PSO components are in turn investigated to determine their effect on FFNN overfitting. A study of the maximum velocity (Vmax) parameter is performed and it is found that smaller Vmax values are optimal for FFNN training. The analysis is extended to the inertia and acceleration coefficient parameters, where it is shown that specific interactions among the parameters have a dominant effect on the resultant FFNN accuracy and may be used to reduce overfitting. Further, the significant effect of the swarm size on network accuracy is also shown, with a critical range being identified for the swarm size for effective training. The study is concluded with an investigation into the effect of the different activation functions. Given strong empirical evidence, an hypothesis is made that stating the gradient of the activation function significantly affects the convergence of the PSO. Lastly, the PSO is shown to be a very effective algorithm for the training of self-adaptive FFNNs, capable of learning from unscaled data.
4 citations
Cites methods from "Inferring network interactions usin..."
...PSO has also been used successfully in training a variety of NNs, including radial bases function networks [49, 68, 84], recurrent networks [10, 55, 56, 85, 90] and product unit NNs [52]....
[...]
Journal Article•
TL;DR: Reconstruction method of Gene Regulatory Network based on Modified Particle Swarm Optimization uses PSO to identify the optimal architecture and parameter of the weight matrices model, so that a recurrent neural network consistent with experimental data is inferred, and uses weightMatrices model to simulate GRN.
Abstract: This paper presents reconstruction method of Gene Regulatory Network(GRN) based on Modified Particle Swarm Optimization(MPSO).It uses PSO to identify the optimal architecture and parameter of the weight matrices model,so that a recurrent neural network consistent with experimental data is inferred,and uses weight matrices model to simulate GRN.Experimental results show that the method is effective to infer complex interaction such as GRN.
Additional excerpts
...为了测试 MPSO 对基因调控网络重构的效果，将本算法 和具有代表性的遗传算法[3]进行比较，分别推理图 2 的基因 调控网络[ 4 ]。...
[...]
Dissertation•
20 Jul 2010
TL;DR: This dissertation proposes and develops innovative systems approaches to integrate multi-source biological data in a modular manner for network inference and biomarker discovery in complex diseases such as breast cancer and presents a hybrid computational intelligence method to infer gene regulatory modules.
Abstract: Systems biology comprises the global, integrated analysis of large-scale data encoding different levels of biological information with the aim to obtain global insight into the cellular networks. Several studies have unveiled the modular and hierarchical organization inherent in these networks. In this dissertation, we propose and develop innovative systems approaches to integrate multi-source biological data in a modular manner for network inference and biomarker discovery in complex diseases such as breast cancer. The first part of the dissertation is focused on gene module identification in gene expression data. As the most popular way to identify gene modules, many cluster algorithms have been applied to the gene expression data analysis. For the purpose of evaluating clustering algorithms from a biological point of view, we propose a figure of merit based on Kullback-Leibler divergence between cluster membership and known gene ontology attributes. Several benchmark expression-based gene clustering algorithms are compared using the proposed method with different parameter settings. Applications to diverse public time course gene expression data demonstrated that fuzzy c-means clustering is superior to other clustering methods with regard to the enrichment of clusters for biological functions. These results contribute to the evaluation of clustering outcomes and the estimations of optimal clustering partitions. The second part of the dissertation presents a hybrid computational intelligence method to infer gene regulatory modules. We explore the combined advantages of the nonlinear and dynamic properties of neural networks, and the global search capabilities of the hybrid genetic algorithm and particle swarm optimization method to infer network interactions at modular level. The proposed computational framework is tested in two biological processes: yeast cell cycle, and human Hela cancer cell cycle. The identified gene regulatory modules were evaluated using several validation strategies: 1) gene set enrichment analysis to evaluate the gene modules derived from clustering results; (2) binding site enrichment analysis to determine enrichment of
Additional excerpts
...The PSO generation for NN is set to 1000 [129]....
[...]
07 Mar 2012
TL;DR: R reverse engineering of a global GRN remains challenging because of several limitations including the following:
Abstract: During last two decades, enormous amount of biological data generated by highthroughput analytical methods in biology produces vast patterns of gene activity, highlighting the need for systematic tools to identify the architecture and dynamics of the underlying GRN (He et al. 2009). Here, the system identification problem falls naturally into the category of reverse engineering; a complex genetic network underlies a massive set of expression data, and the task is to infer the connectivity of the genetic circuit (Tegner et al. 2003). However, reverse engineering of a global GRN remains challenging because of several limitations including the following:
References
More filters
TL;DR: A new framework for discovering interactions between genes based on multiple expression measurements is proposed and a method for recovering gene interactions from microarray data is described using tools for learning Bayesian networks.
Abstract: DNA hybridization arrays simultaneously measure the expression level for thousands of genes. These measurements provide a "snapshot" of transcription levels within the cell. A major challenge in computational biology is to uncover, from such measurements, gene/protein interactions and key biological features of cellular systems. In this paper, we propose a new framework for discovering interactions between genes based on multiple expression measurements. This framework builds on the use of Bayesian networks for representing statistical dependencies. A Bayesian network is a graph-based model of joint multivariate probability distributions that captures properties of conditional independence between variables. Such models are attractive for their ability to describe complex stochastic processes and because they provide a clear methodology for learning from (noisy) observations. We start by showing how Bayesian networks can describe interactions between genes. We then describe a method for recovering gene interactions from microarray data using tools for learning Bayesian networks. Finally, we demonstrate this method on the S. cerevisiae cell-cycle measurements of Spellman et al. (1998).
3,507 citations
Book•
01 Jan 2018
TL;DR: Computational Intelligence: An Introduction, Second Edition offers an in-depth exploration into the adaptive mechanisms that enable intelligent behaviour in complex and changing environments, encompassing swarm intelligence, fuzzy systems, artificial neutral networks, artificial immune systems and evolutionary computation.
Abstract: Computational Intelligence: An Introduction, Second Edition offers an in-depth exploration into the adaptive mechanisms that enable intelligent behaviour in complex and changing environments. The main focus of this text is centred on the computational modelling of biological and natural intelligent systems, encompassing swarm intelligence, fuzzy systems, artificial neutral networks, artificial immune systems and evolutionary computation. Engelbrecht provides readers with a wide knowledge of Computational Intelligence (CI) paradigms and algorithms; inviting readers to implement and problem solve real-world, complex problems within the CI development framework. This implementation framework will enable readers to tackle new problems without any difficulty through a single Java class as part of the CI library. Key features of this second edition include: A tutorial, hands-on based presentation of the material. State-of-the-art coverage of the most recent developments in computational intelligence with more elaborate discussions on intelligence and artificial intelligence (AI). New discussion of Darwinian evolution versus Lamarckian evolution, also including swarm robotics, hybrid systems and artificial immune systems. A section on how to perform empirical studies; topics including statistical analysis of stochastic algorithms, and an open source library of CI algorithms. Tables, illustrations, graphs, examples, assignments, Java code implementing the algorithms, and a complete CI implementation and experimental framework. Computational Intelligence: An Introduction, Second Edition is essential reading for third and fourth year undergraduate and postgraduate students studying CI. The first edition has been prescribed by a number of overseas universities and is thus a valuable teaching tool. In addition, it will also be a useful resource for researchers in Computational Intelligence and Artificial Intelligence, as well as engineers, statisticians, operational researchers, and bioinformaticians with an interest in applying AI or CI to solve problems in their domains. Check out http://www.ci.cs.up.ac.za for examples, assignments and Java code implementing the algorithms.
2,198 citations
"Inferring network interactions usin..." refers methods in this paper
...In this paper, we propose to apply a hybrid of ANNs and swarm intelligence (SI) methods [ 12 ] to infer network interactions from time-series data....
[...]
TL;DR: Probabilistic Boolean Networks (PBN) are introduced that share the appealing rule-based properties of Boolean networks, but are robust in the face of uncertainty.
Abstract: Motivation: Our goal is to construct a model for genetic regulatory networks such that the model class: (i) incorporates rule-based dependencies between genes; (ii) allows the systematic study of global network dynamics; (iii) is able to cope with uncertainty, both in the data and the model selection; and (iv) permits the quantification of the relative influence and sensitivity of genes in their interactions with other genes. Results: We introduce Probabilistic Boolean Networks (PBN) that share the appealing rule-based properties of Boolean networks, but are robust in the face of uncertainty. We show how the dynamics of these networks can be studied in the probabilistic context of Markov chains, with standard Boolean networks being special cases. Then, we discuss the relationship between PBNs and Bayesian networks—a family of graphical models that explicitly represent probabilistic relationships between variables. We show how probabilistic dependencies between a gene and its parent genes, constituting the basic building blocks of Bayesian networks, can be obtained from PBNs. Finally, we present methods for quantifying the influence of genes on other genes, within the context of PBNs. Examples illustrating the above concepts are presented throughout the paper.
1,571 citations
Proceedings Article•
01 Jan 1998TL;DR: This study investigates the possibility of completely infer a complex regulatory network architecture from input/output patterns of its variables using binary models of genetic networks, and finds the problem to be tractable within the conditions tested so far.
Abstract: Given the immanent gene expression mapping covering whole genomes during development, health and disease, we seek computational methods to maximize functional inference from such large data sets. Is it possible, in principle, to completely infer a complex regulatory network architecture from input/output patterns of its variables? We investigated this possibility using binary models of genetic networks. Trajectories, or state transition tables of Boolean nets, resemble time series of gene expression. By systematically analyzing the mutual information between input states and output states, one is able to infer the sets of input elements controlling each element or gene in the network. This process is unequivocal and exact for complete state transition tables. We implemented this REVerse Engineering ALgorithm (REVEAL) in a C program, and found the problem to be tractable within the conditions tested so far. For n = 50 (elements) and k = 3 (inputs per element), the analysis of incomplete state transition tables (100 state transition pairs out of a possible 10(exp 15)) reliably produced the original rule and wiring sets. While this study is limited to synchronous Boolean networks, the algorithm is generalizable to include multi-state models, essentially allowing direct application to realistic biological data sets. The ability to adequately solve the inverse problem may enable in-depth analysis of complex dynamic systems in biology and other fields.
1,031 citations
01 Dec 1998
TL;DR: The results suggest that a minor set of temporal data may be sufficient to construct the model at the genome level, and a comprehensive discussion of other extended models are given: the RNA Model, the Protein Model, and the Time Delay Model.
Abstract: We propose a differential equation model for gene expression and provide two methods to construct the model from a set of temporal data. We model both transcription and translation by kinetic equations with feedback loops from translation products to transcription. Degradation of proteins and mRNAs is also incorporated. We study two methods to construct the model from experimental data: Minimum Weight Solutions to Linear Equations (MWSLE), which determines the regulation by solving under-determined linear equations, and Fourier Transform for Stable Systems (FTSS), which refines the model with cell cycle constraints. The results suggest that a minor set of temporal data may be sufficient to construct the model at the genome level. We also give a comprehensive discussion of other extended models: the RNA Model, the Protein Model, and the Time Delay Model.
777 citations