Journal Article•
A Neural Architecture Based on the Adaptive Resonant Theory and Recurrent Neural Networks
TL;DR: It is verified that the ART2/EKF is capable of finding patterns in the traffic time series as well as to obtain the transmission rate that should be made available in order to avoid byte losses in a computer network link.
Abstract: In this paper, we propose a novel neural architecture that adaptively learns an input -output mapping using both supervised and non-supervised trainings. This neural architecture consists of a combination of an ART2 (Adaptive Resonance Theory) neural network and recurrent neural networks. For this end, we developed an Extended Kalman Filter (EKF) based training algorithm for the involved recurrent neural networks. The proposed ART2/EKF neural network is inspired in the visual cortex and the brain mechanisms. More precisely, the non-supervised ART2 neural network is used to coordinate specialized recurrent neural networks in a specific input space domain. Our aim is to design a neural system that learns in real time a new input pattern without retraining the neural network with the whole training set. The proposed neural architecture is used to adaptively predict the traffic volume in a computer network. We verify that the ART2/EKF is capable of finding patterns in the traffic time series as well as to obtain the transmission rate that should be made available in order to avoid byte losses in a computer network link.
Citations
More filters
TL;DR: In this article, the authors provide an overview of unsupervised learning in the domain of networking, and provide a comprehensive review of the current state of the art in this area, by synthesizing insights from previous survey papers.
Abstract: While machine learning and artificial intelligence have long been applied in networking research, the bulk of such works has focused on supervised learning. Recently, there has been a rising trend of employing unsupervised machine learning using unstructured raw network data to improve network performance and provide services, such as traffic engineering, anomaly detection, Internet traffic classification, and quality of service optimization. The growing interest in applying unsupervised learning techniques in networking stems from their great success in other fields, such as computer vision, natural language processing, speech recognition, and optimal control (e.g., for developing autonomous self-driving cars). In addition, unsupervised learning can unconstrain us from the need for labeled data and manual handcrafted feature engineering, thereby facilitating flexible, general, and automated methods of machine learning. The focus of this survey paper is to provide an overview of applications of unsupervised learning in the domain of networking. We provide a comprehensive survey highlighting recent advancements in unsupervised learning techniques, and describe their applications in various learning tasks, in the context of networking. We also provide a discussion on future directions and open research issues, while identifying potential pitfalls. While a few survey papers focusing on applications of machine learning in networking have previously been published, a survey of similar scope and breadth is missing in the literature. Through this timely review, we aim to advance the current state of knowledge, by carefully synthesizing insights from previous survey papers, while providing contemporary coverage of the recent advances and innovations.
182 citations
Journal Article•
TL;DR: The paper posits the basic theories and definitions of business intelligence and machine learning, and describes the insights on the role and requirement of real time BI by examining the business needs.
Abstract: Growing technological progressions have given rise to many issues concerning the contemporary decision making in business, which is a difficult phenomenon without Business Intelligence/ Machine Learning. The linking of machine learning with business intelligence is not only pivotal for business decision making but also for the business intelligence in totality, owing to the reason that in absence of machine learning, decision making couldn’t take place efficaciously. Machines need to learn, re-learn, and then only they can help your learning process. The below paper seeks to make this concept simple/ easy by removing the ambiguities using a general framework. In order to prove the impact of machine learning on business intelligence, we need to forecast the trends, what is going around the world – business has to stay updated, then only it can be a successful endeavour. The paper posits the basic theories and definitions of business intelligence and machine learning. To learn from the past and forecast the future trends, many companies are adopting business intelligence tools and systems. Companies have understood the brilliance of enforcing achievements of the goals defined by their business strategies through business intelligence concepts and with the help of machine learning. It describes the insights on the role and requirement of real time BI by examining the business needs. Keywords: Business Intelligence (BI); Machine Learning (ML); Artificial Neural Networks (ANN); Self-Organizing Maps (SOM); Data Mining (DM); Data Warehousing (DW).
11 citations
Cites background from "A Neural Architecture Based on the ..."
...(Vieira & Lee, 2007) 4.3 Reinforcement Learning: It is not as popular as the other two types due to the fact that it is not as mature and its applicability is limited to a small set of real world situations....
[...]
TL;DR: This experiment measures the Hausdorff timing values, mean, standard deviation, and median of keystroke features, such as latency, duration, digraph, and their combinations, and compares their performance with the stochastic diffusion search for feature subset selection.
Abstract: Authentication plays an important role in dealing with security. Securing sensitive data and computer systems by allowing easy access for authenticated users and withstanding the attacks of imposters is one of the major challenges in the field of computer security. Nowadays, passwords have become the trend to control access to computer systems. Biometrics are used to measure and analyze an individual's unique behavioral or physiological patterns for authentication purposes. Keystroke dynamics have emerged as an important method in analyzing the typing rhythm in biometric techniques, as they provide an ease of use and increased trustworthiness associated with biometrics for creating username and password schemes. In this experiment, we measure the Hausdorff timing values, mean, standard deviation, and median of keystroke features, such as latency, duration, digraph, and their combinations, and compare their performance. The stochastic diffusion search is used for feature subset selection.
6 citations
Cites background from "A Neural Architecture Based on the ..."
...There are two types of ART, ART 1 and ART 2 [8]....
[...]
01 Dec 2012
TL;DR: This paper proposes a variant of Self-Organizing Map approach to finding a compromised solution for a multi-objective path planning problem that does not require user involvement, and shows the immunity of the algorithm to local minima, and its efficiency to respond to multiple objectives.
Abstract: The demand for Unmanned Aerial Vehicle (UAV) extends to various civil and military missions. While the use of remotely controlled UAV reduces the rate of human casualties in hazardous environments, it is reported that most of UAV accidents are caused by human factor errors. Automated path planning is required and because of the multi-objective nature of UAV's missions, several heuristic approaches to path planning have been proposed in order to automate UAV's navigation. While solving multi-objective problems requires the search for a set of pareto-optimal points, it requires the involvement of the user to select the desired result from the solution space. In this paper, we propose a variant of Self-Organizing Map approach to finding a compromised solution for a multi-objective path planning problem that does not require user involvement. Preliminary tests conducted in virtual environments have shown the immunity of our algorithm to local minima, and its efficiency to respond to multiple objectives.
5 citations
Dissertation•
01 Dec 2016
TL;DR: This thesis presents a new anticorrelation-based spectral clustering formulation with the objective to improve both: speed and quality of segmentation and designed robust non-parametric similarity models according to Gestaltic laws of visual segregation.
Abstract: Recent advances in cellular and subcellular microscopy demonstrated its potential towards unravelling the mechanisms of various diseases at the molecular level. The biggest challenge in both human- and computer-based visual analysis of micrographs is the variety of nanostructures and mitochondrial morphologies. The state-of-the-art is, however, dominated by supervised manual data annotation and early attempts to automate the segmentation process were based on supervised machine learning techniques which require large datasets for training. Given a minimal number of training sequences or none at all, unsupervised machine learning formulations, such as spectral dimensionality reduction, are known to be superior in detecting salient image structures.
This thesis presents three major contributions developed around the spectral clustering framework which is proven to capture perceptual organization features. Firstly, we approach the problem of mitochondria localization. We propose a novel grouping method for the extracted line segments which describes the normal mitochondrial morphology. Experimental findings show that the clusters obtained successfully model the inner mitochondrial membrane folding and therefore can be used as markers for the subsequent segmentation approaches. Secondly, we developed an unsupervised mitochondria segmentation framework. This method follows the evolutional ability of human vision to extrapolate salient membrane structures in a micrograph. Furthermore, we designed robust non-parametric similarity models according to Gestaltic laws of visual segregation. Experiments demonstrate that such models automatically adapt to the statistical structure of the biological domain and return optimal performance in pixel classification tasks under the wide variety of distributional assumptions. The last major contribution addresses the computational complexity of spectral clustering. Here, we introduced a new anticorrelation-based spectral clustering formulation with the objective to improve both: speed and quality of segmentation. The experimental findings showed the applicability of our dimensionality reduction algorithm to very large scale problems as well as asymmetric, dense and non-Euclidean datasets.
4 citations
Cites methods from "A Neural Architecture Based on the ..."
...A combination of an ART and a RNN which adaptively learns an input-output mapping using both supervised and unsupervised formulations is presented, for example, by Vieira and Lee in [215]....
[...]
References
More filters
Book•
01 Jan 2007
TL;DR: A circular cribbage board having a circular base plate on which a circular counter disc, bearing a circular scale having 122 divisions numbered consecutively from 0, is mounted for rotation.
Abstract: From the Publisher:
Dramatically updating and extending the first edition, published in 1995, the second edition of The Handbook of Brain Theory and Neural Networks presents the enormous progress made in recent years in the many subfields related to the two great questions: How does the brain work? and, How can we build intelligent machines?
Once again, the heart of the book is a set of almost 300 articles covering the whole spectrum of topics in brain theory and neural networks. The first two parts of the book, prepared by Michael Arbib, are designed to help readers orient themselves in this wealth of material. Part I provides general background on brain modeling and on both biological and artificial neural networks. Part II consists of "Road Maps" to help readers steer through articles in part III on specific topics of interest. The articles in part III are written so as to be accessible to readers of diverse backgrounds. They are cross-referenced and provide lists of pointers to Road Maps, background material, and related reading.
The second edition greatly increases the coverage of models of fundamental neurobiology, cognitive neuroscience, and neural network approaches to language. It contains 287 articles, compared to the 266 in the first edition. Articles on topics from the first edition have been updated by the original authors or written anew by new authors, and there are 106 articles on new topics.
3,487 citations
TL;DR: ART 2, a class of adaptive resonance architectures which rapidly self-organize pattern recognition categories in response to arbitrary sequences of either analog or binary input patterns, is introduced.
Abstract: A neural network includes a feature representation field which receives input patterns. Signals from the feature representation field select a category from a category representation field through a first adaptive filter. Based on the selected category, a template pattern is applied to the feature representation field, and a match between the template and the input is determined. If the angle between the template vector and a vector within the representation field is too great, the selected category is reset. Otherwise the category selection and template pattern are adapted to the input pattern as well as the previously stored template. A complex representation field includes signals normalized relative to signals across the field and feedback for pattern contrast enhancement.
1,865 citations
"A Neural Architecture Based on the ..." refers background or methods in this paper
...The adaptive resonance theory (ART) was introduced by Grossberg and Carpenter [5], during their studies of the modeling of systems of neurons....
[...]
...In this paper, we will focus on the ART2 architecture developed by Carpenter and Grossberg, because this neural network can deal with analog as well as binary inputs [5] [1]....
[...]
...The ART2 neural network algorithm used in this work is summarized below [5]: • Neural network configuration: There are some ART2 neural network parameters that should be initially chosen (Fig....
[...]
...ART2 [5] is a class of architectures categorizing arbitrary sequences of analog input patterns....
[...]
Book•
01 Dec 2001
TL;DR: This text is a practical guide to building Kalman filters and shows how the filtering equations can be applied to real-life problems and spends a great deal of time setting up a problem before the Kalman filter is actually formulated to give the reader an intuitive feel for the problem being addressed.
Abstract: This text is a practical guide to building Kalman filters and shows how the filtering equations can be applied to real-life problems Numerous examples are presented in detail, showing the many ways in which Kalman filters can be designed Computer code written in FORTRAN, MATLAB[registered], and True BASIC accompanies all of the examples so that the interested reader can verify concepts and explore issues beyond the scope of the text Sometimes mistakes are introduced intentionally to the initial filter designs to show the reader what happens when the filter is not working properly The text spends a great deal of time setting up a problem before the Kalman filter is actually formulated to give the reader an intuitive feel for the problem being addressed Real problems are seldom presented in the form of differential equations and they usually do not have unique solutions Therefore, the authors illustrate several different filtering approaches for tackling a problem Readers will gain experience in software and performance tradeoffs for determining the best filtering approach for the application at hand The second edition has two new chapters and an additional appendix In the first new chapter, a recursive digital filter known as the fading memory filter is introduced and it is shown that for some radar tracking applications the fading memory filter can yield similar performance to a Kalman filter at far less computational cost A second new chapter presents techniques for improving Kalman filter performance Included is a practical method for preprocessing measurement data when there are too many measurements for the filter to utilize in a given amount of time The chapter also contains practical methods for making the Kalman filter adaptive A new appendix has been added which serves as a central location and summary for the text's most important concepts and formulas MATLAB is a registered trademark of The MathWorks, Inc
724 citations
"A Neural Architecture Based on the ..." refers methods in this paper
...The Kalman filter consists of a group of equations that provide an efficient and recursive computation for the solution of the least squares method [11]....
[...]
TL;DR: These simulations suggest that recurrent controller networks trained by Kalman filter methods can combine the traditional features of state-space controllers and observers in a homogeneous architecture for nonlinear dynamical systems, while simultaneously exhibiting less sensitivity than do purely feedforward controller networks to changes in plant parameters and measurement noise.
Abstract: Although the potential of the powerful mapping and representational capabilities of recurrent network architectures is generally recognized by the neural network research community, recurrent neural networks have not been widely used for the control of nonlinear dynamical systems, possibly due to the relative ineffectiveness of simple gradient descent training algorithms. Developments in the use of parameter-based extended Kalman filter algorithms for training recurrent networks may provide a mechanism by which these architectures will prove to be of practical value. This paper presents a decoupled extended Kalman filter (DEKF) algorithm for training of recurrent networks with special emphasis on application to control problems. We demonstrate in simulation the application of the DEKF algorithm to a series of example control problems ranging from the well-known cart-pole and bioreactor benchmark problems to an automotive subsystem, engine idle speed control. These simulations suggest that recurrent controller networks trained by Kalman filter methods can combine the traditional features of state-space controllers and observers in a homogeneous architecture for nonlinear dynamical systems, while simultaneously exhibiting less sensitivity than do purely feedforward controller networks to changes in plant parameters and measurement noise. >
586 citations
"A Neural Architecture Based on the ..." refers methods in this paper
...Puskorius and Feldcamp applied the Kalman algorithm in the recurrent neural network training [13]....
[...]
Proceedings Article•
01 Jan 1988TL;DR: It is shown that training multilayer perceptrons is an identification problem for a nonlinear dynamic system which can be solved using the Extended Kalman Algorithm.
Abstract: A large fraction of recent work in artificial neural nets uses multilayer perceptrons trained with the back-propagation algorithm described by Rumelhart et. al. This algorithm converges slowly for large or complex problems such as speech recognition, where thousands of iterations may be needed for convergence even with small data sets. In this paper, we show that training multilayer perceptrons is an identification problem for a nonlinear dynamic system which can be solved using the Extended Kalman Algorithm. Although computationally complex, the Kalman algorithm usually converges in a few iterations. We describe the algorithm and compare it with back-propagation using two-dimensional examples.
360 citations