Showing papers on "Artificial neural network published in 2007"

Learning to rank: from pairwise approach to listwise approach

Abstract: The paper is concerned with learning to rank, which is to construct a model or a function for ranking objects. Learning to rank is useful for document retrieval, collaborative filtering, and many other applications. Several methods for learning to rank have been proposed, which take object pairs as 'instances' in learning. We refer to them as the pairwise approach in this paper. Although the pairwise approach offers advantages, it ignores the fact that ranking is a prediction task on list of objects. The paper postulates that learning to rank should adopt the listwise approach in which lists of objects are used as 'instances' in learning. The paper proposes a new probabilistic method for the approach. Specifically it introduces two probability models, respectively referred to as permutation probability and top k probability, to define a listwise loss function for learning. Neural Network and Gradient Descent are then employed as model and algorithm in the learning method. Experimental results on information retrieval show that the proposed listwise approach performs better than the pairwise approach.

Robust Object Recognition with Cortex-Like Mechanisms

Abstract: We introduce a new general framework for the recognition of complex visual scenes, which is motivated by biology: We describe a hierarchical system that closely follows the organization of visual cortex and builds an increasingly complex and invariant feature representation by alternating between a template matching and a maximum pooling operation. We demonstrate the strength of the approach on a range of recognition tasks: From invariant single object recognition in clutter to multiclass categorization problems and complex scene understanding tasks that rely on the recognition of both shape-based as well as texture-based objects. Given the biological constraints that the system had to satisfy, the approach performs surprisingly well: It has the capability of learning from only a few training examples and competes with state-of-the-art systems. We also discuss the existence of a universal, redundant dictionary of features that could handle the recognition of most object categories. In addition to its relevance for computer vision, the success of this approach suggests a plausibility proof for a class of feedforward models of object recognition in cortex

Greedy Layer-Wise Training of Deep Networks

Abstract: Complexity theory of circuits strongly suggests that deep architectures can be much more ef cient (sometimes exponentially) than shallow architectures, in terms of computational elements required to represent some functions Deep multi-layer neural networks have many levels of non-linearities allowing them to compactly represent highly non-linear and highly-varying functions However, until recently it was not clear how to train such deep networks, since gradient-based optimization starting from random initialization appears to often get stuck in poor solutions Hinton et al recently introduced a greedy layer-wise unsupervised learning algorithm for Deep Belief Networks (DBN), a generative model with many layers of hidden causal variables In the context of the above optimization problem, we study this algorithm empirically and explore variants to better understand its success and extend it to cases where the inputs are continuous or where the structure of the input distribution is not revealing enough about the variable to be predicted in a supervised task Our experiments also confirm the hypothesis that the greedy layer-wise unsupervised training strategy mostly helps the optimization, by initializing weights in a region near a good local minimum, giving rise to internal distributed representations that are high-level abstractions of the input, bringing better generalization

An empirical evaluation of deep architectures on problems with many factors of variation

Abstract: Recently, several learning algorithms relying on models with deep architectures have been proposed. Though they have demonstrated impressive performance, to date, they have only been evaluated on relatively simple problems such as digit recognition in a controlled environment, for which many machine learning algorithms already report reasonable results. Here, we present a series of experiments which indicate that these models show promise in solving harder learning problems that exhibit many factors of variation. These models are compared with well-established algorithms such as Support Vector Machines and single hidden-layer feed-forward neural networks.

EEG signal classification using wavelet feature extraction and a mixture of expert model

Abstract: Mixture of experts (ME) is modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME network structure for detection of epileptic seizure. The detection of epileptiform discharges in the EEG is an important component in the diagnosis of epilepsy. EEG signals were decomposed into the frequency sub-bands using discrete wavelet transform (DWT). Then these sub-band frequencies were used as an input to a ME network with two discrete outputs: normal and epileptic. In order to improve accuracy, the outputs of expert networks were combined according to a set of local weights called the ''gating function''. The invariant transformations of the ME probability density functions include the permutations of the expert labels and the translations of the parameters in the gating functions. The performance of the proposed model was evaluated in terms of classification accuracies and the results confirmed that the proposed ME network structure has some potential in detecting epileptic seizures. The ME network structure achieved accuracy rates which were higher than that of the stand-alone neural network model.

A Leaf Recognition Algorithm for Plant Classification Using Probabilistic Neural Network

Abstract: In this paper, we employ probabilistic neural network (PNN) with image and data processing techniques to implement a general purpose automated leaf recognition for plant classification. 12 leaf features are extracted and orthogonalized into 5 principal variables which consist the input vector of the PNN. The PNN is trained by 1800 leaves to classify 32 kinds of plants with an accuracy greater than 90%. Compared with other approaches, our algorithm is an accurate artificial intelligence approach which is fast in execution and easy in implementation.

TMVA - Toolkit for Multivariate Data Analysis

Abstract: n high-energy physics, with the search for ever smaller signals in ever larger data sets, it has become essential to extract a maximum of the available information from the data. Multivariate classification methods based on machine learning techniques have become a fundamental ingredient to most analyses. Also the multivariate classifiers themselves have significantly evolved in recent years. Statisticians have found new ways to tune and to combine classifiers to further gain in performance. Integrated into the analysis framework ROOT, TMVA is a toolkit which hosts a large variety of multivariate classification algorithms. They range from rectangular cut optimization using a genetic algorithm and from one- and multidimensional likelihood estimators, over linear and nonlinear discriminants and neural networks, to sophisticated more recent classifiers such as a support vector machine, boosted decision trees and rule ensemble fitting. TMVA manages the simultaneous training, testing, and performance evaluation of all these classifiers with a user-friendly interface, and expedites the application of the trained classifiers to data.

Approximate Entropy-Based Epileptic EEG Detection Using Artificial Neural Networks

Abstract: The electroencephalogram (EEG) signal plays an important role in the diagnosis of epilepsy. The EEG recordings of the ambulatory recording systems generate very lengthy data and the detection of the epileptic activity requires a time-consuming analysis of the entire length of the EEG data by an expert. The traditional methods of analysis being tedious, many automated diagnostic systems for epilepsy have emerged in recent years. This paper proposes a neural-network-based automated epileptic EEG detection system that uses approximate entropy (ApEn) as the input feature. ApEn is a statistical parameter that measures the predictability of the current amplitude values of a physiological signal based on its previous amplitude values. It is known that the value of the ApEn drops sharply during an epileptic seizure and this fact is used in the proposed system. Two different types of neural networks, namely, Elman and probabilistic neural networks, are considered in this paper. ApEn is used for the first time in the proposed system for the detection of epilepsy using neural networks. It is shown that the overall accuracy values as high as 100% can be achieved by using the proposed system

New Delay-Dependent Stability Criteria for Neural Networks With Time-Varying Delay

Abstract: In this letter, a new method is proposed for stability analysis of neural networks (NNs) with a time-varying delay. Some less conservative delay-dependent stability criteria are established by considering the additional useful terms, which were ignored in previous methods, when estimating the upper bound of the derivative of Lyapunov functionals and introducing the new free-weighting matrices. Numerical examples are given to demonstrate the effectiveness and the benefits of the proposed method

Multiple linear regression and artificial neural networks based on principal components to predict ozone concentrations

Abstract: The prediction of tropospheric ozone concentrations is very important due to the negative impacts of ozone on human health, climate and vegetation. The development of models to predict ozone concentrations is thus very useful because it can provide early warnings to the population and also reduce the number of measuring sites. The aim of this study was to predict next day hourly ozone concentrations through a new methodology based on feedforward artificial neural networks using principal components as inputs. The developed model was compared with multiple linear regression, feedforward artificial neural networks based on the original data and also with principal component regression. Results showed that the use of principal components as inputs improved both models prediction by reducing their complexity and eliminating data collinearity.

Neural Network Applications in Power Electronics and Motor Drives—An Introduction and Perspective

Abstract: Artificial intelligence (AI) techniques, particularly the neural networks, are recently having significant impact on power electronics and motor drives. Neural networks have created a new and advancing frontier in power electronics, which is already a complex and multidisciplinary technology that is going through dynamic evolution in the recent years. This paper gives a comprehensive introduction and perspective of neural network applications in the intelligent control and estimation for power electronics and motor drives area. The principal topologies of neural networks that are currently most relevant for applications in power electronics have been reviewed including the detailed description of their properties. Both feedforward and feedback or recurrent architectures have been covered in the description. The application examples that are discussed in this paper include nonlinear function generation, delayless filtering and waveform processing, feedback signal processing of vector drive, space vector PWM of two-level and multilevel inverters, adaptive flux vector estimation, and some of their combination for vector-controlled ac drive. Additional selected applications in the literature are included in the references. From the current trend of the technology, it appears that neural networks will find widespread applications in power electronics and motor drives in future

Artificial Bee Colony (ABC) Optimization Algorithm for Training Feed-Forward Neural Networks

Abstract: Training an artificial neural network is an optimization task since it is desired to find optimal weight set of a neural network in training process. Traditional training algorithms has some drawbacks such as getting stuck in local minima and computational complexity. Therefore, evolutionary algorithms are employed to train neural networks to overcome these issues. In this work, Artificial Bee Colony (ABC) Algorithm which has good exploration and exploitation capabilities in searching optimal weight set is used in training neural networks.

Mixed-Band Wavelet-Chaos-Neural Network Methodology for Epilepsy and Epileptic Seizure Detection

Abstract: A novel wavelet-chaos-neural network methodology is presented for classification of electroencephalograms (EEGs) into healthy, ictal, and interictal EEGs. Wavelet analysis is used to decompose the EEG into delta, theta, alpha, beta, and gamma sub-bands. Three parameters are employed for EEG representation: standard deviation (quantifying the signal variance), correlation dimension, and largest Lyapunov exponent (quantifying the non-linear chaotic dynamics of the signal). The classification accuracies of the following techniques are compared: 1) unsupervised-means clustering; 2) linear and quadratic discriminant analysis; 3) radial basis function neural network; 4) Levenberg-Marquardt backpropagation neural network (LMBPNN). To reduce the computing time and output analysis, the research was performed in two phases: band-specific analysis and mixed-band analysis. In phase two, over 500 different combinations of mixed-band feature spaces consisting of promising parameters from phase one of the research were investigated. It is concluded that all three key components of the wavelet-chaos-neural network methodology are important for improving the EEG classification accuracy. Judicious combinations of parameters and classifiers are needed to accurately discriminate between the three types of EEGs. It was discovered that a particular mixed-band feature space consisting of nine parameters and LMBPNN result in the highest classification accuracy, a high value of 96.7%.

Hybrid neural network models for hydrologic time series forecasting

Abstract: The need for increased accuracies in time series forecasting has motivated the researchers to develop innovative models. In this paper, a new hybrid time series neural network model is proposed that is capable of exploiting the strengths of traditional time series approaches and artificial neural networks (ANNs). The proposed approach consists of an overall modelling framework, which is a combination of the conventional and ANN techniques. The steps involved in the time series analysis, e.g. de-trending and de-seasonalisation, can be carried out before gradually presenting the modified time series data to the ANN. The proposed hybrid approach for time series forecasting is tested using the monthly streamflow data at Colorado River at Lees Ferry, USA. Specifically, results from four time series models of auto-regressive (AR) type and four ANN models are presented. The results obtained in this study suggest that the approach of combining the strengths of the conventional and ANN techniques provides a robust modelling framework capable of capturing the non-linear nature of the complex time series and thus producing more accurate forecasts. Although the proposed hybrid neural network models are applied in hydrology in this study, they have tremendous scope for application in a wide range of areas for achieving increased accuracies in time series forecasting.

Introduction to artificial neural networks.

Abstract: The coupling of computer science and theoretical bases such as nonlinear dynamics and chaos theory allows the creation of 'intelligent' agents, such as artificial neural networks (ANNs), able to adapt themselves dynamically to problems of high complexity. ANNs are able to reproduce the dynamic interaction of multiple factors simultaneously, allowing the study of complexity; they can also draw conclusions on individual basis and not as average trends. These tools can offer specific advantages with respect to classical statistical techniques. This article is designed to acquaint gastroenterologists with concepts and paradigms related to ANNs. The family of ANNs, when appropriately selected and used, permits the maximization of what can be derived from available data and from complex, dynamic, and multidimensional phenomena, which are often poorly predictable in the traditional 'cause and effect' philosophy.

A comparison of machine learning techniques for phishing detection

Abstract: There are many applications available for phishing detection. However, unlike predicting spam, there are only few studies that compare machine learning techniques in predicting phishing. The present study compares the predictive accuracy of several machine learning methods including Logistic Regression (LR), Classification and Regression Trees (CART), Bayesian Additive Regression Trees (BART), Support Vector Machines (SVM), Random Forests (RF), and Neural Networks (NNet) for predicting phishing emails. A data set of 2889 phishing and legitimate emails is used in the comparative study. In addition, 43 features are used to train and test the classifiers.

Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity

Abstract: The persistent modification of synaptic efficacy as a function of the relative timing of pre- and postsynaptic spikes is a phenomenon known as spike-timing-dependent plasticity (STDP). Here we show that the modulation of STDP by a global reward signal leads to reinforcement learning. We first derive analytically learning rules involving reward-modulated spike-timing-dependent synaptic and intrinsic plasticity, by applying a reinforcement learning algorithm to the stochastic spike response model of spiking neurons. These rules have several features common to plasticity mechanisms experimentally found in the brain. We then demonstrate in simulations of networks of integrate-and-fire neurons the efficacy of two simple learning rules involving modulated STDP. One rule is a direct extension of the standard STDP model (modulated STDP), and the other one involves an eligibility trace stored at each synapse that keeps a decaying memory of the relationships between the recent pairs of pre- and postsynaptic spike pairs (modulated STDP with eligibility trace). This latter rule permits learning even if the reward signal is delayed. The proposed rules are able to solve the XOR problem with both rate coded and temporally coded input and to learn a target output firing-rate pattern. These learning rules are biologically plausible, may be used for training generic artificial spiking neural networks, regardless of the neural model used, and suggest the experimental investigation in animals of the existence of reward-modulated STDP.

Synaptic Dynamics in Analog VLSI

Abstract: Synapses are crucial elements for computation and information transfer in both real and artificial neural systems. Recent experimental findings and theoretical models of pulse-based neural networks suggest that synaptic dynamics can play a crucial role for learning neural codes and encoding spatiotemporal spike patterns. Within the context of hardware implementations of pulse-based neural networks, several analog VLSI circuits modeling synaptic functionality have been proposed. We present an overview of previously proposed circuits and describe a novel analog VLSI synaptic circuit suitable for integration in large VLSI spike-based neural systems. The circuit proposed is based on a computational model that fits the real postsynaptic currents with exponentials. We present experimental data showing how the circuit exhibits realistic dynamics and show how it can be connected to additional modules for implementing a wide range of synaptic properties.

Streamflow Forecasting Using Different Artificial Neural Network Algorithms

Abstract: Forecasts of future events are required in many activities associated with planning and operation of the components of a water resources system. For the hydrologic component, there is a need for both short term and long term forecasts of streamflow events in order to optimize the system or to plan for future expansion or reduction. This paper presents a comparison of different artificial neural networks (ANNs) algorithms for short term daily streamflow forecasting. Four different ANN algorithms, namely, backpropagation, conjugate gradient, cascade correlation, and Levenberg–Marquardt are applied to continuous streamflow data of the North Platte River in the United States. The models are verified with untrained data. The results from the different algorithms are compared with each other. The correlation analysis was used in the study and found to be useful for determining appropriate input vectors to the ANNs.

The application of machine learning to structural health monitoring

Abstract: In broad terms, there are two approaches to damage identification Model-driven methods establish a high-fidelity physical model of the structure, usually by finite element analysis, and then establish a comparison metric between the model and the measured data from the real structure If the model is for a system or structure in normal (ie undamaged) condition, any departures indicate that the structure has deviated from normal condition and damage is inferred Data-driven approaches also establish a model, but this is usually a statistical representation of the system, eg a probability density function of the normal condition Departures from normality are then signalled by measured data appearing in regions of very low density The algorithms that have been developed over the years for data-driven approaches are mainly drawn from the discipline of pattern recognition, or more broadly, machine learning The object of this paper is to illustrate the utility of the data-driven approach to damage identification by means of a number of case studies

A Graph-Based Evolutionary Algorithm: Genetic Network Programming (GNP) and Its Extension Using Reinforcement Learning

Abstract: This paper proposes a graph-based evolutionary algorithm called Genetic Network Programming (GNP). Our goal is to develop GNP, which can deal with dynamic environments efficiently and effectively, based on the distinguished expression ability of the graph (network) structure. The characteristics of GNP are as follows. 1) GNP programs are composed of a number of nodes which execute simple judgment/processing, and these nodes are connected by directed links to each other. 2) The graph structure enables GNP to re-use nodes, thus the structure can be very compact. 3) The node transition of GNP is executed according to its node connections without any terminal nodes, thus the past history of the node transition affects the current node to be used and this characteristic works as an implicit memory function. These structural characteristics are useful for dealing with dynamic environments. Furthermore, we propose an extended algorithm, “GNP with Reinforcement Learning (GNPRL)” which combines evolution and reinforcement learning in order to create effective graph structures and obtain better results in dynamic environments. In this paper, we applied GNP to the problem of determining agents' behavior to evaluate its effectiveness. Tileworld was used as the simulation environment. The results show some advantages for GNP over conventional methods.

Multiclass Support Vector Machines for EEG-Signals Classification

Abstract: In this paper, we proposed the multiclass support vector machine (SVM) with the error-correcting output codes for the multiclass electroencephalogram (EEG) signals classification problem. The probabilistic neural network (PNN) and multilayer perceptron neural network were also tested and benchmarked for their performance on the classification of the EEG signals. Decision making was performed in two stages: feature extraction by computing the wavelet coefficients and the Lyapunov exponents and classification using the classifiers trained on the extracted features. The purpose was to determine an optimum classification scheme for this problem and also to infer clues about the extracted features. Our research demonstrated that the wavelet coefficients and the Lyapunov exponents are the features which well represent the EEG signals and the multiclass SVM and PNN trained on these features achieved high classification accuracies