Showing papers presented at "The European Symposium on Artificial Neural Networks in 2006"

A survey of Sparse Component Analysis for blind source separation: principles, perspectives, and new challenges

Abstract: In this survey, we highlight the appealing features and challenges of Sparse Component Analysis (SCA) for blind source separation (BSS). SCA is a simple yet powerful framework to separate several sources from few sensors, even when the independence assumption is dropped. So far, SCA has been most successfully applied when the sources can be represented sparsely in a given basis, but many other potential uses of SCA remain unexplored. Among other challenging perspectives, we discuss how SCA could be used to exploit both the spatial diversity corresponding to the mixing process and the morphological diversity between sources to unmix even underdetermined convolutive mixtures. This raises several challenges, including the design of both provably good and numerically efficient algorithms for large-scale sparse approximation with overcomplete signal dictionaries.

Spatial filters for the classification of event-related potentials

Abstract: Spatial filtering is a widely used dimension reduction method in electroencephalogram based brain-computer interface systems. In this paper a new algorithm is proposed, which learns spatial filters from a training dataset. In contrast to existing approaches the proposed method yields spatial filters that are explicitly designed for the classification of event-related potentials, such as the P300 or movement-related potentials. The algorithm is tested, in combination with support vector machines, on several benchmark datasets from past BCI competitions and achieves state of the art results.

Visualizing Gene Interaction Graphs with Local Multidimensional Scaling

Abstract: Several bioinformatics data sets are naturally represented as graphs, for instance gene regulation, metabolic pathways, and protein- protein interactions. The graphs are often large and complex, and their straightforward visualizations are incomprehensible. We have recently de- veloped a new method called local multidimensional scaling for visualizing high-dimensional data sets. In this paper we adapt it to visualize graphs, and compare it with two commonly used graph visualization packages in visualizing yeast gene interaction graphs. The new method outperforms the alternatives in two crucial respects: It produces graph layouts that are both more trustworthy and have fever edge crossings.

Time Series Prediction using DirRec Strategy

Abstract: This paper demonstrates how the selection of Prediction Strategy is important in the Long-Term Prediction of Time Series. Two strategies are already used in the prediction purposes called Recursive and Direct. This paper presents a third one, DirRec, which combines the advantages of the two already used ones. A simple k-NN approximation method is used and all three strategies are applied to two benchmarks: Santa Fe and Poland Electricity Load time series.

FastISA: A fast fixed-point algorithm for Independent Subspace Analysis

Abstract: Independent Subspace Analysis (ISA; Hyvarinen & Hoyer, 2000) is an extension of ICA. In ISA, the components are divided into subspaces and compo- nents in different subspaces are assumed independent, whereas components in the same subspace have dependencies.In this paper we describe a fixed-point algorithm for ISA estimation, formulated in analogy to FastICA. In particular we give a proof of the quadratic convergence of the algorithm, and present simulations that confirm the fast convergence, but also show that the method is prone to convergence to local minima.

Nonlinear dynamics in neural computation.

Abstract: This tutorial reports on the use of nonlinear dynamics in several different models of neural systems. We discuss a number of dis- tinct approaches to neural information processing based on nonlinear dy- namics. The models we consider combine controlled chaotic models with phenomenological models of spiking mechanisms as well as using weakly chaotic systems. The recent work of several major researchers in this field is briefly introduced.

Learning vector quantization: The dynamics of winner-takes-all algorithms

Abstract: Winner-Takes-All (WTA) prescriptions for learning vector quantization (LVQ) are studied in the framework of a model situation: two competing prototype vectors are updated according to a sequence of example data drawn from a mixture of Gaussians. The theory of on-line learning allows for an exact mathematical description of the training dynamics, even if an underlying cost function cannot be identified. We compare the typical behavior of several WTA schemes including basic LVQ and unsupervised vector quantization. The focus is on the learning curves, i.e. the achievable generalization ability as a function of the number of training examples.

Linking non-binned spike train kernels to several existing spike train metrics.

Abstract: This work presents three kernel functions that can be used as inner product operators on non-binned spike trains, allowing the use of state-of-the-art classification techniques. One of the main advantages is that this approach does not require the spike trains to be binned. Thus a high temporal resolution is preserved which is needed when temporal coding is used. The kernels are closely related to several recent and often-used spike train metrics which take into account the biological variability of spike trains. It follows that the different existing metrics are unified by the spike train kernels presented. As a test of the classification potential of the new kernel functions, a jittered spike train template classification problem is solved.

Parallel Hardware Implementation of a Broad Class of Spiking Neurons using Serial Arithmetic

Abstract: Current digital, directly mapped implementations of spiking neural networks use serial processing and parallel arithmetic. On a standard CPU, this might be the good choice, but when using a Field Programmable Gate Array (FPGA), other implementation architectures are possible. This work present a hardware implementation of a broad class of integrate and fire spiking neurons with synapse models using parallel processing and serial arithmetic. This results in very fast and compact implementations of spiking neurons on FPGA.

Magnification Control for Batch Neural Gas

Abstract: Neural gas (NG) constitutes a very robust clustering algorithm which can be derived as stochastic gradient descent from a cost function closely connected to the quantization error. In the limit, an NG network samples the underlying data distribution. Thereby, the connection is not linear, rather, it follows a power law with magnification exponent different from the information theoretically optimum one in adaptive map formation. There exists a couple of schemes to explicitly control the exponent such as local learning which leads to a small change of the learning algorithm of NG. Batch NG constitutes a fast alternative optimization scheme for NG vector quantizers which has been derived from the same cost function and which constitutes a fast Newton optimization scheme. It possesses the same magnification factor (different from 1) as standard online NG. In this paper, we propose a method to integrate magnification control by local learning into batch NG. Thereby, the key observation is a link of local learning to an underlying cost function which opens the way towards alternative, e.g.batch optimization schemes. We validate the learning rule derived from this altered cost function in an artificial experimental setting and we demonstrate the benefit of magnification control to sample rare events for a real data set.

Data topology visualization for the Self-Organizing Map

Abstract: The Self-Organizing map (SOM), a powerful method for data mining and cluster extraction, is very useful for processing data of high dimensionality and complexity. Visualization methods present different aspects of the information learned by the SOM to gain insight and guide segmentation of the data. In this work, we propose a new visualization scheme that represents data topology superimposed on the SOM grid, and we show how it helps in the discovery of data structure.

The permutation test for feature selection by mutual information

Abstract: The estimation of mutual information for feature selection is often subject to inaccuracies due to noise, small sample size, bad choice of parameter for the estimator, etc The choice of a threshold above which a feature will be considered useful is thus difficult to make Therefore, the use of the permutation test to assess the reliability of the estimation is proposed The permutation test allows performing a non-parametric hypothesis test to select the relevant features and to build a Feature Relevance Diagram that visually synthesizes the result of the test

Learning what is important: feature selection and rule extraction in a virtual course

Abstract: Virtualcampusenvironmentsarebecomingamainstr eamalternativeto� traditionaldistancehighereducation.�TheInternet �mediumtheyuseallowsthe� gatheringofinformationonstudents'�usagebehavio ur.�Theknowledgeextracted� fromthisinformationcanbefedbacktothee/lear ningenvironmenttoease� teachers'�workload.�Inthiscontext,�twoproblemsa readdressedinthecurrent� study:�findingwhichusagefeaturesarebestatpre dictingonlinestudents'�marks,� andexplainingmarkpredictionintheformofsimpl eandinterpretablerules.�To� thateffect,�twomethodsareused:�FuzzyInductive� Reasoning�(FIR)�forfeature� selectionandOrthogonalSearch/BasedRuleExtraction � (OSRE).� Experiments� carriedoutontheavailabledataindicatethatstu dents'�markscanbeaccurately� predictedandthatasmallsubsetofvariablesexpl ainstheaccuracyofsuch� prediction,�whichcanbedescribedthroughasetof �actionablerules.��

Neural Networks and Machine Learning in Bioinformatics - Theory and Applications

Abstract: Bioinformatics is a promising and innovative research field. Despite of a high number of techniques specifically dedicated to bioinfor- matics problems as well as many successful applications, we are in the beginning of a process to massively integrate the aspects and experiences in the dierent core subjects such as biology, medicine, computer science, engineering, chemistry, physics, and mathematics. Within this rather wide area we focus on neural networks and machine learning related approaches in bioinformatics with particular emphasis on integrative research against the background of the above mentioned scope.

Data mining techniques for feature selection in blood cell recognition

Abstract: The paper presents and compares the data mining techniques for selection of the diagnostic features in the problem of blood cell recognition in leukemia. Different techniques are compared, including the linear SVM ranking, correlation and statistical analysis of centers and variances of clusters corresponding to different classes. We have applied radial kernel SVM as the classifier. The results of recognition of 10 classes of cells are presented and discussed.

Reducing policy degradation in neuro-dynamic programming.

Abstract: We focus on neuro-dynamic programming methods to learn state-action value functions and outline some of the inherent problems to be faced, when performing reinforcement learning in combination with function approximation. In an attempt to overcome some of these problems, we develop a reinforcement learning method that monitors the learning process, enables the learner to reflect whether it is better to cease learning, and thus obtains more stable learning results.

Stochastic Processes for Canonical Correlation Analysis.

Abstract: We consider two stochastic process methods for performing canonical correlation analysis (CCA). The flrst uses a Gaussian Process formulation of regression in which we use the current projection of one data set as the target for the other and then repeat in the opposite direction. The second uses a Dirichlet process of Gaussian models where the Gaussian models are determined by Probabilistic CCA [1]. The latter method is more computationally intensive but has the advantages of non-parametric approaches.

A Gaussian process latent variable model formulation of canonical correlation analysis.

Abstract: We investigate a nonparametric model with which to vi- sualize the relationship between two datasets. We base our model on Gaussian Process Latent Variable Models (GPLVM)(1),(2), a probabilisti- cally defined latent variable model which takes the alternative approach of marginalizing the parameters and optimizing the latent variables; we optimize a latent variable set for each dataset, which preserves the corre- lations between the datasets, resulting in a GPLVM formulation of canon- ical correlation analysis which can be nonlinearised by choice of covariance function.

Recent Trends in Online Learning for Cognitive Robotics

Abstract: We present a review of recent trends in cognitive robotics that deal with online learning approaches to the acquisition of knowledge, control strategies and behaviors of a cognitive robot or agent. Along this line we focus on the topics of object recognition in cognitive vision, tra- jectory learning and adaptive control of multi-DOF robots, task learning from demonstration, and general developmental approaches in robotics. We argue for the relevance of online learning as a key ability for future intelligent robotic systems to allow flexible and adaptive behavior within a changing and unpredictable environment.

Bayesian Source Separation: Beyond PCA and ICA

Abstract: Blind source separation (BSS) has become one of the major signal and image processing area in many applications. Principal com- ponent analysis (PCA) and Independent component analysis (ICA) have become two main classical approaches for this problem. However, these two approaches have their limits which are mainly, the assumptions that the data are temporally iid and that the model is exact (no noise). In this paper, we first show that the Bayesian inference framework gives the pos- sibility to go beyond these limits while obtaining PCA and ICA algorithms as particular cases. Then, we propose different a priori models for sources which progressively account for different properties of the sources. Finally, we illustrate the application of these different models in spectrometry, in astrophysical imaging, in satellite imaging and in hyperspectral imaging.

Fuzzy Image Segmentation with Fuzzy Labelled Neural Gas

Abstract: Processing biological data often requires handling of uncer- tain and sometimes inconsistent information Particularly when coping with image segmentation tasks against biomedical background, a clear de- scription of for example tissue borders is often hard to obtain On the other hand, there are only a few promising segmentation algorithms being able to process fuzzy input data This paper describes one novel alterna- tive applying the recently introduced Fuzzy Labelled Neural Gas (FLNG) as subsequent classiflcation step to a biologically relevant fuzzy labelling with underlying image feature extraction

Classification of Boar Sperm Head Images using Learning Vector Quantization

Abstract: We apply Learning Vector Quantization (LVQ) in automated boar semen quality assessment. The classification of single boar sperm heads into healthy (normal) and non-normal ones is based on grey-scale microscopic images only. Sample data was classified by veterinary experts and is used for training a system with a number of prototypes for each class. We apply as training schemes Kohonen’s LVQ1 and the variants Generalized LVQ (GLVQ) and Generalized Relevance LVQ (GRLVQ). We compare their performance and study the influence of the employed metric.

Visual Data Mining and Machine Learning.

Abstract: Information visualization and visual data mining leverage the human visual system to provide insight and understanding of unorganized data. In order to scale to massive sets of high dimensional data, simplification methods are needed, so as to select important dimensions and objects. Some machine learning algorithms try to solve those problems. We give in this paper an overview of information visualization and survey the links between this field and machine learning.

Saliency extraction with a distributed spiking neural network

Abstract: We present a distributed spiking neuron network (SNN) for handling low-level visual perception in order to extract salient locations in robot camera images. We describe a new method which reduce the computional load of the whole system, stemming from our choices of architecture. We also describe a modeling of post-synaptic potential, which allows to quickly compute the contribution of a sum of incoming spikes to a neuron's membrane potential. The interests of this saliency extraction method, which differs from classical image processing, are also exposed.

Using sampling methods to improve binding site predictions

Abstract: Currently the best algorithms for transcription factor binding site prediction are severely limited in accuracy. In previous work we combine random selection under-sampling into SMOTE over-sampling technique, working with several classification algorithms from machine learning field to integrate binding site predictions. In this paper, we improve the classification result with the aid of Tomek links as an either undersampling or cleaning technique.

An algorithm for fast and reliable ESOM learning

Abstract: The training of Emergent Self-organizing Maps (ESOM )w ith large datasets can be a computationally demanding task. Batch learning may be used to speed up training. It is demonstrated here, however, that the representation of clusters in the data space on maps trained with batch learning is poor compared to sequential training. This effect occurs even for very clear cluster structures. The k-batch learning algorithm is preferrable, because it creates the same quality of representation as sequential learning but maintains important properties of batch learning that can be exploited for speedup.

Cluster detection algorithm in neural networks

Abstract: Complex networks have received much attention in the last few years, and reveal global properties of interacting systems in domains like biology, social sciences and technology. One of the key feature of complex networks is their clusterized structure. Most methods applied to study complex networks are based on undirected graphs. However, when considering neural networks, the directionality of links is fundamental. In this article, a method of cluster detection is extended for directed graphs. We show how the extended method is more e‐cient to detect a clusterized structure in neural networks, without signiflcant increase of the computational cost.

Gaussian and Exponential Architectures in Small-World Associative Memories

Abstract: The performance of sparsely-connected associative memory models built from a set of perceptrons is investigated using different patterns of connectivity. Architectures based on Gaussian and exponential distributions are compared to networks created by progressively rewiring a locally-connected network. It is found that while all three architectures are capable of good pattern-completion performance, the Gaussian and exponential architectures require a significantly lower mean wiring length to achieve the same results. In the case of networks of low connectivity, relatively tight Gaussian and exponential distributions achieve the best overall performance.

FPGA implementation of an integrate-and-fire legion model for image segmentation

Abstract: Despite several previous studies, little progress has been made in build- ing successful neural systems for image segmentation in digital hardware. Spiking neural networks offer an opportunity to develop models of visual perception with- out any complex structure based on multiple neural maps. Such models use ele- mentary asynchronous computations that have motivated several implementations on analog devices, whereas digital implementations appear as quite unable to han- dle large spiking neural networks, for lack of density. In this work, we consider a model of integrate-and-fire neurons organized according to the standard LEGION architecture to segment grey-level images. Taking advantage of the local and dis- tributed structure of the model, a massively distributed implementation on FPGA using pipelined serial computations is developed. Results show that digital and flexible solutions may efficiently handle large networks of spiking neurons.

Semi-Blind Approaches for Source Separation and Independent component Analysis

Abstract: This paper is a survey of semi-blind source separation ap- proaches. Since Gaussian iid signals are not separable, simplest priors suggest to assume non Gaussian iid signals, or Gaussian non iid signals. Other priors can also been used, for instance discrete or bounded sources, positivity, etc. Although providing a generic framework for semi-blind source separation, Sparse Component Analysis and Bayesian ICA will just sketched in this paper, since two other survey papers develop in depth these approaches.