Showing papers presented at "The European Symposium on Artificial Neural Networks in 2008"

Safe exploration for reinforcement learning.

Abstract: In this paper we define and address the problem of safe exploration in the context of reinforcement learning. Our notion of safety is concerned with states or transitions that can lead to damage and thus must be avoided. We introduce the concepts of a safety function for determining a state’s safety degree and that of a backup policy that is able to lead the system under control from a critical state back to a safe one. Moreover, we present a level-based exploration scheme that is able to generate a comprehensive base of observations while adhering safety constraints.We evaluate our approach on a simplified simulation of a gas turbine.

Learning Inverse Dynamics: A Comparison

Abstract: While it is well-known that model can enhance the control performance in terms of precision or energy eciency, the practical appli- cation has often been limited by the complexities of manually obtaining suciently accurate models. In the past, learning has proven a viable al- ternative to using a combination of rigid-body dynamics and handcrafted approximations of nonlinearities. However, a major open question is what nonparametric learning method is suited best for learning dynamics? Tra- ditionally, locally weighted projection regression (LWPR), has been the standard method as it is capable of online, real-time learning for very com- plex robots. However, while LWPR has had signicant impact on learning in robotics, alternative nonparametric regression methods such as support vector regression (SVR) and Gaussian processes regression (GPR) oer interesting alternatives with fewer open parameters and potentially higher accuracy. In this paper, we evaluate these three alternatives for model learning. Our comparison consists out of the evaluation of learning qual- ity for each regression method using original data from SARCOS robot arm, as well as the robot tracking performance employing learned models. The results show that GPR and SVR achieve a superior learning precision and can be applied for real-time control obtaining higher accuracy. How- ever, for the online learning LWPR presents the better method due to its lower computational requirements.

A methodology for Building Regression Models using Extreme Learning Machine: OP-ELM

Abstract: This paper proposes a methodology named OP-ELM, based on a recent development –the Extreme Learning Machine– decreasing drastically the training speed of networks Variable selection is beforehand performed on the original dataset for proper results by OP-ELM: the network is first created using Extreme Learning Process, selection of the most relevant nodes is performed using Least Angle Regression (LARS) ranking of the nodes and a Leave-One-Out estimation of the performances Results are globally equivalent to LSSVM ones with reduced computational time

Mixtures of robust probabilistic principal component analyzers

Abstract: Mixtures of probabilistic principal component analyzers model high-dimensional nonlinear data by combining local linear models. Each mixture component is specifically designed to extract the local principal orientations in the data. An important issue with this generative model is its sensitivity to data lying off the low-dimensional manifold. In order to address this problem, the mixtures of robust probabilistic principal component analyzers are introduced. They take care of atypical points by means of a long tail distribution, the Student-t. It is shown that the resulting mixture model is an extension of the mixture of Gaussians, suitable for both robust clustering and dimensionality reduction. Finally, we briefly discuss how to construct a robust version of the closely related mixture of factor analyzers.

Using the Delta Test for Variable Selection

Abstract: Input selection is an important consideration in all large-scale modelling problems. We propose that using an established noise variance estimator known as the Delta test as the target to minimise can provide an effective input selection methodology. Theoretical justifications and experimental results are presented.

Rank-based quality assessment of nonlinear dimensionality reduction

Abstract: Nonlinear dimensionality reduction aims at providing low dimensional representations of high-dimensional data sets. Many new methods have been proposed in the recent years, but the question of their assessment and comparison remains open. This paper reviews some of the existing quality measures that are based on distance ranking and K-ary neighborhoods. Many quality criteria actually rely on the analysis of one or several sub-blocks of a co-ranking matrix. The analogy between the co-ranking matrix and a Shepard diagram is highlighted. Finally, a unifying framework is sketched, new measures are proposed and illustrated in a short experiment.

Similarities and differences between policy gradient methods and evolution strategies

Abstract: Natural policy gradient methods and the covariance matrix adaptation evolution strategy, two variable metric methods proposed for solving reinforcement learning tasks, are contrasted to point out their con- ceptual similarities and differences Experiments on the cart pole bench- mark are conducted as a first attempt to compare their performance Reinforcement learning (RL) algorithms search for a policy mapping states of the environment to (a probability distribution over) the actions an agent can take in those states The goal is to find a behavior such that some notion of future reward is maximized Direct policy search methods address this task by directly learning parameters of a function explicitly representing the policy Here we consider two general approaches to conduct direct policy search, namely policy gradient methods (PGMs) and evolution strategies (ESs) We will argue that these approaches are quite similar This makes it all the more surprising that so far there has been no systematic comparison of PGMs and ESs applied to the same test problems and operating on the same class of policies with the same parameterization This paper is our attempt to draw such a comparison, on a conceptual level and by conducting first empirical studies We restrict our consideration to the natural actor critic algorithm (NAC, (1, 2)) and the covari- ance matrix adaptation ES (CMA-ES, (3)), which are compared in the context of optimization in (4) We picked these two because they can be considered state- of-the-art, they are our favorite direct policy search method and evolutionary RL algorithm, respectively, and they are both variable metric methods In section 2 we briefly review the NAC algorithm and the CMA-ES Section 3 describes the conceptual relations of these two approaches and in section 4 we use a simple toy problem to compare the methods empirically

Clustering of Self-Organizing Map

Abstract: In this paper, we present a new similarity measure for a clus- tering self-organizing map which will be reached using a new approach of hierarchical clustering (1) The similarity measure is composed from two terms: weighted Ward distance and Euclidean distance weighted by neigh- bourhood function (2) An algorithm inspired from artificial ants named AntTree will be used to cluster a self-organizing map This algorithm has the advantage to provide a hierarchy of referents with a low complexity (near the nlog(n)) The SOM clustering including the new measure is validated on several public data bases

Survival SVM: a Practical Scalable Algorithm

Abstract: This work advances the Support Vector Machine (SVM) based approach for predictive modelling of failure time data as proposed in (1). The main results concern a drastic reduction in computation time, an improved criterion for model selection, and the use of additive models for improved interpretability in this context. Particular attention is given towards the influence of right-censoring in the methods. The approach is illustrated on a case-study in prostate cancer.

Learning topology of a labeled data set with the supervised generative Gaussian graph

Abstract: Extracting the topology of a set of a labeled data is expected to provide important information in order to analyze the data or to design a better decision system. In this work, we propose to extend the generative Gaussian graph to supervised learning in order to extract the topology of labeled data sets. The graph obtained learns the intra-class and inter-class connectedness and also the manifold-overlapping of the different classes. We propose a way to vizualize these topological features. We apply it to analyze the well-known Iris database and the three-phase pipe flow database.

Model-Based Reinforcement Learning with Continuous States and Actions

Abstract: , and Jan Peters1- University of Cambridge - Department of EngineeringTrumpington Street, Cambridge CB2 1PZ - UKfmpd37|cer54g@cam.ac.uk2- Max Planck Institute for Biological CyberneticsSpemannstraˇe 38, 72070 Tubingen - Germanyjan.peters@tuebingen.mpg.deAbstract.Finding an optimal policy in a reinforcement learning (RL) framework withcontinuous state and action spaces is challenging. Approximate solutionsare often inevitable. GPDP is an approximate dynamic programming algo-rithm based on Gaussian process (GP) models for the value functions. Inthis paper, we extend GPDP to the case of unknown transition dynamics.After building a GP model for the transition dynamics, we apply GPDPto this model and determine a continuous-valued policy in the entire statespace. We apply the resulting controller to the underpowered pendulumswing up. Moreover, we compare our results on this RL task to a nearlyoptimal discrete DP solution in a fully known environment.

Collaborative filtering with interlaced generalized linear models

Abstract: Collaborative filtering (CF) is a data analysis task appearing in many challenging applications, in particular data mining in Internet and e-commerce. CF can often be formulated as identifying patterns in a large and mostly empty rating matrix. In this paper, we focus on predicting unobserved ratings. This task is often a part of a recommendation procedure. We propose a new CF approach called interlaced generalized linear models (GLM); it is based on a factorization of the rating matrix and uses probabilistic modeling to represent uncertainty in the ratings. The advantage of this approach is that different configurations, encoding different intuitions about the rating process can easily be tested while keeping the same learning procedure. The GLM formulation is the keystone to derive an efficient learning procedure, applicable to large datasets. We illustrate the technique on three public domain datasets.

Self-Organizing Maps for cyclic and unbounded graphs.

Abstract: This paper introduces a new concept to the processing of graph structured information using self organising map framework. Previous approaches to this problem were limited to the processing of bounded graphs. The computational complexity of such methods grows rapidly with the level of connectivity, and are restricted to the processing of positional graphs. The concept proposed in this paper addresses these issues by reducing the computational demand, and by allowing the processing of non-positional graphs. This is achieved by utilising the state space of the self organising map instead of the states of the nodes in the graph for processing.

A method for robust variable selection with significance assessment

Abstract: Our goal is proposing an unbiased framework for gene expression analysis based on variable selection combined with a significance assessment step. We start by discussing the need of such a framework by illustrating the dramatic effect of a biased approach especially when the sample size is small. Then we describe our analysis protocol, based on two main ingredients. The first is a gene selection core, based on elastic net regularization where we explicitly take into account regularization parameter tuning. The second is a general architecture to assess the statistical significance of the model via cross validation and permutation testing. Finally we challenge the system on real data experiments, and study its performance when changing variable selection algorithm or the dataset size.

Learning Data Representations with Sparse Coding Neural Gas

Abstract: We consider the problem of learning an unknown (overcom- plete) basis from an unknown sparse linear combination. Introducing the "sparse coding neural gas" algorithm, we show how to employ a combina- tion of the original neural gas algorithm and Oja's rule in order to learn a simple sparse code that represents each training sample by a multiple of one basis vector. We generalise this algorithm using orthogonal matching pursuit in order to learn a sparse code where each training sample is rep- resented by a linear combination of k basis elements. We show that this method can be used to learn artificial sparse overcomplete codes.

Learning dynamics and robustness of vector quantization and neural gas

Abstract: Various alternatives have been developed to improve the winner-takes-all (WTA) mechanism in vector quantization, including the neural gas (NG). However, the behavior of these algorithms including their learning dynamics, robustness with respect to initialization, asymptotic results, etc. has only partially been studied in a rigorous mathematical analysis. The theory of on-line learning allows for an exact mathematical description of the training dynamics in model situations. We demonstrate using a system of three competing prototypes trained from a mixture of Gaussian clusters that the NG can improve convergence speed and achieves robustness to initial conditions. However, depending on the structure of the data, the NG does not always obtain the best asymptotic quantization error.

Machine learning approches and pattern recognition for spectral data.

Abstract: The adaptive and automated analysis of spectral data plays an important role in many areas of research such as physics, astronomy and geophysics, chemistry, bioinformatics, biochemistry, engineering, and others. The amount of data may range from several billion samples in geophysics to only a few in medical applications. Further, a vectorial rep- resentation of spectra typically leads to huge-dimensional problems. This scenario gives the background for particular requirements of respective machine learning approaches which will be the focus of this overview.

Generalized Matrix Learning Vector Quantizer for the Analysis of Spectral Data

Abstract: The analysis of spectral data constitutes new challenges for machine learning algorithms due to the functional nature of the data. Special attention is paid to the metric used in the analysis. Recently, a prototype based algorithm has been proposed which allows the integration of a full adaptive matrix in the metric. In this contribution we study this approach with respect to band matrices and its use for the analysis of functional spectral data. The method is tested on data taken from food chemistry and satellite image data.

Detecting zebra crossings utilizing AdaBoost

Abstract: 1- University Hospital Ulm - Department of Internal Medicine IRobert Koch Str. 8, 89069 Ulm - Germany2- University of Ulm - Dept of Neural Information Processing89069 Ulm - GermanyAbstract. This paper introduces a visual zebra crossing detector basedon the Viola-Jones approach. The basic properties of this cascaded clas-sifier and the use of integral images are explained. Additional pre- andpostprocessing for this task are introduced and evaluated.

Machine Learning in cancer research: implications for personalised medicine

Abstract: Driven by the growing demand of personalization of medical procedures, data-based, computer-aided cancer research in human patients is advancing at an accelerating pace, providing a broadening landscape of opportunity for Machine Learning methods. This landscape can be observed from the wide-reaching view of population studies down to the genotype detail. In this brief paper, we provide a sweeping glimpse, by no means exhaustive, of the state-of-the-art in this field at the different scales of data measurement and analysis.

A Method for Time Series Prediction using a Combination of Linear Models

Abstract: This paper presents a new approach for time series prediction using local dynamic modeling. The proposed method is composed of three blocks: a Time Delay Line that transforms the original time series into a set of Ndimensional vectors, an Information-Theoretic based clustering method that segments the previous set into subspaces of similar vectors and a set of single layer neural networks that adjust a local model for each subspace created by the clustering stage. The results of this model are compared with those of another local modeling approach and of two representative global models in time series prediction: Tapped Delay Line Multilayer Perceptron (TDL-MLP) and Support Vector Regression (SVR).

Handling almost-deterministic relationships in constraint-based Bayesian network discovery : Application to cancer risk factor identification.

Abstract: In this paper, we discuss simple methods for identification and handling of almost-deterministic relationships (ADR) in automatic constraint-based Bayesian network structure discovery. The problem with ADR is that conditional independence tests become unreliable when the conditional set almost-determine one of the variables in the test. Such errors have usually a cascading effect that causes many errors in the final graph. Several methods for identification and handling of ADR are discussed to provide insight into their advantages and disadvantages. The methods are applied on standard benchmarks to recover the original structure from data in order to assess their capabilities. We then discuss efforts to apply ours findings to Nasopharyngeal Carcinoma (NPC) survey data. The aim is to help identify the important risk factors involved in the NPC cancer.

Visual focus with spiking neurons

Abstract: 1- Universit´e Paris-Sud XIOrsay - France2- LIMSI - CNRS UPR 3251Orsay - France3- Ecole Normale Sup´´ erieureParis - France{sylvain.chevallier,philippe.tarroux}@limsi.frAbstract. Attentional focusing can be implemented with a neural field[1], which uses a discharge rate code. As an alternative, we propose inthe present work an implementation based on spiking neurons. Such im-plementation will allow to investigate the possible contribution of a spiketime based code with a network of leaky integrate-and-fire neurons. Thenetwork is able to detect and to focus on a stimulus even in the presenceof distractors. Experimental data show that this behavior is very robustto noise. This process implements an early visual attention mechanism.

An Accelerated MDM Algorithm for SVM Training

Abstract: In this work we will propose an acceleration procedure for the Mitchell{Demyanov{Malozemov (MDM) algorithm (a fast geometric algo- rithm for SVM construction) that may yield quite large training savings. While decomposition algorithms such as SVMLight or SMO are usually the SVM methods of choice, we shall show that there is a relationship between SMO and MDM that suggests that, at least in their simplest implemen- tations, they should have similar training speeds. Thus, and although we will not discuss it here, the proposed MDM acceleration might be used as a starting point to new ways of accelerating SMO.

Initialization Mechanism in Kohonen Neural Network Implemented in CMOS Technology

Abstract: An initialization mechanism is presented for Kohonen neural network implemented in CMOS technology. Proper selection of initial values of neurons’ weights has a large influence on speed of the learning algorithm and finally on the quantization error of the network, which for different initial parameters can vary even by several orders of magnitude. Experiments with the software model of designed network show that results can be additionally improved when conscience mechanism is used during the learning phase. This mechanism additionally decreases number of dead neurons, which minimizes the quantization error. The initialization mechanism together with experimental Kohonen neural network with four neurons and 3 inputs have been designed in CMOS 0.18 μm technology.

K-Nearest Neighbours based on Mutual Information for Incomplete Data Classification

Abstract: Incomplete data is a common drawback that machine learning techniques need to deal with when solving real-life classification tasks. One of the most popular procedures for solving this kind of problems is the K-nearest neighbours (KNN) algorithm. In this paper, we present a weighted KNN approach using mutual information to impute and classify incomplete input data. Numerical results on both artificial and real data are given to demonstrate the effectiveness of the proposed method.

An Emphasized Target Smoothing Procedure to Improve MLP Classifiers Performance

Abstract: Standard learning procedures are better fitted to estimation than to classification problems, and focusing the training on appropriate samples provides performance advantages in classification tasks. In this paper, we combine these ideas creating smooth targets for classification by means of a convex combination of the original target and the output of an auxiliary classifier, the combination parameter being a function of the auxiliary classifier error. Experimental results with Multilayer Perceptron architectures support the usefulness of this approach.

Pruning and Regularisation in Reservoir Computing: a First Insight

Abstract: Reservoir Computing is a new paradigm for using Recurrent Neural Networks which shows promising results. However, as the recurrent part is created randomly, it typically needs to be large enough to be able to capture the dynamic features of the data considered. Moreover, this random creation is still lacking a strong methodology. We propose to study how pruning some connections from the reservoir to the readout can help on the one hand to increase the generalisation ability, in much the same way as regularisation techniques do, and on the other hand to improve the implementability of reservoirs in hardware. Furthermore we study the actual sub-reservoir which is kept after pruning which leads to important insights in what we have to expect from a good reservoir.

Regularization path for Ranking SVM.

Abstract: Ranking algorithms are often introduced with the aim of auto- matically personalising search results. However, most ranking algorithms developed in the machine learning community rely on a careful choice of some regularisation parameter. Building upon work on the regularisation path for kernel methods, we propose a parameter selection algorithm for ranking SVM. Empirical results are promising.