https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

The self-organizing map (SOM) is an excellent tool in exploratory phase of data mining. It projects input space on prototypes of a low-dimensional regular grid that can be effectively utilized to visualize and explore properties of the data. When the number of SOM units is large, to facilitate quantitative analysis of the map and the data, similar units need to be grouped, i.e., clustered. In this paper, different approaches to clustering of the SOM are considered. In particular, the use of hierarchical agglomerative clustering and partitive clustering using K-means are investigated. The two-stage procedure-first using SOM to produce the prototypes that are then clustered in the second stage-is found to perform well when compared with direct clustering of the data and to reduce the computation time.

/pdf/clustering-of-the-self-organizing-map-1na7a1h6p3.pdf

Clustering of the self-organizing map

/pdf/neural-networks-and-deep-learning-3ccs3ain8p.pdf

Neural Networks and Deep Learning

Continual lifelong learning with neural networks: A review.

An incremental network model is introduced which is able to learn the important topological relations in a given set of input vectors by means of a simple Hebb-like learning rule. In contrast to previous approaches like the "neural gas" method of Martinetz and Schulten (1991, 1994), this model has no parameters which change over time and is able to continue learning, adding units and connections, until a performance criterion has been met. Applications of the model include vector quantization, clustering, and interpolation.

/pdf/a-growing-neural-gas-network-learns-topologies-2df9a64rcg.pdf

A Growing Neural Gas Network Learns Topologies

http://www.wi.hs-wismar.de/~cleve/vorl/projects/dm/ss13/SOTA/quellen/1-s2.0-0893608094900914-main.pdf

Growing cell structures—a self-organizing network for unsupervised and supervised learning

We present a novel self-organizing network which is generated by a growth process. The application range of the model is the same as for Kohonen’s feature map: generation of topology-preserving and dimensionality-reducing mappings, e.g., for the purpose of data visualization. The network structure is a rectangular grid which, however, increases its size during self-organization. By inserting complete rows or columns of units the grid may adapt its height/width ratio to the given pattern distribution. Both the neighborhood range used to co-adapt units in the vicinity of the winning unit and the adaptation strength are constant during the growth phase. This makes it possible to let the network grow until an application-specific performance criterion is fulfilled or until a desired network size is reached. A final approximation phase with decaying adaptation strength finetunes the network.

Growing Grid — a self-organizing network with constant neighborhood range and adaptation strength

A new on-line criterion for identifying “useless” neurons of a self-organizing network is proposed. When this criterion is used in the context of the (formerly developed) growing neural gas model to guide deletions of units, the resulting method is able to closely track nonstationary distributions. Slow changes of the distribution are handled by adaptation of existing units. Rapid changes are handled by removal of “useless” neurons and subsequent insertions of new units in other places.

A Self-Organizing Network that Can Follow Non-stationary Distributions

We present a new algorithm for the construction of radial basis function (RBF) networks. The method uses accumulated error information to determine where to insert new units. The diameter of the localized units is chosen based on the mutual distances of the units. To have the distance information always available, it is held up-to-date by a Hebbian learning rule adapted from the “Neural Gas≓ algorithm. The new method has several advantages over existing methods and is able to generate small, well-generalizing networks with comparably few sweeps through the training data.

Bernd Fritzke

Papers

A Growing Neural Gas Network Learns Topologies

Growing cell structures—a self-organizing network for unsupervised and supervised learning

Growing Grid — a self-organizing network with constant neighborhood range and adaptation strength

A Self-Organizing Network that Can Follow Non-stationary Distributions

Fast learning with incremental RBF networks