Learning and evolution are two fundamental forms of adaptation. There has been a great interest in combining learning and evolution with artificial neural networks (ANNs) in recent years. This paper: 1) reviews different combinations between ANNs and evolutionary algorithms (EAs), including using EAs to evolve ANN connection weights, architectures, learning rules, and input features; 2) discusses different search operators which have been used in various EAs; and 3) points out possible future research directions. It is shown, through a considerably large literature review, that combinations between ANNs and EAs can lead to significantly better intelligent systems than relying on ANNs or EAs alone.

/pdf/evolving-artificial-neural-networks-4kzrbu7pkn.pdf

Evolving artificial neural networks

The Nature of Remote Sensing: Introduction. Remote Sensing. Information Extraction from Remote-Sensing Images. Spectral Factors in Remote Sensing. Spectral Signatures. Remote-Sensing Systems. Optical Sensors. Temporal Characteristics. Image Display Systems. Data Systems. Summary. Exercises. References. Optical Radiation Models: Introduction. Visible to Short Wave Infrared Region. Solar Radiation. Radiation Components. Surface-Reflected. Unscattered Component. Surface-Reflected. Atmosphere-Scattered Component. Path-Scattered Component. Total At-Sensor. Solar Radiance. Image Examples in the Solar Region. Terrain Shading. Shadowing. Atmospheric Correction. Midwave to Thermal Infrared Region. Thermal Radiation. Radiation Components. Surface-Emitted Component. Surface-Reflected. Atmosphere-Emitted Component. Path-Emitted Component. Total At-Sensor. Emitted Radiance. Total Solar and Thermal Upwelling Radiance. Image Examples in the Thermal Region. Summary. Exercises. References. Sensor Models: Introduction. Overall Sensor Model. Resolution. The Instrument Response. Spatial Resolution. Spectral Resolution. Spectral Response. Spatial Response. Optical PSFopt. Image Motion PSFIM. Detector PSFdet. Electronics PSFel. Net PSFnet. Comparison of Sensor PSFs. PSF Summary for TM. Imaging System Simulation. Amplification. Sampling and Quantization. Simplified Sensor Model. Geometric Distortion. Orbit Models. Platform Attitude Models. Scanner Models. Earth Model. Line and Whiskbroom ScanGeometry. Pushbroom Scan Geometry. Topographic Distortion. Summary. Exercises. References. Data Models: Introduction. A Word on Notation. Univariate Image Statistics. Histogram. Normal Distribution. Cumulative Histogram. Statistical Parameters. Multivariate Image Statistics. Reduction to Univariate Statistics. Noise Models. Statistical Measures of Image Quality. Contrast. Modulation. Signal-to-Noise Ratio (SNR). Noise Equivalent Signal. Spatial Statistics. Visualization of Spatial Covariance. Covariance with Semivariogram. Separability and Anisotropy. Power Spectral Density. Co-occurrence Matrix. Fractal Geometry. Topographic and Sensor Effects. Topography and Spectral Statistics. Sensor Characteristics and Spectral Stastistics. Sensor Characteristics and Spectral Scattergrams. Summary. Exercises. References. Spectral Transforms: Introduction. Feature Space. Multispectral Ratios. Vegetation Indexes. Image Examples. Principal Components. Standardized Principal Components (SPC) Transform. Maximum Noise Fraction (MNF) Transform. Tasseled Cap Tranformation. Contrast Enhancement. Transformations Based on Global Statistics. Linear Transformations. Nonlinear Transformations. Normalization Stretch. Reference Stretch. Thresholding. Adaptive Transformation. Color Image Contrast Enhancement. Min-max Stretch. Normalization Stretch. Decorrelation Stretch. Color Spacer Transformations. Summary. Exercises. References. Spatial Transforms: Introduction. An Image Model for Spatial Filtering. Convolution Filters. Low Pass and High Pass Filters. High Boost Filters. Directional Filters. The Border Region. Characterization of Filtered Images. The Box Filter Algorithm. Cascaded Linear Filters. Statistical Filters. Gradient Filters. Fourier Synthesis. Discrete Fourier Transforms in 2-D. The Fourier Components. Filtering with the Fourier Transform. Transfer Functions. The Power Spectrum. Scale Space Transforms. Image Resolution Pyramids. Zero-Crossing Filters. Laplacian-of-Gaussian (LoG) Filters. Difference-of-Gaussians (DoG) Filters.Wavelet Transforms. Summary. Exercises. References. Correction and Calibration: Introduction. Noise Correction. Global Noise. Sigma Filter. Nagao-Matsuyama Filter. Local Noise. Periodic Noise. Distriping 359. Global,Linear Detector Matching. Nonlinear Detector Matching. Statistical Modification to Linear and Nonlinear Detector. Matching. Spatial Filtering Approaches. Radiometric Calibration. Sensor Calibration. Atmospheric Correction. Solar and Topographic Correction. Image Examples. Calibration and Normalization of Hyperspectral Imagery. AVIRIS Examples. Distortion Correction. Polynomial Distortion Models. Ground Control Points (GCPs). Coordinate Transformation. Map Projections. Resampling. Summary. Exercises References. Registration and Image Fusion: Introduction. What is Registration? Automated GCP Location. Area Correlation. Other Spatial Features. Orthrectification. Low-Resolution DEM. High-Resolution DEM. Hierarchical Warp Stereo. Multi-Image Fusion. Spatial Domain Fusion. High Frequency Modulation. Spectral Domain Fusion. Fusion Image Examples. Summary. Exercises. References. Thematic Classification: Introduction. The Importance of Image Scale. The Notion of Similarity. Hard Versus Soft Classification. Training the Classifier. Supervised Training. Unsupervised Training. K-Means Clustering Algorithm. Clustering Examples. Hybrid Supervised/Unsupervised Training. Non-Parametric Classification Algorithms. Level-Slice. Nearest-Mean. Artificial Neural Networks (ANNs). Back-Propagation Algorithm. Nonparametric Classification Examples. Parametric Classification Algorithms. Estimation of Model-Parameters. Discriminant Functions. The Normal Distribution Model. Relation to the Nearest-Mean Classifier. Supervised Classification Examples and Comparison to Nonparametric Classifiers. Segmentation. Region Growing. Region Labeling. Sub-Pixel Classification. The Linear Mixing Model. Unmixing Model. Hyperspectral Image Analysis. Visualization of the Image Cube. Feature Extraction. Image Residuals. Pre-Classification Processing and Feature Extraction. Classification Algorithms. Exercises. Error Analysis. Multitemporal Images. Summary. References. Index.

Remote sensing, models, and methods for image processing

This paper informs a statistical readership about Artificial Neural Networks (ANNs), points out some of the links with statistical methodology and encourages cross-disciplinary research in the directions most likely to bear fruit. The areas of statistical interest are briefly outlined, and a series of examples indicates the flavor of ANN models. We then treat various topics in more depth. In each case, we describe the neural network architectures and training rules and provide a statistical commentary. The topics treated in this way are perceptrons (from single-unit to multilayer versions), Hopfield-type recurrent networks (including probabilistic versions strongly related to statistical physics and Gibbs distributions) and associative memory networks trained by so-called unsupervised learning rules. Perceptrons are shown to have strong associations with discriminant analysis and regression, and unsupervized networks with cluster analysis. The paper concludes with some thoughts on the future of the interface between neural networks and statistics.

/pdf/neural-networks-a-review-from-a-statistical-perspective-1re2v41n6l.pdf

Neural Networks: A Review from a Statistical Perspective

Image processing with neural networks–a review

Neural networks and fuzzy systems

In pattern recognition, it is often necessary to deal with problems to classify a transformed pattern. A neural pattern recognition system which is insensitive to rotation of input pattern by various degrees is proposed. The system consists of a fixed invariance network with many slabs and a trainable multilayered network. The system was used in a rotation-invariant coin recognition problem to distinguish between a 500 yen coin and a 500 won coin. The results show that the approach works well for variable rotation pattern recognition. >

Rotation-invariant neural pattern recognition system with application to coin recognition

Part 1 Mathematical theory: some basic results in the theory of partial differential equations - Bellman-Gronwall inequality, Sobolev spaces, Green's formula stochastic partial differential equations - radon measures, cylindrical probability, Gaussian cylindrical probability, nuclear and Hilbert-Schmidt operators, conditional expectation, Hilbert- space-valued Wiener processes optimal control of deterministic distributed parameter systems - elliptic systems, the Dirichlet problem, the Neumann problem, parabolic systems, Riccati equation, Hamilton-Jacobi equation, hyperbolic systems controllability and observability linear estimation theory - finite-dimensional estimation theory, estimation for random linear functionals optimal filter for distributed parameter systems - the filtering problems, Wiener filter, Kalman-Bucy filter, recursive formula for the optimal filter, innovation theory, duality between estimation and control, optimal filter for hyperbolic systems stochastic optimal control of distributed parameter systems formulation of the model, the stochastic optimal control problem, necessary and sufficient conditions for optimality, the separation principle identification of distributed parameter systems - the basic concept of system identification, modal approximation for identification, regularization. Part 2 Engineering Applications: formal approach to optimal filtering and control of distributed parameter systems - Wiener-Hopf theorem, the optimal filter, predictor and smoothing estimator, various approaches to linear estimation problems stochastic optimal control problems optimal sensor and actuator location problems - optimal sensor location problems, optimal actuator locations computational techniques for identification of distributed parameter systems - stochastic approximation, least squares identification, the Galerkin finite-element model, discrete regularization and minimization.

Distributed parameter systems: theory and applications

A pattern classification method is proposed for remote sensing data using neural networks. First, the authors apply the error backpropagation (BP) algorithm to classify the remote sensing data. In this case, the classification performance depends on a training data set. In order to get stable and precise classification results, the training data set is selected based on geographical information and Kohonen's self-organizing feature map. Using the training data set and the error backpropagation algorithm, a layered neural network is trained such that the training patterns are classified with a specified accuracy. After training the neural network, some pixels are deleted from the original training data set if they are incorrectly classified and a new training data set is built up. Once training is complete, a testing data set is classified by using the trained neural network. The classification results of LANDSAT TM data show that this approach produces excellent results which are more realistic and noiseless compared with a conventional Bayesian method. >

Neural network approach to land cover mapping

A bill-recognition apparatus includes a neural network having a learning capability and performs high-efficiency pattern recognition of seven kinds of U.S. dollar bills. Pattern image data optically inputted through a sensor is compressed using plurality of column masks, and then a plurality of values representative of images (slab values) are determined. The image data is divided into a large number of strip-shaped segments, and some of theses segments are masked with column areas of masks. The values representative of images compressed through column masks are not influenced by a slight inclination of the pattern image during the reading operation. These values representative of images are inputted to a separation processing unit (neural network). From these values, the separation processing unit calculates separation values corresponding to respective decision patterns associated with pattern images, using weights which have been adjusted to optimum values for respective decision patterns. A correct pattern image is determined from the maximum value of the separation values. The above arrangement allows for a reduction in scale of the neural network and control system. Furthermore, bill recognition may also be achieved by separation processing using a plurality of small-scaled neural networks connected in cascade, or replacing weight functions in the same neural network and performing separation processing a plurality of times for the same slab values (cascade processing). In this way, it is possible to reduce the scale of the neural network and the control system.

Sigeru Omatu

Papers

Rotation-invariant neural pattern recognition system with application to coin recognition

Distributed parameter systems: theory and applications

Neural network approach to land cover mapping

Neural network approach to land cover mapping

Pattern recognition apparatus and method of optimizing mask for pattern recognition according to genetic algorithm