Showing papers on "Convolutional neural network published in 2003"

Best practices for convolutional neural networks applied to visual document analysis

Abstract: Neural networks are a powerful technology forclassification of visual inputs arising from documents.However, there is a confusing plethora of different neuralnetwork methods that are used in the literature and inindustry. This paper describes a set of concrete bestpractices that document analysis researchers can use toget good results with neural networks. The mostimportant practice is getting a training set as large aspossible: we expand the training set by adding a newform of distorted data. The next most important practiceis that convolutional neural networks are better suited forvisual document tasks than fully connected networks. Wepropose that a simple "do-it-yourself" implementation ofconvolution with a flexible architecture is suitable formany visual document problems. This simpleconvolutional neural network does not require complexmethods, such as momentum, weight decay, structure-dependentlearning rates, averaging layers, tangent prop,or even finely-tuning the architecture. The end result is avery simple yet general architecture which can yieldstate-of-the-art performance for document analysis. Weillustrate our claims on the MNIST set of English digitimages.

Subject independent facial expression recognition with robust face detection using a convolutional neural network

Abstract: Reliable detection of ordinary facial expressions (e.g. smile) despite the variability among individuals as well as face appearance is an important step toward the realization of perceptual user interface with autonomous perception of persons. We describe a rule-based algorithm for robust facial expression recognition combined with robust face detection using a convolutional neural network. In this study, we address the problem of subject independence as well as translation, rotation, and scale invariance in the recognition of facial expression. The result shows reliable detection of smiles with recognition rate of 97.6% for 5600 still images of more than 10 subjects. The proposed algorithm demonstrated the ability to discriminate smiling from talking based on the saliency score obtained from voting visual cues. To the best of our knowledge, it is the first facial expression recognition model with the property of subject independence combined with robustness to variability in facial appearance.

Convolutional Neural Networks for Image Processing: An Application in Robot Vision

Abstract: Convolutional neural networks (CNNs) represent an interesting method for adaptive image processing, and form a link between general feed-forward neural networks and adaptive filters. Two dimensional CNNs are formed by one or more layers of two dimensional filters, with possible non-linear activation functions and/or down-sampling. CNNs possess key properties of translation invariance and spatially local connections (receptive fields). We present a description of the convolutional network architecture, and an application to practical image processing on a mobile robot. A CNN is used to detect and characterize cracks on an autonomous sewer inspection robot. The filter sizes used in all cases were 4x4, with non-linear activations between each layer. The number of feature maps used in the three hidden layers was, from input to output, 4, 4, 4. The network was trained using a dataset of 48x48 sub-regions drawn from 30 still image 320x240 pixel frames sampled from a pre-recorded sewer pipe inspection video. 15 frames were used for training and 15 for validation of network performance. Although development of a CNN system for civil use is on-going, the results support the notion that data-based adaptive image processing methods such as CNNs are useful for image processing, or other applications where the input arrays are large, and spatially / temporally distributed. Further refinements of the CNN architecture, such as the implementation of separable filters, or extensions to three dimensional (ie. video) processing, are suggested.

System and method facilitating pattern recognition

Abstract: A system and method facilitating pattern recognition is provided. The invention includes a pattern recognition system having a convolutional neural network employing feature extraction layer(s) and classifier layer(s). The feature extraction layer(s) comprises convolutional layers and the classifier layer(s) comprises fully connected layers. The pattern recognition system can be trained utilizing a calculated cross entropy error. The calculated cross entropy error is utilized to update trainable parameters of the pattern recognition system.

A new class of convolutional neural networks (SICoNNets) and their application of face detection

Abstract: Artificial neural networks (ANNs), evolved from biological insights, have equipped computers with the capacity to actually learn from examples using real world data. With this remarkable ability, ANNs are able to extract patterns and detect trends that are too complex to be noticed or perceived by either humans or classical computer techniques. Nevertheless, as the amount of data to be processed increases significantly there is a demand for developing other types of artificial neural networks to perform complex pattern recognition tasks. In this article, a new class of convolutional neural networks, namely shunting inhibitory convolutional neural networks (SICoNNets), is introduced, and a training algorithm is developed using supervised learning based on resilient backpropagation with momentum. Three different network topologies, ranging from fully-connected to partially-connected, are implemented and trained to discriminate between face and non-face patterns. All three architectures achieve more than 96% correct face classification; the best architecture achieves 97.6% correct face classification at a false alarm rate of 3.4%.

Artificial neural nets problem solving methods

Abstract: Nets Design.- FPGA Implementation of a Perceptron-like Neural Network for Embedded Applications.- NSP: a Neuro-Symbolic Processor.- Reconfigurable Hardware Architecture for Compact and Efficient Stochastic Neuron.- Current mode CMOS synthesis of a motor-control neural system.- New emulated discrete model of CNN architecture for FPGA and DSP applications.- A Binary Multiplier Using RTD Based Threshold Logic Gates.- Split-Precharge Differential Noise-Immune Threshold Logic Gate (SPD-NTL).- UV-programmable Floating-Gate CMOS Linear Threshold Element "P1N3".- CMOS Implementation of Generalized Threshold Functions.- A-DELTA: A 64-bit High Speed, Compact, Hybrid Dynamic-CMOS/Threshold-Logic Adder.- Validation of a Cortical Electrode Model for Neuroprosthetics purposes.- XMLP: a Feed-Forward Neural Network with Two-Dimensional Layers and Partial Connectivity.- An analogue current-mode hardware design proposal for preprocessing layers in ART-based neural networks.- On the effects of dimensionality on data analysis with neural networks.- Hardware Optimization of a Novel Spiking Neuron Model for the POEtic tissue..- Implementing a Margolus Neighborhood Cellular Automata on a FPGA.- An Empirical Comparison of Training Algorithms for Radial Basis Functions.- Ensemble Methods for Multilayer Feedforward: An Experimental Study.- Post-synaptic Time-dependent Conductances in Spiking Neurons: FPGA Implementation of a Flexible Cell Model.- Applications in Robotics.- A Recurrent Neural Network for Robotic Sensory-based Search.- The Knowledge Engineering approach to Autonomous Robotics.- Multimodule Artificial Neural Network Architectures for Autonomous Robot Control Through Behavior Modulation.- Solving the Inverse Kinematics in Humanoid Robots: A Neural Approach.- Sensory-motor control scheme based on Kohonen Maps and AVITE model.- Validation of Features for Characterizing Robot Grasps.- Self-Organizing Maps versus Growing Neural Gas in a Robotic Application.- Towards reactive navigation and attention skills for 3D intelligent characters.- From Continuous Behaviour to Discrete Knowledge.- Sources Separation.- Initialisation of Nonlinearities for PNL and Wiener systems Inversion.- Evolutionary Algorithm Using Mutual Information for Independent Component Analysis.- Blind separation of linear-quadratic mixtures of real sources using a recurrent structure.- Advances in Neyman-Pearson Neural Detectors Design.- A novel unsupervised strategy to separate convolutive mixtures in the frequency domain.- An improved geometric overcomplete blind source separation algorithm.- Application of Independent Component Analysis to Edge Detection and Watermarking.- A new Geometrical ICA-based method for Blind Separation of Speech Signals..- A time-frequency blind source separation method based on segmented coherence function.- ISFET Source Separation based on linear ICA.- An application of ICA to blind DS-CDMA detection: a joint optimization criterion.- Genetics Algorithms.- A Genetic Algorithm for Controlling Elevator Group Systems.- Protein Structure Prediction Using Evolutionary Algorithms Hybridized with Backtracking.- Hybridizing a Genetic Algorithm with Local Search and Heuristic Seeding.- A Genetic Algorithm for Assembly Sequence Planning.- Genetic Algorithm applied to Paroxysmal Atrial Fibrillation Prediction.- Optimizing supply strategies in the Spanish Electrical Market.- Improving the Efficiency of Multiple Sequence Alignment by Genetic Algorithms.- A real application example of a control structure selection by means of a multiobjective genetic algorithm.- Weighting and Feature Selection on Gene-Expression data by the use of Genetic Algorithms.- Supervised Segmentation of the Cervical Cell Images by using the Genetic Algorithms.- Using Genetic Algorithms for solving partitioning problem in codesign.- Soft-Computing.- Neuro-Fuzzy Modeling Applied to GIS: a Case Study for Solar Radiation.- Evolutionary Multi-Model Estimators for ARMA System Modeling and Time Series Prediction.- Real-Coded GA for Parameter Optimization in Short-Term Load Forecasting.- Parallel Computation of an Adaptive Optimal RBF Network Predictor.- New Method for Filtered ICA Signals Applied To Volatile Time Series..- Robust Estimation of Confidence Interval in Neural Networks applied to Time Series.- Modelling the HIV-AIDS Cuban Epidemics with Hopfield Neural Networks.- Comparison of Neural Models, Off-line and On-line Learning Algorithms for a Benchmark Problem.- Using Neural Networks in a Parallel Adaptative Algorithm for the System Identification Optimization.- Nonlinear Parametric Model Identification using Genetic Algorithms.- Input-Output Fuzzy Identification of Nonlinear Multivariable Systems. Application to a Case of AIDS Spread Forecast.- Recovering Missing Data with Functional and Bayesian Networks.- Estimation of train speed via neuro-fuzzy techniques.- Neuro-Fuzzy Techniques for Image Tracking.- Images.- A Comparative Study of Fuzzy Classifiers on Breast Cancer Data.- A New Information Measure for Natural Images.- Defects Detection in Continuous Manufacturing by means of Convolutional Neural Networks.- Removal of Impulse Noise in Images by Means of the Use of Support Vector Machines.- Recognizing Images from ICA Filters and Neural Network Ensembles with Rule Extraction.- Independent Component Analysis for Cloud Screening of Meteosat Images.- Neural Solutions for High Range Resolution Radar Classification.- On the application of Associative Morphological Memories to Hyperspectral Image Analysis.- A Generalized Eigendecomposition Approach using Matrix Pencils to remove artefacts from 2D NMR Spectra.- Medical Applications.- Feature Vectors Generation for Detection of Microcalcifications in Digitized Mammography Using Neural Networks.- Simulation of the Neuronal Regulator of the Lower Urinary Tract using a Multiagent System.- Neural network modeling of ambulatory systolic blood pressure for hypertension diagnosis.- Multiple MLP Neural Networks Applied on the Determination of Segment Limits in ECG Signals.- Acoustic Features Analysis for Recognition of Normal and Hypoacustic Infant Cry Based on Neural Networks.- A Back Propagation Neural Network for Localizing Abnormal Cortical Regions in FDG PET images in Epileptic Children.- Other Applications.- ANN based tools in Astrophysics. Prospects and first results for GOA and the AVO.- An Artificial Neural Network Approach to Automatic Classification of Stellar Spectra.- Non Linear Process Identification Using a Neural Network Based Multiple Models Generator.- Application of HLVQ and G-Prop Neural Networks to the Problem of Bankruptcy Prediction.- Improved AURA k-Nearest Neighbour Approach.- Non-Linear Speech coding with MLP, RBF and Elman based prediction1.- An Independent Component Analysis Evolution Based Method for Nonlinear Speech Processing.- Generalizing Geometric ICA to Nonlinear Settings.- An Adaptive Approach to Blind Source Separation Using a Self-Organzing Map and a Neural Gas.- METAL A: a Distributed System for Web Usage Mining.- Web Meta-search using Unsupervised Neural Networks.- Virtual Labs for Neural Networks E-courses.- MISTRAL: A Knowledge-Based System for Distance Education that Incorporates Neural Networks Techniques for Teaching Decisions.- A recurrent neural network model for the p-hub problem.- A comparison of the performance of SVM and ARNI on Text Categorization with new filtering measures on an unbalanced collection.- Neural Networks & Antennas Design: an Application for Avoiding Interferences.- Feedback Linearization Using Neural Networks: Application to an Electromechanical Process.- Automatic Size Determination of Codifications for the Vocabularies of the RECONTRA Connectionist Translator*.- Integrating Ensemble of Intelligent Systems for Modeling Stock Indices.- Resolution of joint maintenance/production scheduling by sequential and integrated strategies.- MLP and RBFN for detecting white gaussian signals in white gaussian interference.- Feature reduction using Support Vector Machines for binary gas detection.- Artificial Neural Networks Applications for Total Ozone Time Series.

System and method for mode recognising

Abstract: A system and method facilitating pattern recognition is provided. The invention includes a pattern recognition system having a convolutional neural network employing feature extraction layer(s) and classifier layer(s). The feature extraction layer(s) comprises convolutional layers and the classifier layer(s) comprises fully connected layers. The pattern recognition system can be trained utilizing a calculated cross entropy error. The calculated cross entropy error is utilized to update trainable parameters of the pattern recognition system.

An introduction to bio-inspired artificial neural network architectures.

Abstract: In this introduction to artificial neural networks we attempt to give an overview of the most important types of neural networks employed in engineering and explain shortly how they operate and also how they relate to biological neural networks. The focus will mainly be on bio-inspired artificial neural network architectures and specifically to neo-perceptions. The latter belong to the family of convolutional neural networks. Their topology is somewhat similar to the one of the human visual cortex and they are based on receptive fields that allow, in combination with sub-sampling layers, for an improved robustness with regard to local spatial distortions. We demonstrate the application of artificial neural networks to face analysis--a domain we human beings are particularly good at, yet which poses great difficulties for digital computers running deterministic software programs.

A generalised framework for convolutional decoding using a recurrent neural network

Abstract: This paper introduces a model of the conventional convolutional coding system based on representing encoder outputs as n-dimensional vectors in Euclidean space. Previously, it has been shown that the gradient descent algorithm can be used for bit decoding at the receiver, and can be implemented using a recurrent neural network (RNN). In this paper we generalise the mathematical framework for the general rate 1/n encoder. Our simulation results confirm that the RNN decoder is capable of performing very close to the Viterbi decoder, and has been found here to work extremely well for some simple convolutional codes.

Target detection in sea clutter using convolutional neural networks

Abstract: A detector based on convolutional neural networks is proposed for radar detection of floating targets in highly complex and nonstationary cluttered environments. This detector is coherent and monocell, i.e. it works with the complex envelope of the echoes from the same range cell. It includes a pre-processing time-frequency block implemented by the Wigner-Ville distribution, which provides a constant false alarm rate (CFAR) behavior regarding the clutter power when normalization is utilized. Simple theoretical models for the clutter and targets were allowed to study the impact of the correlation and Doppler of both target and clutter on its performance. This detector has also been tested with real-life sea clutter with an improved performance compared to classic detectors.

Facial expression recognition combined with robust face detection in a convolutional neural network

Abstract: Reliable detection of ordinary facial expressions (e.g., smile) despite the variability among individuals as well as face appearance is an important step toward the realization of perceptual user interface and the next generation imaging system with autonomous perception of persons. We describe a robust facial expression recognition system using the result of face detection by a convolutional neural network and rule-based processing. In this study, we address the problem of subject independence as well as translation, rotation, and scale invariance in the recognition of facial expression. The result shows reliable detection of smiles with recognition rate of 97.6% for 5600 still images of more than 10 subjects. The proposed algorithm demonstrated the ability to discriminate smiling from talking based on the saliency score in the proposed algorithm. To the best of our knowledge, it is the first facial expression recognition model with the property of subject independence combined with robustness to variability in facial appearance.

Effective neural network approach to image recognition and control

Abstract: A new approach to structuring and training of feed-forward artificial neural networks (ANN) is proposed. That leads to overcome many shortcomings of multilayer perceptrons and ANNs with radial basis functions (RBF-nets). A dynamical training algorithm is developed in order to keep the optimal number of neurons in the hidden layers and to guarantee the finiteness of the training procedure due to individual training of each neuron. Results of applying of the proposed neural network to recognizing frontal images of human faces look very promising and give rise to propose a non-expensive security system.

A genetic algorithm based neural-tuned neural network

Abstract: This paper presents a neural-tuned neural network, which is trained by genetic algorithm (GA). The neural-tuned neural network consists of a neural network and a modified neural network. In the modified neural network, a neuron model with two activation functions is introduced. Some parameters of these activation functions is tuned by neural network. The proposed network structure can increase the search space of the network and gives better performance than traditional feedforward neural networks. Some application examples are given to illustrate the merits of the proposed network.

Face Localization in the Neural Abstraction Pyramid

Abstract: One of the major parts in human-computer interface applications, such as face recognition and video-telephony, consists in the localization of a face in an image. I propose to use hierarchical neural networks with local recurrent connectivity to solve this task, even in presence of complex backgrounds, difficult lighting, and noise. The network is trained using a database of gray-scale still images and manually determined eye coordinates. It is able to produce reliable and accurate eye coordinates for unknown images by iteratively refining an initial solution. Since the network processes an entire image, no time consuming scanning across positions and scales is needed. Its fast update allows for real-time face tracking.

The research on an algorithm for special two-hidden layer artificial neural networks

Abstract: Generalization ability of the network and training time are the two important aspects that we must consider when we design the neural network algorithms. At the same time, the optimization of neural network architecture must be considered in each artificial neural network based on the BP algorithm. But to the larger networks, there are no more suitable ways to solve this problem. This paper proposes an especial two-hidden-layer artificial neural network. After describing the major steps of this algorithm, some experimental results and analysis are given out. Those experimental results indicate that the generalization ability, training time and the architecture optimization of the networks have been improved obviously in this algorithm.

Enhancing multi-neural systems through the use of hybrid structures

Abstract: This paper investigates the performance of multi-neural systems, focusing on the benefits that can be gained when integrating different types of neural experts (hybrid multi-neural system). An empirical evaluation shows that the integration of different types of neural networks leads to an improvement in performance in a practical classification task for a range of combination methods.