Showing papers by "Yoshua Bengio published in 1991"

Learning a synaptic learning rule

Abstract: Summary form only given, as follows. The authors discuss an original approach to neural modeling based on the idea of searching, with learning methods, for a synaptic learning rule which is biologically plausible and yields networks that are able to learn to perform difficult tasks. The proposed method of automatically finding the learning rule relies on the idea of considering the synaptic modification rule as a parametric function. This function has local inputs and is the same in many neurons. The parameters that define this function can be estimated with known learning methods. For this optimization, particular attention is given to gradient descent and genetic algorithms. In both cases, estimation of this function consists of a joint global optimization of the synaptic modification function and the networks that are learning to perform some tasks. Both network architecture and the learning function can be designed within constraints derived from biological knowledge. >

Global optimization of a neural network-hidden Markov model hybrid

Abstract: An original method for integrating artificial neural networks (ANN) with hidden Markov models (HMM) is proposed. ANNs are suitable for performing phonetic classification, whereas HMMs have been proven successful at modeling the temporal structure of the speech signal. In the approach described, the ANN outputs constitute the sequence of observation vectors for the HMM. An algorithm is proposed for global optimization of all the parameters. Results on speaker-independent recognition experiments using this integrated ANN-HMM system on the TIMIT continuous speech database are reported. >

Phonetically motivated acoustic parameters for continuous speech recognition using artificial neural networks.

Abstract: Dans le cadre d'un decodeur acoustique-phonetique hybride (ANN/HMM), trois reseaux de neurones (ANNs) specialises ont ete developpes et evalues. Un de ces ANNs detecte le mode d'articulation. Les deux autres ANNs decrivent le signal en termes du lieu d'articulation. Un reseau est utilise pour classifier les consonnes nasales et plosives. Un autre est utilise pour la classification des fricatives Le design de ces reseaux est inspire par des connaissances acoustiques et phonetiques. Les entrees, la topologie et le codage des sorties ont ete optimises pour chacun des reseaux

Connectionist Models and their Application to Automatic Speech Recognition

Abstract: The purpose of this chapter is to study the application of some connectionist models to automatic speech recognition. Ways to take advantage of a-priori knowledge in the design of those models are first considered. Then algorithms for some recurrent networks are described since they are well-suited to handling temporal dependences such as those found in speech. Some simple methods that accelerate the convergence of gradient descent with the back-propagation algorithm are discussed. An alternative approach to speed-up the networks are systems based on Radial Basis Functions (local representation). Detailed results of several experiments with these networks on the recognition of phonemes for the TIMIT database are presented. In conclusion, a cognitively relevant model is proposed. This model combines both a local representation and and a distributed representation subnetworks to which correspond respectively a fast-learning and a slow-learning capability.

A comparative study on hybrid acoustic phonetic decoders based on artificial neural networks.

Abstract: In this paper we compare two hybrid acoustic-phonetic decoders based on Artiicial Neural Networks (ANN). We evaluate them on the task of recognizing stop phones in continuous speech independently from the speaker. ANNs are well suited to perform detailed phonetic distinctions. In general, techniques based on Dynamic Programming (DP), in particular Hidden Markov Models (HMMs), have proven to be successful at modeling the temporal structure of the speech signal. In the approach described here, the ANN outputs constitute the sequence of observation vectors for the HMM. An algorithm is proposed for global optimization of all the parameters of the ANN/HMM decoder. Comparative experiments using this ANN/HMM hybrid decoder and another ANN-DP hybrid are reported for the TIMIT database.