Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

National Institute of Standards and Technology における超伝導研究及び生活

Digital processing of speech signals

Lecture Notes in Economics and Mathematical Systems

Preface Bibliography 1. Interpolation. Introduction The Taylor expansion The finite Taylor series with the remainder term Interpolation by polynomials The remainder of Lagrangian interpolation formula Equidistant interpolation Local and global interpolation Interpolation by central differences Interpolation around the midpoint of the range The Laguerre polynomials Binomial expansions The decisive integral transform Binomial expansions of the hypergeometric type Recurrence relations The Laplace transform The Stirling expansion Operations with the Stirling functions An integral transform of the Fourier type Recurrence relations associated with the Stirling series Interpolation of the Fourier transform The general integral transform associated with the Stirling series Interpolation of the Bessel functions 2. Harmonic Analysis. Introduction The Fourier series for differentiable functions The remainder of the finite Fourier expansion Functions of higher differentiability An alternative method of estimation The Gibbs oscillations of the finite Fourier series The method of the Green's function Non-differentiable functions Dirac's delta function Smoothing of the Gibbs oscillations by Fejer's method The remainder of the arithmetic mean method Differentiation of the Fourier series The method of the sigma factors Local smoothing by integration Smoothing of the Gibbs oscillations by the sigma method Expansion of the delta function The triangular pulse Extension of the class of expandable functions Asymptotic relations for the sigma factors The method of trigonometric interpolation Error bounds for the trigonometric interpolation method Relation between equidistant trigonometric and polynomial interpolations The Fourier series in the curve fitting 3. Matrix Calculus. Introduction Rectangular matrices The basic rules of matrix calculus Principal axis transformation of a symmetric matrix Decomposition of a symmetric matrix Self-adjoint systems Arbitrary n x m systems Solvability of the general n x m system The fundamental decomposition theorem The natural inverse of a matrix General analysis of linear systems Error analysis of linear systems Classification of linear systems Solution of incomplete systems Over-determined systems The method of orthogonalisation The use of over-determined systems The method of successive orthogonalisation The bilinear identity Minimum property of the smallest eigenvalue 4. The Function Space. Introduction The viewpoint of pure and applied mathematics The language of geometry Metrical spaces of infinitely many dimensions The function as a vector The differential operator as a matrix The length of a vector The scalar product of two vectors The closeness of the algebraic approximation The adjoint operator The bilinear identity The extended Green's identity The adjoint boundary conditions Incomplete systems Over-determined systems Compatibility under inhomogeneous boundary conditions Green's identity in the realm of partial differential operators The fundamental field operations of vector analysis Solution of incomplete systems 5. The Green's Function. Introduction The role of the adjoint equation The role of Green's identity The delta function -- The existence of the Green's function Inhomogeneous boundary conditions The Green's vector Self-adjoint systems The calculus of variations The canonical equations of Hamilton The Hamiltonisation of partial operators The reciprocity theorem Self-adjoint problems Symmetry of the Green's function Reciprocity of the Green's vector The superposition principle of linear operators The Green's function in the realm of ordinary differential operators The change of boundary conditions The remainder of the Taylor series The remainder of the Lagrangian interpolation formula

Linear Differential Operators. By C. Lanczos. Pp. 580. 80s. 1961. (Van Nostrand, London)

The most widely spread measure of performance, accuracy, suffers from a paradox: predictive models with a given level of accuracy may have greater predictive power than models with higher accuracy. Despite optimizing classification error rate, high accuracy models may fail to capture crucial information transfer in the classification task. We present evidence of this behavior by means of a combinatorial analysis where every possible contingency matrix of 2, 3 and 4 classes classifiers are depicted on the entropy triangle, a more reliable information-theoretic tool for classification assessment.

Motivated by this, we develop from first principles a measure of classification performance that takes into consideration the information learned by classifiers. We are then able to obtain the entropy-modulated accuracy (EMA), a pessimistic estimate of the expected accuracy with the influence of the input distribution factored out, and the normalized information transfer factor (NIT), a measure of how efficient is the transmission of information from the input to the output set of classes.

The EMA is a more natural measure of classification performance than accuracy when the heuristic to maximize is the transfer of information through the classifier instead of classification error count. The NIT factor measures the effectiveness of the learning process in classifiers and also makes it harder for them to “cheat” using techniques like specialization, while also promoting the interpretability of results. Their use is demonstrated in a mind reading task competition that aims at decoding the identity of a video stimulus based on magnetoencephalography recordings. We show how the EMA and the NIT factor reject rankings based in accuracy, choosing more meaningful and interpretable classifiers.

/pdf/100-classification-accuracy-considered-harmful-the-23lttu4mgr.pdf

100% classification accuracy considered harmful: the normalized information transfer factor explains the accuracy paradox.

The Internet Protocol (IP) environment poses two relevant sources of distortion to the speech recognition problem: lossy speech coding and packet loss. In this paper, we propose a new front-end for speech recognition over IP networks. Specifically, we suggest extracting the recognition feature vectors directly from the encoded speech (i.e., the bit stream) instead of decoding it and subsequently extracting the feature vectors. This approach offers two significant benefits. First, the recognition system is only affected by the quantization distortion of the spectral envelope. Thus, we are avoiding the influence of other sources of distortion due to the encoding-decoding process. Second, when packet loss occurs, our front-end becomes more effective since it is not constrained to the error handling mechanism of the codec. We have considered the ITU G.723.1 standard codec, which is one of the most preponderant coding algorithms in voice over IP (VoIP) and compared the proposed front-end with the conventional approach in two automatic speech recognition (ASR) tasks, namely, speaker-independent isolated digit recognition and speaker-independent continuous speech recognition. In general, our approach outperforms the conventional procedure, for a variety of simulated packet loss rates. Furthermore, the improvement is higher as network conditions worsen.

/pdf/recognizing-voice-over-ip-a-robust-front-end-for-speech-4osnww1vnl.pdf

Recognizing voice over IP: a robust front-end for speech recognition on the world wide web

Robust ASR using Support Vector Machines

Extending conceptualisation modes for generalised Formal Concept Analysis

Hidden Markov Models (HMMs) are, undoubtedly, the most employed core technique for Automatic Speech Recognition (ASR). Nevertheless, we are still far from achieving high-performance ASR systems. Some alternative approaches, most of them based on Artificial Neural Networks (ANNs), were proposed during the late eighties and early nineties. Some of them tackled the ASR problem using predictive ANNs, while others proposed hybrid HMM/ANN systems. However, despite some achievements, nowadays, the preponderance of Markov Models is a fact.

During the last decade, however, a new tool appeared in the field of machine learning that has proved to be able to cope with hard classification problems in several fields of application: the Support Vector Machines (SVMs). The SVMs are effective discriminative classifiers with several outstanding characteristics, namely: their solution is that with maximum margin; they are capable to deal with samples of a very higher dimensionality; and their convergence to the minimum of the associated cost function is guaranteed.

These characteristics have made SVMs very popular and successful. In this chapter we discuss their strengths and weakness in the ASR context and make a review of the current state-of-the-art techniques. We organize the contributions in two parts: isolated-word recognition and continuous speech recognition. Within the first part we review several techniques to produce the fixed-dimension vectors needed for original SVMs. Afterwards we explore more sophisticated techniques based on the use of kernels capable to deal with sequences of different length. Among them is the DTAK kernel, simple and effective, which rescues an old technique of speech recognition: Dynamic Time Warping (DTW). Within the second part, we describe some recent approaches to tackle more complex tasks like connected digit recognition or continuous speech recognition using SVMs. Finally we draw some conclusions and outline several ongoing lines of research.

/pdf/svms-for-automatic-speech-recognition-a-survey-2fa3zxdy49.pdf

Carmen Peláez-Moreno

Papers

100% classification accuracy considered harmful: the normalized information transfer factor explains the accuracy paradox.

Recognizing voice over IP: a robust front-end for speech recognition on the world wide web

Robust ASR using Support Vector Machines

Extending conceptualisation modes for generalised Formal Concept Analysis

SVMs for automatic speech recognition: a survey