Statistical learning theory

About: Statistical learning theory is a research topic. Over the lifetime, 1618 publications have been published within this topic receiving 158033 citations.

The Informational Complexity of Learning: Perspectives on Neural Networks and Generative Grammar

Abstract: From the Publisher: Among other topics, The Informational Complexity of Learning: Perspectives on Neural Networks and Generative Grammar brings together two important but very different learning problems within the same analytical framework. The first concerns the problem of learning functional mappings using neural networks, followed by learning natural language grammars in the principles and parameters tradition of Chomsky. These two learning problems are seemingly very different. Neural networks are real-valued, infinite-dimensional, continuous mappings. On the other hand, grammars are boolean-valued, finite-dimensional, discrete (symbolic) mappings. Furthermore the research communities that work in the two areas almost never overlap. The book's objective is to bridge this gap. It uses the formal techniques developed in statistical learning theory and theoretical computer science over the last decade to analyze both kinds of learning problems. By asking the same question - how much information does it take to learn - of both problems, it highlights their similarities and differences. Specific results include model selection in neural networks, active learning, language learning and evolutionary models of language change.

Research on the hybrid models of granular computing and support vector machine

Abstract: The hybrid models of granular computing and support vector machine are a kind of new machine learning algorithms based on granular computing and statistical learning theory. These hybrid models can effectively use the advantage of each algorithm, so that their performance are better than a single method. In view of their excellent learning performance, the hybrid models of granular computing and support vector machine have become one of the focus at home and abroad. In this paper, the research on the hybrid models are reviewed, which include fuzzy support vector machine, rough support vector machine, quotient space support vector machine, rough fuzzy support vector machine and fuzzy rough support vector machine. Firstly, we briefly introduce the typical granular computing models and the basic theory of support vector machines. Secondly, we describe the latest progress of these hybrid models in recent years. Finally, we point out the research and development prospects of the hybrid algorithms.

Fundamental Limitations of Semi-Supervised Learning

Abstract: The emergence of a new paradigm in machine learning known as semi-supervised learning (SSL) has seen benefits to many applications where labeled data is expensive to obtain. However, unlike supervised learning (SL), which enjoys a rich and deep theoretical foundation, semi-supervised learning, which uses additional unlabeled data for training, still remains a theoretical mystery lacking a sound fundamental understanding. The purpose of this research thesis is to take a first step towards bridging this theory-practice gap. We focus on investigating the inherent limitations of the benefits semi-supervised learning can provide over supervised learning. We develop a framework under which one can analyze the potential benefits, as measured by the sample complexity of semi-supervised learning. Our framework is utopian in the sense that a semi-supervised algorithm trains on a labeled sample and an unlabeled distribution, as opposed to an unlabeled sample in the usual semi-supervised model. Thus, any lower bound on the sample complexity of semi-supervised learning in this model implies lower bounds in the usual model. Roughly, our conclusion is that unless the learner is absolutely certain there is some non-trivial relationship between labels and the unlabeled distribution (“SSL type assumption”), semi-supervised learning cannot provide significant advantages over supervised learning. Technically speaking, we show that the sample complexity of SSL is no more than a constant factor better than SL for any unlabeled distribution, under a no-prior-knowledge setting (i.e. without SSL type assumptions). We prove that for the class of thresholds in the realizable setting the sample complexity of SL is at most twice that of SSL. Also, we prove that in the agnostic setting for the classes of thresholds and union of intervals the sample complexity of SL is at most a constant factor larger than that of SSL. We conjecture this to be a general phenomenon applying to any hypothesis class. We also discuss issues regarding SSL type assumptions, and in particular the popular cluster assumption. We give examples that show even in the most accommodating circumstances, learning under the cluster assumption can be hazardous and lead to prediction performance much worse than simply ignoring unlabeled data and doing supervised learning. This thesis concludes with a look into future research directions that builds on our investigation.

Analysis, Design, and Comparison of Machine-Learning Techniques for Networking Intrusion Detection

Abstract: The use of machine-learning techniques is becoming more and more frequent in solving all those problems where it is difficult to rationally interpret the process of interest Intrusion detection in networked systems is a problem in which, although it is not fundamental to interpret the measures that one is able to obtain from a process, it is important to obtain an answer from a classification algorithm if the network traffic is characterized by anomalies (and hence, there is a high probability of an intrusion) or not Due to the increased adoption of SW-defined autonomous systems that are distributed and interconnected, the probability of a cyber attack is increased, as well as its consequence in terms of system reliability, availability, and even safety In this work, we present the application of different machine-learning models to the problem of anomaly classification in the context of local area network (LAN) traffic analysis In particular, we present the application of a K-nearest neighbors (KNN) and of an artificial neural network (ANN) to realize an algorithm for intrusion detection systems (IDS) The dataset used in this work is representative of the communication traffic in common LAN networks in military application in particular typical US Air Force LAN This work presents a training phase of the different models based on a multidimensional-scaling preprocessing procedure, based on different metrics, to provide higher performance and generalization with respect to model prediction capability The obtained results of KNN and ANN classifiers are compared with respect to a commonly used index of performance for classifiers evaluation

From learning theory to connectionist theory

Abstract: Volume I: From Learning Theory to Connectionist Theory. Contents: P. Suppes, Estes' Statistical Learning Theory: Past, Present, and Future. G. Bower, E. Heit, Choosing Between Uncertain Options: A Reprise to the Estes Scanning Model. R.D. Luce, A Path Taken: Aspects of Modern Measurement Theory. J.T. Townsend, Chaos Theory: A Brief Tutorial and Discussion. S. Link, Imitatio Estes: Stimulus Sampling Origins of Weber's Law. D. LaBerge, A Mathematical Theory of Attention in a Distractor Task. J.I. Yellot, Jr., Triple Correlation and Texture Discrimination. R.M. Nosofsky, Exemplars, Prototypes, and Similarity Rules. M.A. Gluck, Stimulus Sampling and Distributed Representations in Adaptive Network Theories of Learning. B.B. Murdock, Serial Organization in a Distributed Memory Model. S.A. Sloman, D.E. Rumelhart, Reducing Interference in Distributed Memories Through Episodic Gating. J.G. Rueckl, S.M. Kosslyn, What Good is Connectionist Modeling? A Dialogue.