Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

From the Publisher:
With this text, you gain an understanding of the fundamental concepts of algorithms, the very heart of computer science. It introduces the basic data structures and programming techniques often used in efficient algorithms. Covers use of lists, push-down stacks, queues, trees, and graphs. Later chapters go into sorting, searching and graphing algorithms, the string-matching algorithms, and the Schonhage-Strassen integer-multiplication algorithm. Provides numerous graded exercises at the end of each chapter.


0201000296B04062001

The Design and Analysis of Computer Algorithms

A digital computer is generally believed to be an efficient universal computing device; that is, it is believed able to simulate any physical computing device with an increase in computation time by at most a polynomial factor. This may not be true when quantum mechanics is taken into consideration. This paper considers factoring integers and finding discrete logarithms, two problems which are generally thought to be hard on a classical computer and which have been used as the basis of several proposed cryptosystems. Efficient randomized algorithms are given for these two problems on a hypothetical quantum computer. These algorithms take a number of steps polynomial in the input size, e.g., the number of digits of the integer to be factored.

Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer

Our growing dependence on increasingly complex computer and software systems necessitates the development of formalisms, techniques, and tools for assessing functional properties of these systems. One such technique that has emerged in the last twenty years is model checking, which systematically (and automatically) checks whether a model of a given system satisfies a desired property such as deadlock freedom, invariants, and request-response properties. This automated technique for verification and debugging has developed into a mature and widely used approach with many applications. Principles of Model Checking offers a comprehensive introduction to model checking that is not only a text suitable for classroom use but also a valuable reference for researchers and practitioners in the field. The book begins with the basic principles for modeling concurrent and communicating systems, introduces different classes of properties (including safety and liveness), presents the notion of fairness, and provides automata-based algorithms for these properties. It introduces the temporal logics LTL and CTL, compares them, and covers algorithms for verifying these logics, discussing real-time systems as well as systems subject to random phenomena. Separate chapters treat such efficiency-improving techniques as abstraction and symbolic manipulation. The book includes an extensive set of examples (most of which run through several chapters) and a complete set of basic results accompanied by detailed proofs. Each chapter concludes with a summary, bibliographic notes, and an extensive list of exercises of both practical and theoretical nature.

/pdf/principles-of-model-checking-249iimickz.pdf

Principles of Model Checking

SPIN is an efficient verification system for models of distributed software systems. It has been used to detect design errors in applications ranging from high-level descriptions of distributed algorithms to detailed code for controlling telephone exchanges. The paper gives an overview of the design and structure of the verifier, reviews its theoretical foundation, and gives an overview of significant practical applications.

/pdf/the-model-checker-spin-1bgo4jtefp.pdf

The model checker SPIN

In this paper, the problem of determining economical state assignments for finite-state sequential machines is studied. The fundamental idea in this study is to find methods for selection of these assignments in which each binary variable describing the new state depends on as few variables of the old state as possible. In general, these variable assignments in which the dependence is reduced yield more economical implementation for the sequential machine than the assignments in which the dependence is not reduced. The main tool used in this study is the partition with the substitution property on the set of states of a sequential machine. It is shown that for a sequential machine the existence of assignments with reduced dependence is very closely connected with the existence of partitions with the substitution property on the set of states of the machine. It is shown how to determine these partitions for a given sequential machine and how they can be used to obtain assignments with reduced dependence.

On the State Assignment Problem for Sequential Machines II

Generalized Kolmogorov Complexity and the Structure of Feasible Computations (Preliminary Report)

In this paper we define a generalized, two-parameter, Kolmogorov complexity of finite strings which measures how much and how fast a string can be compressed and we show that this string complexity measure is an efficient tool for the study of computational complexity. The advantage of this approach is that it not only classifies strings as random or not random, but measures the amount of randomness detectable in a given time. This permits the study how computations change the amount of randomness of finite strings and thus establish a direct link between computational complexity and generalized Kolmogorov complexity of strings. This approach gives a new viewpoint for computational complexity theory, yields natural formulations of new problems and yields new results about the structure of feasible computations.

/pdf/generalized-kolmogorov-complexity-and-the-structure-of-r5kphwh39x.pdf

Generalized Kolmogorov complexity and the structure of feasible computations

A quantum jump in computer science.- Artificial life and real world computing.- Recurrent neural networks.- Scalable computing.- Efficient use of parallel & distributed systems: From theory to practice.- Experimental validation of models of parallel computation.- Quo vadetis, parallel machine models?.- Templates for linear algebra problems.- The ART behind IDEAS.- Algorithmic number theory and its relationship to computational complexity.- Edge-coloring algorithms.- Towards a computational theory of genome rearrangements.- Algebraic topology and distributed computing a primer.- Differential BDDs.- Algorithmic techniques for geometric optimization.- All the needles in a haystack: Can exhaustive search overcome combinatorial chaos?.- Fundamental limitations on search algorithms: Evolutionary computing in perspective.- Mathematical system models as a basis of software engineering.- Formulations and formalisms in software architecture.- The Oz Programming Model.- Standard Generalized Markup Language: Mathematical and philosophical issues.- Avoiding the undefined by underspecification.- Towards a theory of recursive structures.- Chu spaces and their interpretation as concurrent objects.- Abstracting unification: A key step in the design of logic program analyses.- Programming Satan's computer.- Petri Net models of distributed algorithms.- Symmetry and induction in model checking.- Alternating automata and program verification.- Reasoning about actions and change with ramification.- Trends in active vision.- Computational machine learning in theory and praxis.- Fuzzy sets as a tool for modeling.- Information retrieval and informative reasoning.- Database transaction models.- Multimedia authoring tools: State of the art and research challenges.- Computational models for distributed multimedia applications.- Hypermedia systems as internet tools.

Computer Science Today: Recent Trends and Developments

The purpose of this paper is to outline the theory of computational complexity which has emerged as a comprehensive theory during the last decade. This theory is concerned with the quantitative aspects of computations and its central theme is the measuring of the difficulty of computing functions. The paper concentrates on the study of computational com- plexity measures defined for all computable functions and makes no attempt to survey the whole field exhaustively nor to present the material in historical order. Rather it presents the basic concepts, results, and techniques of computational complexity from a new point of view from which the ideas are more easily understood and fit together as a coherent whole. It is clear that a viable theory of computation must deal realistically with the quantitative aspects of computing and must develop a general theory which studies the properties of possible measures of the difficulty of computing functions. Such a theory must go beyond the classification of functions as computable and non- computable, or elementary and primitive reeursive, etc. It must concern itself with computational complexity measures which are defined for all possible computations and which assign a complexity to each computation which terminates. Furthermore, this theory must eventually reflect some aspects of real computing to justify its existence by contributing to the general development of computer science. During the last decade, considerable progress has been made in the development of such a theory dealing with the complexity of computations. It is our conviction that by now this theory is an essential part of the theory of computation, and that in the future it will be an important theory which will permeate much of the theoretical work in computer science. Our purpose in this paper is to outline the recently developed theory of computa- tional complexity by presenting its central concepts, results, and techniques. The paper is primarily concerned with the study of computational complexity measures defined for all computable partial functions and no attempt is made to survey the whole field exhaustively nor to present the material in historical order. Rather, we

/pdf/an-overview-of-the-theory-of-computational-complexity-3fqpvdk17l.pdf

Juris Hartmanis

Papers

On the State Assignment Problem for Sequential Machines II

Generalized Kolmogorov Complexity and the Structure of Feasible Computations (Preliminary Report)

Generalized Kolmogorov complexity and the structure of feasible computations

Computer Science Today: Recent Trends and Developments

An Overview of the Theory of Computational Complexity