Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

From the Publisher:
With this text, you gain an understanding of the fundamental concepts of algorithms, the very heart of computer science. It introduces the basic data structures and programming techniques often used in efficient algorithms. Covers use of lists, push-down stacks, queues, trees, and graphs. Later chapters go into sorting, searching and graphing algorithms, the string-matching algorithms, and the Schonhage-Strassen integer-multiplication algorithm. Provides numerous graded exercises at the end of each chapter.


0201000296B04062001

The Design and Analysis of Computer Algorithms

A digital computer is generally believed to be an efficient universal computing device; that is, it is believed able to simulate any physical computing device with an increase in computation time by at most a polynomial factor. This may not be true when quantum mechanics is taken into consideration. This paper considers factoring integers and finding discrete logarithms, two problems which are generally thought to be hard on a classical computer and which have been used as the basis of several proposed cryptosystems. Efficient randomized algorithms are given for these two problems on a hypothetical quantum computer. These algorithms take a number of steps polynomial in the input size, e.g., the number of digits of the integer to be factored.

Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer

Our growing dependence on increasingly complex computer and software systems necessitates the development of formalisms, techniques, and tools for assessing functional properties of these systems. One such technique that has emerged in the last twenty years is model checking, which systematically (and automatically) checks whether a model of a given system satisfies a desired property such as deadlock freedom, invariants, and request-response properties. This automated technique for verification and debugging has developed into a mature and widely used approach with many applications. Principles of Model Checking offers a comprehensive introduction to model checking that is not only a text suitable for classroom use but also a valuable reference for researchers and practitioners in the field. The book begins with the basic principles for modeling concurrent and communicating systems, introduces different classes of properties (including safety and liveness), presents the notion of fairness, and provides automata-based algorithms for these properties. It introduces the temporal logics LTL and CTL, compares them, and covers algorithms for verifying these logics, discussing real-time systems as well as systems subject to random phenomena. Separate chapters treat such efficiency-improving techniques as abstraction and symbolic manipulation. The book includes an extensive set of examples (most of which run through several chapters) and a complete set of basic results accompanied by detailed proofs. Each chapter concludes with a summary, bibliographic notes, and an extensive list of exercises of both practical and theoretical nature.

/pdf/principles-of-model-checking-249iimickz.pdf

Principles of Model Checking

SPIN is an efficient verification system for models of distributed software systems. It has been used to detect design errors in applications ranging from high-level descriptions of distributed algorithms to detailed code for controlling telephone exchanges. The paper gives an overview of the design and structure of the verifier, reviews its theoretical foundation, and gives an overview of significant practical applications.

/pdf/the-model-checker-spin-1bgo4jtefp.pdf

The model checker SPIN

Abstr(~ct. Recently it has been showa how we can obJ~ain all possible realizations of a given sequential machine :]i (ks a submachine of a machine reMized) from interconnected sets of smaller machines. In this paper we shall derive further results about the realization of a sequential machine M from two smaller machines M~ and M= which are eotmeeted in series or parallel. We shall relate the structural properties of the machine M to the structural properties of the component machines M, and M~ and derive results about the uniqtte-hess and economy of such realizat.ions, The possibility of realizing a given sequential machine M from interconnected sets of smaller machines has been discussed in [1, 2, 3], where necessary ~md sufficient conditions were given for the decomposition of M into a serial or parMlel connection of two machines M~ and M2. We are considering only the case when M is a sub-machine of the machine obtahmd from M~ and M~. In this paper we continue the investigation of the structure of sequential machines by deriving further results on the serial and parallel decompositions. We shall assume that the reader is familiar with the notation, definitions and the basic results of [3]. Furthermore, we shall discuss only Moore-type sequential machines in this paper since the results can be easily extended to Mealy-type machines [4, 5]. First., we discuss the serial realization of M from 21:I1 and 2br2 and then deal with the properties of the parallel realizations. Let M be a reduced, completely specified sequential machine which we want to realize as u sub-machine of a machine realized from two serially connected machines M1 and M2, each having fewer states than M (i.e. to every state of M there wilt correspond a pair of states from M~ and M2 and this correspondence is preserved by the operation of the machines). We know from [3] that such a realization of M exists if and only if there exists a nomtrivial partition ~r with the substitution property on the set of states of M. If such a partition 7r exists, then the first machine M~ will have as its states the blocks of ~r and its state behavior is determined by the machine M. The second machine M2 is determined by a second partition r on the set of states of the machine M such that rr.r = O. In the terminology of [3] …

Further Results on the Structure of Sequential Machines

Linear multivalued sequential coding networks are circuits whose input and output are synchronized sequences of nonnegative integers less than some fixed number m . The output depends linearly on the present input and a finite number of previous inputs and outputs. The transfer characteristics of such a network are described by a ratio of polynomials in the delay operator, where the multiplication and addition are performed with respect to the fixed modulus m . An algebraic theory of the delay polynomials is obtained. It is shown that a polynomial has a complete set of null sequences if, and only if, its first and last coefficients are prime to the modulus m. The polynomials with no null sequences are characterized. It is shown when common null sequences imply that the polynomials have common factors and that a complete set of null sequences defines the polynomial. It is also shown that a transfer function can be realized if the denominator contains a constant term prime to m and explicit constructions are given. A network is stable if the polynomial in the denominator of the transfer junction has no null sequence. Thus any nontrivial polynomial or its inverse is unstable if we are working modulo a prime. If the modulus is not prime, stable networks with stable inverses are constructed. Finally it is indicated how polynomials with no null sequences can be used to simplify the construction of coding networks.

http://ieeexplore.ieee.org/iel5/8148/23609/01086508.pdf

Linear Multivalued Sequential Coding Networks

Some sequential machinies contain input-independent parts and can therefore be input-inependent machine (an autonomous clock) and an input-dependent machine. Necessary and sufficient conditions on the flow table are obtained for the existence of autonomous clocks in sequential machines, and the properties of these machines and their clocks are studied.

Maximal Autonomous Clocks of Sequential Machines

This paper establishes two bounds for the rate of growth of memory in automata which recognize the set of primes.

/pdf/two-memory-bounds-for-the-recognition-of-primes-by-automata-1z6e6xkciv.pdf

Two Memory Bounds for the Recognition of Primes by Automata

This paper is part of a general study of efficient information selection, storage and processing. It is assumed that the information is contained in binary time series generated by a stochastic source. The main problem is to determine how to approximate the statistical properties of this information source by lower order probability distributions. First, it is determined what restrictions are imposed by known lower order probability distributions on the higher order distributions which are to be determined or estimated. This study suggests an optimal way to estimate the higher order distributions. In the second part the entropy changes which occur in going from lower to higher order probability distributions are studied. The upper and lower bounds for the entropy of the higher order distributions are computed in terms of the entropies of the lower order distributions. These results allow the computation of the “strength” of the conditions which are imposed on the higher order distribution and are not induced by the lower order ones. From this one can compute the importance of knowing the higher order probability distributions for information processing. In conclusion, these results are applied to a coding problem to determine the delay required before encoding a message in order to achieve a prescribed fraction of the optimal comparison given by Shannon's coding theorem.

Juris Hartmanis

Papers

Further Results on the Structure of Sequential Machines

Linear Multivalued Sequential Coding Networks

Maximal Autonomous Clocks of Sequential Machines

Two Memory Bounds for the Recognition of Primes by Automata

The application of some basic inequalities for entropy