Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

From the Publisher:
With this text, you gain an understanding of the fundamental concepts of algorithms, the very heart of computer science. It introduces the basic data structures and programming techniques often used in efficient algorithms. Covers use of lists, push-down stacks, queues, trees, and graphs. Later chapters go into sorting, searching and graphing algorithms, the string-matching algorithms, and the Schonhage-Strassen integer-multiplication algorithm. Provides numerous graded exercises at the end of each chapter.


0201000296B04062001

The Design and Analysis of Computer Algorithms

A digital computer is generally believed to be an efficient universal computing device; that is, it is believed able to simulate any physical computing device with an increase in computation time by at most a polynomial factor. This may not be true when quantum mechanics is taken into consideration. This paper considers factoring integers and finding discrete logarithms, two problems which are generally thought to be hard on a classical computer and which have been used as the basis of several proposed cryptosystems. Efficient randomized algorithms are given for these two problems on a hypothetical quantum computer. These algorithms take a number of steps polynomial in the input size, e.g., the number of digits of the integer to be factored.

Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer

Our growing dependence on increasingly complex computer and software systems necessitates the development of formalisms, techniques, and tools for assessing functional properties of these systems. One such technique that has emerged in the last twenty years is model checking, which systematically (and automatically) checks whether a model of a given system satisfies a desired property such as deadlock freedom, invariants, and request-response properties. This automated technique for verification and debugging has developed into a mature and widely used approach with many applications. Principles of Model Checking offers a comprehensive introduction to model checking that is not only a text suitable for classroom use but also a valuable reference for researchers and practitioners in the field. The book begins with the basic principles for modeling concurrent and communicating systems, introduces different classes of properties (including safety and liveness), presents the notion of fairness, and provides automata-based algorithms for these properties. It introduces the temporal logics LTL and CTL, compares them, and covers algorithms for verifying these logics, discussing real-time systems as well as systems subject to random phenomena. Separate chapters treat such efficiency-improving techniques as abstraction and symbolic manipulation. The book includes an extensive set of examples (most of which run through several chapters) and a complete set of basic results accompanied by detailed proofs. Each chapter concludes with a summary, bibliographic notes, and an extensive list of exercises of both practical and theoretical nature.

/pdf/principles-of-model-checking-249iimickz.pdf

Principles of Model Checking

SPIN is an efficient verification system for models of distributed software systems. It has been used to detect design errors in applications ranging from high-level descriptions of distributed algorithms to detailed code for controlling telephone exchanges. The paper gives an overview of the design and structure of the verifier, reviews its theoretical foundation, and gives an overview of significant practical applications.

/pdf/the-model-checker-spin-1bgo4jtefp.pdf

The model checker SPIN

To develop an abstract theory which deals with the quantitative aspects of computing we need a deeper understanding of how to define ``natural'''' computational complexity measures axiomatically. To this end, this paper summarizess the principal properties which hold for some natural complexity measures but not for all measures and which have been proposed as desirable properties of natural measuress. The paper discusses the nature of these properties, studies their interrelations and their possible values towards defining natural computational complexity measures. A number off open problems are discussed and directions for further research are suggested.

On the Problem of Finding Natural Computational Complexity Measures

In this column we explore what relativization says about space bounded computations and what recent results about space bounded computations say about relativization. There is a strong belief in computational complexity circles that problems which can be relativized in two contradictory ways are very hard to solve. We believe that such problems can only be solved by proof techniques that do not relativize. For example, standard diagonalization methods are powerless in these cases [BGS75, HH76, Hop84]. So far only very simple problems with contradictory relativizations have been solved [Har85, Cha90]. Unfortunately, many important problems in complexity theory have contradictory relativizations and have to be viewed as inaccessible to

Some Observations about Relativization of Space Bounded Computations.

Relations between diagonalization, proof systems, and complexity gaps☆

The purpose of this paper is to give simple new proofs of some interesting recent results about the relative succinctness of different representations of regular, deterministic and unambiguous context-free languages and to derive some new results about how the relative succinctness of representations change when the representations contain a formal proof that the languages generated are in the desired subclass of languages.

On the Succintness of Different Representations of Languages

I am impressed by the twelve essays in response to my Turing Award paper: they show great variety and originality, with interesting observations and insights about computer science. They call for new directions and some express concern for the future of our science. I hope that these essays as well as my paper will stimulate further reflection on the nature of computer science. I cannot agree with the statement, “Debates about ‘What is CS?’ can be counterproductive.” I am convinced that a better understanding of the nature of computer science will lead to more shared values in the community, better understanding of computer science by other scientists, and even to better research. At the same time, I like the eloquent rejection of the question, “Are we scientists or engineers?” I am sorry that time and space do not permit me to give proper attention to the ideas and comments in all the essays and that my response will touch on a few topics only. There is no doubt in my mind that computer science is emerging as a major science with rich intellectual achievements and great potential importance to many of our intellectual, scientific, and commercial activities. Not only that, computer science is forging new research paradigms for a new type of science that is likely to influence and guide other intellectual developments. I do not believe that computer scientists should feel apologetic in any way about what has been achieved. Quite the contrary. Still, in some essays one can detect a longing that computer science be more like physics and the view that if it does not fit the mold of the physical sciences, tlhe fault is that of computer science or its immaturity. I disagree with these views very strongly indeed. Let us not try to force this new science into any preconceived mold. Let us observe this fascinating intellectual development and try to understand it in the broad perspective of other sciences and be proud of it. I fully agree with the call for computer science to get more involved with interdisciplinary work, the encouragement of theoreticians “to move much closer to practice” and the view that “there is progressively greater need for theoretical study of approaches prior to experimentation. Appropriate models and techniques are vital.” I like the formulation of “effective computer science.” I will not argue whether we should call computer science “a new species among the sciences” or “a new species of engineering.” It is only

Juris Hartmanis

Papers

On the Problem of Finding Natural Computational Complexity Measures

Some Observations about Relativization of Space Bounded Computations.

Relations between diagonalization, proof systems, and complexity gaps☆

On the Succintness of Different Representations of Languages

Response to the essay “On computational complexity and the nature of computer science”