Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

From the Publisher:
With this text, you gain an understanding of the fundamental concepts of algorithms, the very heart of computer science. It introduces the basic data structures and programming techniques often used in efficient algorithms. Covers use of lists, push-down stacks, queues, trees, and graphs. Later chapters go into sorting, searching and graphing algorithms, the string-matching algorithms, and the Schonhage-Strassen integer-multiplication algorithm. Provides numerous graded exercises at the end of each chapter.


0201000296B04062001

The Design and Analysis of Computer Algorithms

A digital computer is generally believed to be an efficient universal computing device; that is, it is believed able to simulate any physical computing device with an increase in computation time by at most a polynomial factor. This may not be true when quantum mechanics is taken into consideration. This paper considers factoring integers and finding discrete logarithms, two problems which are generally thought to be hard on a classical computer and which have been used as the basis of several proposed cryptosystems. Efficient randomized algorithms are given for these two problems on a hypothetical quantum computer. These algorithms take a number of steps polynomial in the input size, e.g., the number of digits of the integer to be factored.

Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer

Our growing dependence on increasingly complex computer and software systems necessitates the development of formalisms, techniques, and tools for assessing functional properties of these systems. One such technique that has emerged in the last twenty years is model checking, which systematically (and automatically) checks whether a model of a given system satisfies a desired property such as deadlock freedom, invariants, and request-response properties. This automated technique for verification and debugging has developed into a mature and widely used approach with many applications. Principles of Model Checking offers a comprehensive introduction to model checking that is not only a text suitable for classroom use but also a valuable reference for researchers and practitioners in the field. The book begins with the basic principles for modeling concurrent and communicating systems, introduces different classes of properties (including safety and liveness), presents the notion of fairness, and provides automata-based algorithms for these properties. It introduces the temporal logics LTL and CTL, compares them, and covers algorithms for verifying these logics, discussing real-time systems as well as systems subject to random phenomena. Separate chapters treat such efficiency-improving techniques as abstraction and symbolic manipulation. The book includes an extensive set of examples (most of which run through several chapters) and a complete set of basic results accompanied by detailed proofs. Each chapter concludes with a summary, bibliographic notes, and an extensive list of exercises of both practical and theoretical nature.

/pdf/principles-of-model-checking-249iimickz.pdf

Principles of Model Checking

SPIN is an efficient verification system for models of distributed software systems. It has been used to detect design errors in applications ranging from high-level descriptions of distributed algorithms to detailed code for controlling telephone exchanges. The paper gives an overview of the design and structure of the verifier, reviews its theoretical foundation, and gives an overview of significant practical applications.

/pdf/the-model-checker-spin-1bgo4jtefp.pdf

The model checker SPIN

In this paper we investigate how numbers, functions, and sequences can be classified according to their computational complexity. The computational complexity is measured by how fast the number or function can be computed by a multitape computer (Turing machine). It is shown that by means of this measure the computational complexity of numbers and functions can be submitted to rigorous mathematical study and a number of results are obtained.

Computational complexity of recursive sequnces

In this paper we consider classes of Goedel numberings, viewed as simple models for programming languages, into which all other Goedel numberings can be translated by computationally simple mappings Several such classes of Goedel numberings are defined and their properties are investigated For example, one such class studied is the class of Goedel numberings into which all other Goedel numberings can be translated by finite automatic mappings We also compare these classes of Goedel numberings to the class of optimal Goedel numberings and show that translation into optimal Goedel numberings can be computationally arbitrarily complex Thus indicating that from a computer science point of view optimal Goedel numberings have undesirable properties

/pdf/on-simple-goedel-numberings-and-translations-3f2sqag4im.pdf

On Simple Goedel Numberings and Translations

This paper reviews the impact of some recent results on the research paradigms in structural complexity theory.

/pdf/structural-complexity-theory-recent-surprises-5169uqtogb.pdf

Structural Complexity Theory: recent Surprises

Over the last six years the Open Hypermedia Systems Working Group (OHSWG) has been working in a coordinated effort to produce a protocol which will allow components of an Open Hypermedia System to talk to one another in a standardised manner. In this paper we reflect on this work and the knowledge that has come out of it, evaluating the different approaches to standardisation in the light of our experiences. We discuss the problems we encountered and redefine the goals of the effort to be more realistic, presenting the Fundamental Open Hypermedia Model (FOHM) as an example of this more realistic approach. Finally we describe a possible future path that encompasses the research interests of the OHSWG while still leading ultimately to interoperability. 1 History of the OHP Effort 1.1 Original Proposal The First Workshop on Open Hypermedia [25] was held at Edinburgh in conjunction with ECHT’94. This workshop was concerned with the growing class of hypermedia systems such as Chimera [2], DHM [9], HyperForm [24], Microcosm [5], Multicard [20] and the HB/SP series [21], which clearly separated hypertext structure (links) from the content (documents). The participants in this workshop were keen to provide hypertext link services which could provide hypertext structure for documents which were displayed using existing desktop applications such as Word for Windows and Emacs. This workshop lead to the formation of the Open Hypermedia Systems Working Group (OHSWG), the full history and rationale behind the work of this group can be viewed on their web pages [1]. An interesting finding of this first workshop was that although the major area of interest for the participating research groups was the design and implementation of link servers, most of their time was being spent on the implementation of clients; the researchers were spending significant effort producing text and graphics clients for the link services, either by writing them from scratch or writing macros to adapt existing desktop applications. A proposal from Antoine S. Reich and K.M. Anderson (Eds.): OHS/SC 2000, LNCS 1903, pp. 3–12, 2000. c © Springer-Verlag Berlin Heidelberg 2000 4 D. Millard, H. Davis, and L. Moreau Rizk was that the group could contribute by producing a lightweight message based protocol that could be used to communicate about simple link service functions. The rationale was that all link services had an approximately similar data model and that the operations that the link services could perform were also similar; all that would be required was a simple ”shim” (protocol converter) that could convert between the client protocol and the server protocol, and then it would be possible for groups to share client implementations. The idea was simple and lead to the production of the first draft of the Open Hypermedia Protocol (OHP) [6] which was presented at the next workshop in 1996. What happened next to OHP may well be a familiar story to other groups who have attempted to produce an application level protocol. The committee effect started to take hold and the protocol grew; there were discussions about whether we were going to use a message passing interface or an API; there were arguments about the on-the-wire protocol to be used, and the group became confused about aspects of resource location and naming instead of concentrating on hypertext. Furthermore, the increasing influence of the World Wide Web throughout this period tended to change original assumptions by producing a system that was open in very different ways from the OHSWG systems. However, there were some good outcomes from this stage of the work. The scope of the project changed from attempting to provide a lightweight communication mechanism for shared clients and heterogeneous link servers, to attempting to create a reference model and implementation for open hypertext systems. A standardized data model and basic set of operations was agreed and the groups concerned produced native OHP link servers, and agreed a temporary on the wire protocol that made possible a significant demonstration at Hypertext ’98, and a paper on the experiences to that stage [18]. 1.2 The Hypertext 98 Demonstration Two systems were developed for a demonstration of interoperability at Hypertext ’98 (held in Pittsburgh, USA). One from the University of Arhus, Denmark, and the other developed at the Multimedia Research Group (MMRG) at the University of Southampton. During the development of these systems several problems became evident. Both with the protocol itself and also, more importantly, with the scope of the original draft proposal. The original draft was meant as a standard interface between clients and servers, to allow software reuse. This has increased in scope dramatically and become an all-encompassing effort to understand the nature of hypermedia and thus produce standards to provide for it. However it was soon understood that such a large goal was impossible to realise within a single protocol and as a result the protocol was split into several domains, each domain dealing with a particular type of hypermedia. The original OHP protocol was therefore renamed OHP-Navigational (OHPNav) and reduced in scope to deal exclusively with navigational (node/link) hypermedia. Other domains were envisaged such as Spatial Hypermedia [12] (OHP-Space) and Taxonomic Hypermedia [22] (OHP-Tax). Standardizing Hypertext: Where Next for OHP? 5 The protocol itself had been originally based on the Microcosm message format [8], a sequence of tag/value pairs. However this proved difficult to parse so the OHSWG adopted XML as a suitable format [3] and the OHP-Nav message set and hypermedia objects were all defined as XML elements in a Document Type Definition (DTD). 1.3 The Hypertext 99 Demonstration At the OHSWG’s meeting at Southampton (OHS 4.5) it was decided that as the Hypertext ’98 demonstration had formed such a positive focus point for the group a similar demonstration should be attempted for Hypertext ’99 in Darmstadt, Germany. It was also decided that since we had demonstrated interoperability at Hypertext ’98 we should now concentrate on showing some of the features of the protocol, ironically removing the need to interoperate. Some of the more successful parts of the Hypertext ’98 demonstration were the collaboration aspects. So the Danish contribution to the Hypertext ’99 demonstration was an extension of this simple support into a more advanced system called ‘Construct’. The Southampton contribution was to investigate a definition of computational links, where opaque computation objects are included in the hypertext model and can be referenced similarly to other objects. This resulted in a component based system known as ‘Solent’. Another system demonstrated at Hypertext ’99 was CAOS [19], a spatial hypermedia collaborative system. Discussions within the working group turned to the definition of OHP-Space, starting us thinking about whether the different domains were actually that different after all. 2 What Were the Problems with OHP? 2.1 What Were We Trying to Standardize? It has already been mentioned that the purpose of OHP has changed a great deal since the protocol first appeared. Initially a basic client-server communication protocol it has grown to reflect the concerns of a large community of researchers. Even given that we understood what functionality we were going to standardize there is still the question of what actually will become standard in the system. I.e. how will components actually talk to one another? There are two approaches: 1. A programming API : By using a standardised API client source code compatibility is preserved, whatever the servers involved, and server source code remains valid whatever the clients. This requires specifying: a) the system calls, b) the callbacks to be used, c) the data to be exchanged. 6 D. Millard, H. Davis, and L. Moreau Examples would be to use CORBA and its interface definition language IDL, Microsoft DCOM or a Java component system using Java Beans. One should note that in this approach the data representation is dependent on the binding (i.e. the IDL compiler for the chosen language and the chosen implementation of the communication module). 2. An on-the-wire communication model : This involves defining: a) the syntax of the messages (e.g. an XML hierarchy), b) a set of requests, associated responses and their syntax, c) the data and its syntax, d) how to setup the transport medium (e.g. opening socket on a port, etc.). At one time or another both approaches have been argued for. However the need to produce communicating systems that actually worked resulted in the group returning to the on-the-wire approach, even though the API approach seems cleaner and allows us to concentrate on the hypertext issues rather than the networking ones. The API approach also preserves source code compatibility. The implementers just need to implement two APIs, one for the client and another for the server. If they then find that a new on-the-wire protocol should be used, they can change their implementation without altering the source code of the components involved. However it is still not a perfect solution as it may require recompiling applications when a different medium is required. This is indeed the case with CORBA where binary applications are ORB dependent. This does not give a lot of freedom to the final user as a binary is typically compiled for a fixed communication medium. Recently, Southampton researchers have been experimenting with implementing hypertext functionality on top of agent frameworks; in particular, the Javabased SoFAR [15], the Southampton Framework for Agent Research, was the focus of this experiment. SoFAR adopts an abstract communication model, where agents communicate using “virtual channels”, identified by a startpoint and an endpoint; the latter is a client-side proxy used to initiate communications and the former is a server

Juris Hartmanis

Papers

Computational complexity of recursive sequnces

On Simple Goedel Numberings and Translations

Structural Complexity Theory: recent Surprises

Open Hypermedia Systems and Structural Computing

Independence Results about Context-Free Languages and Lower Bounds