Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9x to 13x; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy efficiency.

https://forum.stanford.edu/events/posterslides/DeepCompressionCompressingDeepNeuralNetworkswithPruning,TrainedQuantizationandHuffmanCoding.pdf

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

Model Checking

A classic introduction to artificial intelligence intended to bridge the gap between theory and practice, "Principles of Artificial Intelligence" describes fundamental AI ideas that underlie applications such as natural language processing, automatic programming, robotics, machine vision, automatic theorem proving, and intelligent data retrieval. Rather than focusing on the subject matter of the applications, the book is organized around general computational concepts involving the kinds of data structures used, the types of operations performed on the data structures, and the properties of the control strategies used. "Principles of Artificial Intelligence"evolved from the author's courses and seminars at Stanford University and University of Massachusetts, Amherst, and is suitable for text use in a senior or graduate AI course, or for individual study.

Principles of Artificial Intelligence

Preface (M Karpinksi and J Van Leeuwen) Input-Driven Languages are Recognized in log n Space (B v Braunmuehl and R Verbeek) Constructive Mathematics as a Programming Logic I: Some Principles of Theory (RL Constable) Space and Reversal Complexity of Probabilistic One-Way Turing Machines (R Freivalds) Recurring Dominoes: Making the Highly Undecidable Highly Understandable (D Harel) A New Transformational Approach to Partial Correctness Proof Calculi for Algol 68-like Programs with Finite Modes and Simple Side Effects (H Langmaack) Effective Determination of the Zeros of p-ADIC Exponential Functions (A MacIntyre) The Logic of Games and Its Applications (R Parikh) A Fast Parallel Construction of Disjoint Paths in Networks (E Shamir and E Upfal) Descriptional Complexity for Classes of Ianov-Schemes (P Trum and D Wotschke)

Topics in the theory of computation : selected papers of the International Conference on "Foundations of Computation Theory", FCT '83, Borgholm, Sweden, August 21-27, 1983

In this paper a close relation is worked out that exists between two distinct problems in the area of distributed computations, namely "on-the-fly" garbage collection and distributed termination detection. It is demonstrated how "on-the-fly" garbage collecting algorithms can be derived from termination detection protocols by applying a few standard transformations. Virtually all existing on-the-fly garbage collecting algorithms can be obtained in this way. Also some new, highly parallel garbage collecting algorithms are easily obtained in this way.

The Derivation of on-the-fly Garbage Collection Algorithms from Distributed Termination Detection Protocols

Incidents like the crash of Lion Air Flight 610 in 2018 challenge the design of reliable and secure cyber-physical systems that operate in the real-world and have to cope with unpredictable external phenomena and error-prone technology. We argue that their design needs to guarantee minimal machine consciousness, which expresses that these systems must operate with full awareness of (the state of) their components and the environment. The concept emerged from our recent effort to develop a computational model for conscious behavior in robots, based on the theory of automata. Making systems ‘minimal machine conscious’ leads to more trustworthy systems, as it strengthens their behavioral flexibility in varying environments and their resilience to operation and cooperation failures of their components and as a whole. The notion of minimal machine consciousness has the potential to become one of the defining attributes of Industry 4.0.

Jan van Leeuwen

Papers

Topics in the theory of computation : selected papers of the International Conference on "Foundations of Computation Theory", FCT '83, Borgholm, Sweden, August 21-27, 1983

The Derivation of on-the-fly Garbage Collection Algorithms from Distributed Termination Detection Protocols

Towards Minimally Conscious Cyber-Physical Systems: A Manifesto

Uniform d‐emulations of rings, with an application to distributed virtual ring construction

Interactive Multimedia Documents