Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9x to 13x; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy efficiency.

https://forum.stanford.edu/events/posterslides/DeepCompressionCompressingDeepNeuralNetworkswithPruning,TrainedQuantizationandHuffmanCoding.pdf

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

Model Checking

A classic introduction to artificial intelligence intended to bridge the gap between theory and practice, "Principles of Artificial Intelligence" describes fundamental AI ideas that underlie applications such as natural language processing, automatic programming, robotics, machine vision, automatic theorem proving, and intelligent data retrieval. Rather than focusing on the subject matter of the applications, the book is organized around general computational concepts involving the kinds of data structures used, the types of operations performed on the data structures, and the properties of the control strategies used. "Principles of Artificial Intelligence"evolved from the author's courses and seminars at Stanford University and University of Massachusetts, Amherst, and is suitable for text use in a senior or graduate AI course, or for individual study.

Principles of Artificial Intelligence

Working Group Report for Informatics Europe. .. decades of stunningly rapid advances in processing speed, storage and networking, along with the development of increasingly clever software, have brought computing into science, business and culture in ways that were barely imagined years ago. The quantitative changes delivered through smart engineering opened the door to qualitative changes. Computing changes what can be seen, simulated and done. Summary Despite the fact that Computing/Informatics impacts on everything around us and is an unprecedented source of new qualities in science, in business and in our daily lives, the number of beginning students in informatics has been steadily dropping over the past years in many countries. The 'enrollment crisis' is cited as one of the prime reasons why science is not profiting from the achievements of computer science in depth, why industry is not able to recruit even a fraction of the highly skilled IT specialists and software engineers that it needs, and why the information society is deprived of the many beautiful intelligent systems that modern computer science could lead to. Universities in Europe are turning out excellent graduates in Informatics but are doing so in numbers that seem way too small. Why is student enrollment a problem in Informatics/Computing, at least in many of the (Western) European countries? Why is enrollment by female students lagging behind? What are the reasons of it, and what can be done about it? Are there best practices in certain countries from which we can all learn and benefit? Do potential students have the right image of Informatics as a field of study, as a science, as a profession? How should the field be positioned so it is " clearly " as attractive and challenging as many other disciplines and perhaps even more so? The Working Group on 'Student Enrollment and Image of the Discipline' was created to collect insights on these issues and to come up with a document that advises the Informatics Europe membership on the state of the problem and on possible measures that could be taken to resolve it. The present document is a very preliminary report on the results of the Working Group. This report is based on the contributions of the following members of the Working Group. The text developed initially through a wiki site provided by Informatics Europe.

/pdf/student-enrollment-and-image-of-the-informatics-discipline-1n189sti2o.pdf

Student Enrollment and Image of the Informatics Discipline

A few years ago J Bentley initiated a general approach to searching problems and their solution by means of dynamic data structures As it is often easier to find a static solution first, his goal was to obtain efficient dynamic data structures by applying transformations to static data structures This approach has become a paradigm (known as "dynamization") in current research in the design of efficient algorithms We shall outline a number of general techniques that were developed for dynamizing decomposable searching problems and will discuss a recent solution to Bentley's original question to devise a dynamization method for such problems with worst-case optimal insertion and deletion routines

The Art of Dynamizing

Geometric intersection graphs are graphs determined by intersections of geometric objects. We study the complexity of visualizing the arrangements of objects that induce such graphs. We give a general framework for describing geometric intersection graphs, using arbitrary finite base sets of rationally given convex polygons and affine transformations. We prove that for every class of intersection graphs that fits the framework, the graphs in the class have a representation using polynomially many bits. Consequently, the recognition problem of these classes is in NP (and thus NP-complete). We also give an algorithm to find a drawing of the objects in the plane, if a graph class fits the framework.

/pdf/convex-polygon-intersection-graphs-4w4pufl0tn.pdf

Convex polygon intersection graphs

We give the first lower bounds on the grid size needed to represent the intersection graphs of~convex polygons. Here each corner of a polygon in the representation must lie on a corner of the grid. We provide a series of geometric constructions showing that for intersection graphs of: translated copies of any fixed parallelogram, grids of size Ω(n2) x Ω(n2) are needed; translated copies of any other fixed convex polygon, grids of size 2Ω(n) x 2Ω(n) are needed; homothetic copies of any fixed convex polygon, grids of size 2Ω(n) x 2Ω(n) are needed.We complement these results by giving a matching upper bound in each case. Hence we settle the complexity of the integer representation problem for these graphs. The upper bounds substantially improve earlier bounds and extend to higher dimensions.

/pdf/integer-representations-of-convex-polygon-intersection-2x6i0c16yc.pdf

Integer representations of convex polygon intersection graphs

In this paper a data structure is presented to efficiently maintain the 2and 3-edge-connected components of a graph, under insertions of edges in the graph. Starting from an "empty" graph of n nodes, the insertion of e edges takes O( n log n + e) time in total. The data structure a.llows for insertions of nodes a.lso (in the same time bounds, taking n as the fina.l number of nodes). Moreover, at any moment, the data structure can answer the following type of query in 0(1) time: given two nodes in the graph, are these nodes 2or 3-edge-connected.

/pdf/maintenance-of-2-and-3-connected-components-of-graphs-part-i-573in23a9x.pdf

Jan van Leeuwen

Papers

Student Enrollment and Image of the Informatics Discipline

The Art of Dynamizing

Convex polygon intersection graphs

Integer representations of convex polygon intersection graphs

Maintenance of 2- and 3-connected components of graphs; Part I: 2- and 3-edge-connected components