Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

We describe latent Dirichlet allocation (LDA), a generative probabilistic model for collections of discrete data such as text corpora. LDA is a three-level hierarchical Bayesian model, in which each item of a collection is modeled as a finite mixture over an underlying set of topics. Each topic is, in turn, modeled as an infinite mixture over an underlying set of topic probabilities. In the context of text modeling, the topic probabilities provide an explicit representation of a document. We present efficient approximate inference techniques based on variational methods and an EM algorithm for empirical Bayes parameter estimation. We report results in document modeling, text classification, and collaborative filtering, comparing to a mixture of unigrams model and the probabilistic LSI model.

/pdf/latent-dirichlet-allocation-4gmscb06en.pdf

Latent dirichlet allocation

We propose a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams [6], and Hof-mann's aspect model, also known as probabilistic latent semantic indexing (pLSI) [3]. In the context of text modeling, our model posits that each document is generated as a mixture of topics, where the continuous-valued mixture proportions are distributed as a latent Dirichlet random variable. Inference and learning are carried out efficiently via variational algorithms. We present empirical results on applications of this model to problems in text modeling, collaborative filtering, and text classification.

/pdf/latent-dirichlet-allocation-rkwbqihm3p.pdf

Latent Dirichlet Allocation

Data Mining: Practical Machine Learning Tools and Techniques offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations. This highly anticipated third edition of the most acclaimed work on data mining and machine learning will teach you everything you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data mining. Thorough updates reflect the technical changes and modernizations that have taken place in the field since the last edition, including new material on Data Transformations, Ensemble Learning, Massive Data Sets, Multi-instance Learning, plus a new version of the popular Weka machine learning software developed by the authors. Witten, Frank, and Hall include both tried-and-true techniques of today as well as methods at the leading edge of contemporary research. *Provides a thorough grounding in machine learning concepts as well as practical advice on applying the tools and techniques to your data mining projects *Offers concrete tips and techniques for performance improvement that work by transforming the input or output in machine learning methods *Includes downloadable Weka software toolkit, a collection of machine learning algorithms for data mining tasks-in an updated, interactive interface. Algorithms in toolkit cover: data pre-processing, classification, regression, clustering, association rules, visualization

Data Mining: Practical Machine Learning Tools and Techniques

抗原变异可使得多种致病微生物易于逃避宿主免疫应答。表达在感染红细胞表面的恶性疟原虫红细胞表面蛋白1（PfPMP1）与感染红细胞、内皮细胞、树突状细胞以及胎盘的单个或多个受体作用，在黏附及免疫逃避中起关键的作用。每个单倍体基因组var基因家族编码约60种成员，通过启动转录不同的var基因变异体为抗原变异提供了分子基础。

疟原虫var基因转换速率变化导致抗原变异[英]／Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A

Appearance-based object detection systems using statistical models have proven quite successful. They can reliably detect textured, rigid objects in a variety of poses, lighting conditions and scales. However, the construction of these systems is time-consuming and difficult because a large number of training examples must be collected and manually labeled in order to capture variations in object appearance. Typically, this requires indicating which regions of the image correspond to the object to be detected, and which belong to background clutter, as well as marking key landmark locations on the object. The goal of this work is to pursue and evaluate approaches which reduce the amount of fully labeled examples needed, by training these models in a semi-supervised manner. To this end, we develop approaches based on Expectation-Maximization and self-training that utilize a small number of fully labeled training examples in combination with a set of “weakly labeled” examples. This is advantageous in that weakly labeled data are inherently less costly to generate, since the label information is specified in an uncertain or incomplete fashion. For example, a weakly labeled image might be labeled as containing the training object, with the object location and scale left unspecified. In this work we analyze the performance of the techniques developed through a comprehensive empirical investigation. We find that supplementing a small fully labeled training set with weakly labeled data in the training process reliably improves detector performance for a variety of detection approaches. The outcome is the identification of successful approaches and key issues that are central to achieving good performance in the semi-supervised training of object detection systems.

Semi-supervised training of models for appearance-based statistical object detection methods

Inversion of multilayer synchronous networks is a method which tries to answer questions like “What kind of input will give a desired output?” or “Is it possible to get a desired (output under special input/output constraints)?”.

Inversion in Time

Localization is one of the fundamental problems in mobile robotics. Without knowledge about their position mobile robots cannot efficiently carry out their tasks. In this paper we present Markov localization as a technique for estimating the position of a mobile robot. The key idea of this technique is to maintain a probability density over the whole state space of the robot within its environment. This way our technique is able to globally localize the robot from scratch and even to recover from localization failures, a property which is essential for truly autonomous robots. The probabilistic framework makes this approach robust against approximate models of the environment as well as noisy sensors. Based on a fine-grained, metric discretization of the state space, Markov localization is able to incorporate raw sensor readings and does not require predefined landmarks. It also includes a filtering technique which allows to reliably estimate the position of a mobile robot even in densely populated environments. We furthermore describe, how the explicit representation of the density can be exploited in a reactive collision avoidance system to increase the robustness and reliability of the robot even in situations in which it is uncertain about its position. The method described here has been implemented and tested in several real-world applications of mobile robots including the deployments of two mobile robots as interactive museum tour-guides.

Markov Localization for Reliable Robot Navigation and People Detection

Machine Learning- Real-World Robotics: Learning, to Plan for Robust Execution- Robot Programming by Demonstration (RPD): Supporting the Induction by Human Interaction- Performance Improvement of Robot Continuous-Path Operation through Iterative Learning Using Neural Networks- Learning Controllers for Industrial Robots- Active Learning for Vision-Based Robot Grasping- Purposive Behavior Acquisition for a Real Robot by Vision-Based Reinforcement Learning- Learning Concepts from Sensor Data of a Mobile Robot

Recent Advances in Robot Learning

The eld of Arti cial Intelligence (AI) is currently undergoing a transition. While in the eighties, rule-based and logical representations were the representation of choice in the majority of AI systems, in recent years various researchers have explored alternative representational frameworks, which emphasis on frameworks that enable systems to represent and handle uncertainty. Out of those, probabilistic methods (and speci cally Bayesian methods) have probably been analyzed most thoroughly and applied most successfully in a variety of problem domains.

Sebastian Thrun

Papers

Semi-supervised training of models for appearance-based statistical object detection methods

Inversion in Time

Markov Localization for Reliable Robot Navigation and People Detection

Recent Advances in Robot Learning

Probabilistic Methods for State Estimation in Robotics