Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Machine Learning : A Probabilistic Perspective

Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

A new approach toward target representation and localization, the central component in visual tracking of nonrigid objects, is proposed. The feature histogram-based target representations are regularized by spatial masking with an isotropic kernel. The masking induces spatially-smooth similarity functions suitable for gradient-based optimization, hence, the target localization problem can be formulated using the basin of attraction of the local maxima. We employ a metric derived from the Bhattacharyya coefficient as similarity measure, and use the mean shift procedure to perform the optimization. In the presented tracking examples, the new method successfully coped with camera motion, partial occlusions, clutter, and target scale variations. Integration with motion filters and data association techniques is also discussed. We describe only a few of the potential applications: exploitation of background information, Kalman tracking using motion models, and face tracking.

/pdf/kernel-based-object-tracking-2pj4gwhvr7.pdf

Kernel-based object tracking

In this book, Andrew Harvey sets out to provide a unified and comprehensive theory of structural time series models. Unlike the traditional ARIMA models, structural time series models consist explicitly of unobserved components, such as trends and seasonals, which have a direct interpretation. As a result the model selection methodology associated with structural models is much closer to econometric methodology. The link with econometrics is made even closer by the natural way in which the models can be extended to include explanatory variables and to cope with multivariate time series. From the technical point of view, state space models and the Kalman filter play a key role in the statistical treatment of structural time series models. The book includes a detailed treatment of the Kalman filter. This technique was originally developed in control engineering, but is becoming increasingly important in fields such as economics and operations research. This book is concerned primarily with modelling economic and social time series, and with addressing the special problems which the treatment of such series poses. The properties of the models and the methodological techniques used to select them are illustrated with various applications. These range from the modellling of trends and cycles in US macroeconomic time series to to an evaluation of the effects of seat belt legislation in the UK.

Forecasting, Structural Time Series Models and the Kalman Filter

Twitter, a popular microblogging service, has received much attention recently. An important characteristic of Twitter is its real-time nature. For example, when an earthquake occurs, people make many Twitter posts (tweets) related to the earthquake, which enables detection of earthquake occurrence promptly, simply by observing the tweets. As described in this paper, we investigate the real-time interaction of events such as earthquakes in Twitter and propose an algorithm to monitor tweets and to detect a target event. To detect a target event, we devise a classifier of tweets based on features such as the keywords in a tweet, the number of words, and their context. Subsequently, we produce a probabilistic spatiotemporal model for the target event that can find the center and the trajectory of the event location. We consider each Twitter user as a sensor and apply Kalman filtering and particle filtering, which are widely used for location estimation in ubiquitous/pervasive computing. The particle filter works better than other comparable methods for estimating the centers of earthquakes and the trajectories of typhoons. As an application, we construct an earthquake reporting system in Japan. Because of the numerous earthquakes and the large number of Twitter users throughout the country, we can detect an earthquake with high probability (96% of earthquakes of Japan Meteorological Agency (JMA) seismic intensity scale 3 or more are detected) merely by monitoring tweets. Our system detects earthquakes promptly and sends e-mails to registered users. Notification is delivered much faster than the announcements that are broadcast by the JMA.

/pdf/earthquake-shakes-twitter-users-real-time-event-detection-by-1tue7cvj9p.pdf

Earthquake shakes Twitter users: real-time event detection by social sensors

Standard target state estimation schemes typically use detections as their source of measurements, which are pro- duced by thresholding the output of a sensor's signal processing stage. This work exploits a track-before-detect (TBD) technique, which simultaneously detects and tracks a target without needing to threshold the sensor data. By removing the need for threshold- ing, TBD can potentially detect and track targets with a much lower signal-to-noise ratio (SNR) than conventional systems. The signal processing that is modelled in this work is designed to match that which might be found in a sensor such as radar. In these systems, the data used by a tracking filter is the magnitude of a complex spectrum. This gives rise to a signal in Rayleigh distributed noise. This paper presents a particle-based TBD fil- ter, operating on the output of this signal processing stage, which estimates the target state incorporating target existence, position, velocity and target signal strength. The main contributions of this work are the development of an efficient method of calculating the probability of target existence and the derivation of a TBD filter which operates in Rayleigh noise. It is shown through simulation that a target can be detected and tracked with an SNR as low as 3dB. The work of Boers and Driessen (4) develops a TBD al- gorithm similar in structure to Ristic but extended to con- sider multiple targets. The algorithm presented in this paper is based heavily on that of Ristic, with the main differences being that the likelihood function is modified to more ac- curately match radar signal processing and a more efficient method for calculating the probability of target existence is developed. In this work the simulated data used as the input to the tracker is a uniform grid, with intensities defined for each bin. The intensities are modelled as the magnitude of a complex spectrum, which corresponds to the output of sig- nal processing stages in operational sensors such as radar. The envelope of complex Gaussian noise is Rayleigh dis- tributed and the magnitude of a signal in complex Gaussian noise is Ricean distributed (5). These densities are properly incorporated into the likelihood function of the TBD filter. This paper introduces a TBD algorithm for processing data with an efficient method for calculating the probability of target existence. The algorithm simultaneously estim- ates target parameters including existence, position, velo- city and signal intensity. Section 2 sets the problem math- ematically, with a summary of the models used for target motion and the assumed signal processing model. Section 3 derives the filter, incorporating target existence, while Sec- tion 4 details the implementation of this filter using sequen- tial Monte-Carlo techniques. Section 5 gives details of the simulation used to test the filter and shows results gained from repeated simulation. The final section summarises the key contributions of this paper and offers further opportun- ities for research.

Efficient Particle-Based Track-Before-Detect in Rayleigh Noise

Detecting adverse drug reactions (ADRs) is an important task that has direct implications for the use of that drug. If we can detect previously unknown ADRs as quickly as possible, then this information can be provided to the regulators, pharmaceutical companies, and health care organizations, thereby potentially reducing drug-related morbidity and saving lives of many patients. A promising approach for detecting ADRs is to use social media platforms such as Twitter and Facebook. A high level of correlation between a drug name and an event may be an indication of a potential adverse reaction associated with that drug. Although numerous association measures have been proposed by the signal detection community for identifying ADRs, these measures are limited in that they detect correlations but often ignore causality.This study aimed to propose a causality measure that can detect an adverse reaction that is caused by a drug rather than merely being a correlated signal.To the best of our knowledge, this was the first causality-sensitive approach for detecting ADRs from social media. Specifically, the relationship between a drug and an event was represented using a set of automatically extracted lexical patterns. We then learned the weights for the extracted lexical patterns that indicate their reliability for expressing an adverse reaction of a given drug.Our proposed method obtains an ADR detection accuracy of 74% on a large-scale manually annotated dataset of tweets, covering a standard set of drugs and adverse reactions.By using lexical patterns, we can accurately detect the causality between drugs and adverse reaction-related events.

/pdf/causality-patterns-for-detecting-adverse-drug-reactions-from-2c9as6vbm6.pdf

Causality Patterns for Detecting Adverse Drug Reactions From Social Media: Text Mining Approach.

Multi-target out-of-sequence data association: Tracking using graphical models

Semi-Markov models are a generalisation of Markov models that explicitly model the state-dependent sojourn time distribution, the time for which the system remains in a given state. Markov models result in an exponentially distributed sojourn time, while semi-Markov models make it possible to define the distribution explicitly. Such models can be used to describe the behaviour of manoeuvring targets, and particle filtering can then facilitate tracking. An architecture is proposed that enables particle filters to be both robust and efficient when conducting joint tracking and classification. It is demonstrated that this approach can be used to classify targets on the basis of their manoeuvrability.

/pdf/joint-tracking-of-manoeuvring-targets-and-classification-of-38pl2cci82.pdf

Joint tracking of manoeuvring targets and classification of their manoeuvrability

Particle filters are often claimed to be readily parallelisable However, the resampling step is non-trivial to implement in a fine-grained parallel architecture While approaches have been proposed that modify the particle filter to be amenable to such implementation, this paper's novelty lies in its description of a Single Instruction Multiple Data (SIMD) implementation of a particle filter that uses N processors to process N particles The resulting algorithm has a time complexity of O((log N)2) when performing resampling using N processors The algorithm has been implemented using C for Graphics (CG), a language that enables the heavily pipelined architecture of modern graphics cards to be used to imitate a SIMD processor Initial results are presented

Simon Maskell

Papers

Efficient Particle-Based Track-Before-Detect in Rayleigh Noise

Causality Patterns for Detecting Adverse Drug Reactions From Social Media: Text Mining Approach.

Multi-target out-of-sequence data association: Tracking using graphical models

Joint tracking of manoeuvring targets and classification of their manoeuvrability

A Single Instruction Multiple Data Particle Filter