Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Machine Learning : A Probabilistic Perspective

Accurate and low-cost sensor localization is a critical requirement for the deployment of wireless sensor networks in a wide variety of applications. In cooperative localization, sensors work together in a peer-to-peer manner to make measurements and then forms a map of the network. Various application requirements influence the design of sensor localization systems. In this article, the authors describe the measurement-based statistical models useful to describe time-of-arrival (TOA), angle-of-arrival (AOA), and received-signal-strength (RSS) measurements in wireless sensor networks. Wideband and ultra-wideband (UWB) measurements, and RF and acoustic media are also discussed. Using the models, the authors have shown the calculation of a Cramer-Rao bound (CRB) on the location estimation precision possible for a given set of measurements. The article briefly surveys a large and growing body of sensor localization algorithms. This article is intended to emphasize the basic statistical signal processing background necessary to understand the state-of-the-art and to make progress in the new and largely open areas of sensor network localization research.

Locating the nodes: cooperative localization in wireless sensor networks

The purpose of this paper is to provide a comprehensive presentation and interpretation of the Ensemble Kalman Filter (EnKF) and its numerical implementation. The EnKF has a large user group, and numerous publications have discussed applications and theoretical aspects of it. This paper reviews the important results from these studies and also presents new ideas and alternative interpretations which further explain the success of the EnKF. In addition to providing the theoretical framework needed for using the EnKF, there is also a focus on the algorithmic formulation and optimal numerical implementation. A program listing is given for some of the key subroutines. The paper also touches upon specific issues such as the use of nonlinear measurements, in situ profiles of temperature and salinity, and data which are available with high frequency in time. An ensemble based optimal interpolation (EnOI) scheme is presented as a cost-effective approach which may serve as an alternative to the EnKF in some applications. A fairly extensive discussion is devoted to the use of time correlated model errors and the estimation of model bias.

The Ensemble Kalman Filter: Theoretical formulation and practical implementation

The paper discusses the theory behind the dual-tree transform, shows how complex wavelets with good properties can be designed, and illustrates a range of applications in signal and image processing The authors use the complex number symbol C in CWT to avoid confusion with the often-used acronym CWT for the (different) continuous wavelet transform The four fundamentals, intertwined shortcomings of wavelet transform and some solutions are also discussed Several methods for filter design are described for dual-tree CWT that demonstrates with relatively short filters, an effective invertible approximately analytic wavelet transform can indeed be implemented using the dual-tree approach

/pdf/the-dual-tree-complex-wavelet-transform-4kfm5jwrrq.pdf

The dual-tree complex wavelet transform

A survey of recent developments in the theory and application of com- posite likelihood is provided, building on the review paper of Varin(2008). A range of application areas, including geostatistics, spatial extremes, and space-time mod- els, as well as clustered and longitudinal data and time series are considered. The important area of applications to statistical genetics is omitted, in light ofLarribe and Fearnhead(2011). Emphasis is given to the development of the theory, and the current state of knowledge on e!ciency and robustness of composite likelihood inference.

/pdf/an-overview-of-composite-likelihood-methods-2lgbmk7ee2.pdf

An overview of composite likelihood methods

Today's Internet is a massive, distributed network which contin- ues to explode in size as e-commerce and related activities grow. The hetero- geneous and largely unregulated structure of the Internet renders tasks such as dynamic routing, optimized service provision, service level verification and detection of anomalous/malicious behavior extremely challenging. The problem is compounded by the fact that one cannot rely on the cooperation of individual servers and routers to aid in the collection of network traffic measurements vital for these tasks. In many ways, network monitoring and inference problems bear a strong resemblance to other "inverse problems" in which key aspects of a system are not directly observable. Familiar sig- nal processing or statistical problems such as tomographic image reconstruc- tion and phylogenetic tree identification have interesting connections to those arising in networking. This article introduces network tomography, a new field which we believe will benefit greatly from the wealth of statistical the- ory and algorithms. It focuses especially on recent developments in the field including the application of pseudo-likelihood methods and tree estimation formulations.

/pdf/network-tomography-recent-developments-2sa7wg7tlp.pdf

Network Tomography: Recent Developments

This paper describes two methodologies for performing distributed particle filtering in a sensor network. It considers the scenario in which a set of sensor nodes make multiple, noisy measurements of an underlying, time-varying state that describes the monitored system. The goal of the proposed algorithms is to perform on-line, distributed estimation of the current state at multiple sensor nodes, whilst attempting to minimize communication overhead. The first algorithm relies on likelihood factorization and the training of parametric models to approximate the likelihood factors. The second algorithm adds a predictive scalar quantizer training step into the more standard particle filtering framework, allowing adaptive encoding of the measurements. As its primary example, the paper describes the application of the quantization-based algorithm to tracking a maneuvering object. The paper concludes with a discussion of the limitations of the presented technique and an indication of future avenues for enhancement.

/pdf/distributed-particle-filters-for-sensor-networks-3u1v3c4dzz.pdf

Distributed particle filters for sensor networks

In this paper, we develop algorithms for distributed computation of averages of the node data over networks with bandwidth/power constraints or large volumes of data. Distributed averaging algorithms fail to achieve consensus when deterministic uniform quantization is adopted. We propose a distributed algorithm in which the nodes utilize probabilistically quantized information, i.e., dithered quantization, to communicate with each other. The algorithm we develop is a dynamical system that generates sequences achieving a consensus at one of the quantization values almost surely. In addition, we show that the expected value of the consensus is equal to the average of the original sensor data. We derive an upper bound on the mean-square-error performance of the probabilistically quantized distributed averaging (PQDA). Moreover, we show that the convergence of the PQDA is monotonic by studying the evolution of the minimum-length interval containing the node values. We reveal that the length of this interval is a monotonically nonincreasing function with limit zero. We also demonstrate that all the node values, in the worst case, converge to the final two quantization bins at the same rate as standard unquantized consensus. Finally, we report the results of simulations conducted to evaluate the behavior and the effectiveness of the proposed algorithm in various scenarios.

/pdf/distributed-average-consensus-with-dithered-quantization-4yucqluxem.pdf

Distributed Average Consensus With Dithered Quantization

Network tomography is a process for inferring "internal" link-level delay and loss performance information based on end-to-end (edge) network measurements. These methods require knowledge of the network topology; therefore a first crucial step in the tomography process is topology identification. This paper considers the problem of discovering network topology solely from host-based, unicast measurements, without internal network cooperation. First, we introduce a novel delay-based measurement scheme that does not require clock synchronization, making it more practical than other previous proposals. In contrast to methods that rely on network cooperation , our methodology has the potential to identify layer two elements (provided they are logical topology branching points and induce some measurable delay). Second, we propose a maximum penalized likelihood criterion for topology identification. This is a global optimality criterion, in contrast to other recent proposals for topology identification that employ suboptimal, pair-merging strategies. We develop a novel Markov Chain Monte Carlo (MCMC) procedure for rapid determination of the most likely topologies. The performance of our new probing scheme and identification algorithm is explored through simulation and Internet experiments.

/pdf/maximum-likelihood-network-topology-identification-from-edge-4fzxz44rzh.pdf

Mark Coates

Papers

Network Tomography: Recent Developments

Distributed particle filters for sensor networks

Distributed Average Consensus With Dithered Quantization

Maximum likelihood network topology identification from edge-based unicast measurements

Multifractal Cross-Traffic Estimation