A comprehensive and self-contained introduction to Gaussian processes, which provide a principled, practical, probabilistic approach to learning in kernel machines. Gaussian processes (GPs) provide a principled, practical, probabilistic approach to learning in kernel machines. GPs have received increased attention in the machine-learning community over the past decade, and this book provides a long-needed systematic and unified treatment of theoretical and practical aspects of GPs in machine learning. The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics. The book deals with the supervised-learning problem for both regression and classification, and includes detailed algorithms. A wide variety of covariance (kernel) functions are presented and their properties discussed. Model selection is discussed both from a Bayesian and a classical perspective. Many connections to other well-known techniques from machine learning and statistics are discussed, including support-vector machines, neural networks, splines, regularization networks, relevance vector machines and others. Theoretical issues including learning curves and the PAC-Bayesian framework are treated, and several approximation methods for learning with large datasets are discussed. The book contains illustrative examples and exercises, and code and datasets are available on the Web. Appendixes provide mathematical background and a discussion of Gaussian Markov processes.

/pdf/gaussian-processes-for-machine-learning-40fp9c190d.pdf

Gaussian Processes for Machine Learning

Introduction to support vector learning roadmap. Part 1 Theory: three remarks on the support vector method of function estimation, Vladimir Vapnik generalization performance of support vector machines and other pattern classifiers, Peter Bartlett and John Shawe-Taylor Bayesian voting schemes and large margin classifiers, Nello Cristianini and John Shawe-Taylor support vector machines, reproducing kernel Hilbert spaces, and randomized GACV, Grace Wahba geometry and invariance in kernel based methods, Christopher J.C. Burges on the annealed VC entropy for margin classifiers - a statistical mechanics study, Manfred Opper entropy numbers, operators and support vector kernels, Robert C. Williamson et al. Part 2 Implementations: solving the quadratic programming problem arising in support vector classification, Linda Kaufman making large-scale support vector machine learning practical, Thorsten Joachims fast training of support vector machines using sequential minimal optimization, John C. Platt. Part 3 Applications: support vector machines for dynamic reconstruction of a chaotic system, Davide Mattera and Simon Haykin using support vector machines for time series prediction, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel. Part 4 Extensions of the algorithm: reducing the run-time complexity in support vector machines, Edgar E. Osuna and Federico Girosi support vector regression with ANOVA decomposition kernels, Mark O. Stitson et al support vector density estimation, Jason Weston et al combining support vector and mathematical programming methods for classification, Bernhard Scholkopf et al.

Advances in kernel methods: support vector learning

Human activities are releasing tiny particles (aerosols) into the atmosphere. These human-made aerosols enhance scattering and absorption of solar radiation. They also produce brighter clouds that are less efficient at releasing precipitation. These in turn lead to large reductions in the amount of solar irradiance reaching Earth's surface, a corresponding increase in solar heating of the atmosphere, changes in the atmospheric temperature structure, suppression of rainfall, and less efficient removal of pollutants. These aerosol effects can lead to a weaker hydrological cycle, which connects directly to availability and quality of fresh water, a major environmental issue of the 21st century.

/pdf/aerosols-climate-and-the-hydrological-cycle-52bxtq0sr5.pdf

Aerosols, climate, and the hydrological cycle

The two main principles underlying the use of isentropic maps of potential vorticity to represent dynamical processes in the atmosphere are reviewed, including the extension of those principles to take the lower boundary condition into account. the first is the familiar Lagrangian conservation principle, for potential vorticity (PV) and potential temperature, which holds approximately when advective processes dominate frictional and diabatic ones. the second is the principle of ‘invertibility’ of the PV distribution, which holds whether or not diabatic and frictional processes are important. the invertibility principle states that if the total mass under each isentropic surface is specified, then a knowledge of the global distribution of PV on each isentropic surface and of potential temperature at the lower boundary (which within certain limitations can be considered to be part of the PV distribution) is sufficient to deduce, diagnostically, all the other dynamical fields, such as winds, temperatures, geopotential heights, static stabilities, and vertical velocities, under a suitable balance condition. the statement that vertical velocities can be deduced is related to the well-known omega equation principle, and depends on having sufficient information about diabatic and frictional processes. Quasi-geostrophic, semigeostrophic, and ‘nonlinear normal mode initialization’ realizations of the balance condition are discussed. an important constraint on the mass-weighted integral of PV over a material volume and on its possible diabatic and frictional change is noted.

Some basic examples are given, both from operational weather analyses and from idealized theoretical models, to illustrate the insights that can be gained from this approach and to indicate its relation to classical synoptic and air-mass concepts. Included are discussions of (a) the structure, origin and persistence of cutoff cyclones and blocking anticyclones, (b) the physical mechanisms of Rossby wave propagation, baroclinic instability, and barotropic instability, and (c) the spatially and temporally nonuniform way in which such waves and instabilities may become strongly nonlinear, as in an occluding cyclone or in the formation of an upper air shear line. Connections with principles derived from synoptic experience are indicated, such as the ‘PVA rule’ concerning positive vorticity advection on upper air charts, and the role of disturbances of upper air origin, in combination with low-level warm advection, in triggering latent heat release to produce explosive cyclonic development. In all cases it is found that time sequences of isentropic potential vorticity and surface potential temperature charts—which succinctly summarize the combined effects of vorticity advection, thermal advection, and vertical motion without requiring explicit knowledge of the vertical motion field—lead to a very clear and complete picture of the dynamics. This picture is remarkably simple in many cases of real meteorological interest. It involves, in principle, no sacrifices in quantitative accuracy beyond what is inherent in the concept of balance, as used for instance in the initialization of numerical weather forecasts.

/pdf/on-the-use-and-significance-of-isentropic-potential-wx69wazwtl.pdf

On the use and significance of isentropic potential vorticity maps

We describe the maximum-likelihood parameter estimation problem and how the ExpectationMaximization (EM) algorithm can be used for its solution. We first describe the abstract form of the EM algorithm as it is often given in the literature. We then develop the EM parameter estimation procedure for two applications: 1) finding the parameters of a mixture of Gaussian densities, and 2) finding the parameters of a hidden Markov model (HMM) (i.e., the Baum-Welch algorithm) for both discrete and Gaussian mixture observation models. We derive the update equations in fairly explicit detail but we do not prove any convergence properties. We try to emphasize intuition rather than mathematical rigor.

/pdf/a-gentle-tutorial-of-the-em-algorithm-and-its-application-to-4o5uraouqh.pdf

A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models

New and/or improved variational methods for simultaneously combining forecast, heterogeneous observational data, a priori climatology, and physics to obtain improved estimates of the initial state of the atmosphere for the purpose of numerical weather prediction are developed. Cross validated spline methods are applied to atmospheric data for the purpose of improved description and analysis of atmospheric phenomena such as the tropopause and frontal boundary surfaces.

Some new mathematical methods for variational objective analysis

Transverse circulations in the exit and entrance regions of jet streaks are investigated through numerical simulation, a case study, and an application of the isallobaric wind equation in isentropic coordinates, to study the interaction between upper and lower tropospheric jets and the development of severe convective storms. A hybrid isentropic-sigma coordinate numerical model is used to simulate the mass and momentum adjustments associated with a jet streak propagating in a zonal channel. The numerical results depict a two-layer mass adjustment in the exit and entrance region of the jet streak. The results also verify that the isallobaric wind on lower isentropic surfaces is a primary component of the return branches of transverse circulations and is foxed by the two-layer mass adjustment accompanying the propagating jet streak. Results from the case study of a severe weather out- break show that 1) a low-level jet (LLJ) beneath the exit region of an upper tropospheric jet streak is embedded in...

The Coupling of Upper and Lower Tropospheric Jet Streaks and Implications for the Development of Severe Convective Storms

Vasoactive intestinal peptide (VIP) labeled with 125I, [Tyr10-125I]VIP, can be hydrolyzed by immunoglobulin G (IgG) purified from a human subject, as judged by trichloroacetic acid precipitation and reversed-phase high-performance liquid chromatography (HPLC). The hydrolytic activity was precipitated by antibody to human IgG, it was bound by immobilized protein G and showed a molecular mass close to 150 kilodaltons by gel filtration chromatography, properties similar to those of authentic IgG. The Fab fragment, prepared from IgG by papain treatment, retained the VIP hydrolytic activity of the IgG. Peptide fragments produced by treatment of VIP with the antibody fraction were purified by reversed-phase HPLC and identified by fast atom bombardment-mass spectrometry and peptide sequencing. The scissile bond in VIP deduced from these experiments was Gln16-Met17. The antibody concentration (73.4 fmol per milligram of IgG) and the Kd (0.4 nM) were computed from analysis of VIP binding under conditions that did not result in peptide hydrolysis. Analysis of the antibody-mediated VIP hydrolysis at varying concentrations of substrate suggested conformity with Michaelis-Menton kinetics (Km). The values for Km (37.9 X 10(-9) M) and the turnover number kcat (15.6 min-1) suggested relatively tight VIP binding and a moderate catalytic efficiency of the antibody.

https://science.sciencemag.org/content/sci/244/4909/1158.full.pdf

Catalytic Hydrolysis of Vasoactive Intestinal Peptide by Human Autoantibody

The seasonal formation of a large (500–800 km diameter) anticyclonic eddy in the upper 300–400 m of the eastern Arabian Sea during the northeast monsoon period (December-April) is indicated from hydrographic and satellite altimetry sea level observations, as well as from numerical model experiments. The center of the eddy circulation is approximately 10°N, 70°E, just to the west of the north-south Laccadive Island chain. In this paper the eddy is called the Laccadive High (LH). In some ways it is a mirrorlike counterpart to the Great Whirl, which develops during the southwest monsoon off the Somali coast (western Arabian Sea). The LH occurs at the same latitude but on the opposite side of the basin during the reversed monsoon. It is different from the Great Whirl, however, in its formation process, its intensity, and its decay. The hydrographic data obtained from surveys all during a single season give sufficiently close station spacing to allow reasonable contouring of the geopotential surfaces and of the properties within and around the LH region with minimum time aliasing. The Geosat altimeter record extends over 4 years, during which the seasonal variability of the LH indicates a dynamic relief of approximately 15–20 cm, which is in good agreement with the hydrographic observations. The altimetry time series also suggests a westward translation of the LH by January with a subsequent dissipation in midbasin. The model used is a wind-forced three-layer primitive equation model which depicts a LH in agreement with the timing, position, and amplitude of both the hydrographic and altimetric measurements. The numerical simulation includes a passive tracer located in the western Bay of Bengal; the western advection of the tracer around the south coasts of Sri Lanka and India in December and January is consistent with the appearance of low-salinity water observed to extend into the Arabian Sea during this period. The modeling studies suggest that both local and remote forcing are important in formation of the LH.

Evidence for eddy formation in the eastern Arabian Sea during the northeast monsoon

Publisher Summary The concept of available potential energy is founded upon the principle of conservation of mass and the idealization that flows, which conserve specific entropy, may exist Under these conditions, the sum of the internal, potential, and kinetic energies is a constant, and; therefore, a state of the atmosphere, which possesses a minimum of total potential energy will likewise have a maximum of kinetic energy The purpose of this chapter to: (1) Develop the theory of available potential energy and its rate of change rigorously and exactly, and thereby clarify the roles of convection, hydrostatic departures, static stability, and diabatic processes (2) Investigate the relationships between the concepts of available potential energy, static stability, and entropy, and investigate contributions to available energy under various circumstances (3) Demonstrate that the methods of the calculus of variations provide an economical method of obtaining exact results in available energy theory (4) Propose an application of variational techniques, which offer considerable intuitive insight and the possibility of analytic and numerical attacks on the problem of why the atmosphere chooses the modes of circulation it does (5) Illustrate the role of available potential energy in maintaining the circulation in the context of the proposed theory of the control of the general circulation Studies of energy transformations in general and those involving available potential energy in particular, can never yield information about the control of the circulation unless they are coupled with a dynamic theory By themselves, such studies are of necessity diagnostic and kinematic, and can yield only a detailed understanding of the mechanisms by which energy is conserved The discovery and application of variational and integral techniques in atmospheric research is just beginning, but their success in other fields is sufficient guarantee that their exploitation will dispel many mysteries

Donald R. Johnson

Papers

Some new mathematical methods for variational objective analysis

The Coupling of Upper and Lower Tropospheric Jet Streaks and Implications for the Development of Severe Convective Storms

Catalytic Hydrolysis of Vasoactive Intestinal Peptide by Human Autoantibody

Evidence for eddy formation in the eastern Arabian Sea during the northeast monsoon

The Theory of Available Potential Energy and a Variational Approach to Atmospheric Energetics