Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

In this book, Andrew Harvey sets out to provide a unified and comprehensive theory of structural time series models. Unlike the traditional ARIMA models, structural time series models consist explicitly of unobserved components, such as trends and seasonals, which have a direct interpretation. As a result the model selection methodology associated with structural models is much closer to econometric methodology. The link with econometrics is made even closer by the natural way in which the models can be extended to include explanatory variables and to cope with multivariate time series. From the technical point of view, state space models and the Kalman filter play a key role in the statistical treatment of structural time series models. The book includes a detailed treatment of the Kalman filter. This technique was originally developed in control engineering, but is becoming increasingly important in fields such as economics and operations research. This book is concerned primarily with modelling economic and social time series, and with addressing the special problems which the treatment of such series poses. The properties of the models and the methodological techniques used to select them are illustrated with various applications. These range from the modellling of trends and cycles in US macroeconomic time series to to an evaluation of the effects of seat belt legislation in the UK.

Forecasting, Structural Time Series Models and the Kalman Filter

[1] Surrogate modeling, also called metamodeling, has evolved and been extensively used over the past decades. A wide variety of methods and tools have been introduced for surrogate modeling aiming to develop and utilize computationally more efficient surrogates of high-fidelity models mostly in optimization frameworks. This paper reviews, analyzes, and categorizes research efforts on surrogate modeling and applications with an emphasis on the research accomplished in the water resources field. The review analyzes 48 references on surrogate modeling arising from water resources and also screens out more than 100 references from the broader research community. Two broad families of surrogates namely response surface surrogates, which are statistical or empirical data-driven models emulating the high-fidelity model responses, and lower-fidelity physically based surrogates, which are simplified models of the original system, are detailed in this paper. Taxonomies on surrogate modeling frameworks, practical details, advances, challenges, and limitations are outlined. Important observations and some guidance for surrogate modeling decisions are provided along with a list of important future research directions that would benefit the common sampling and search (optimization) analyses found in water resources.

https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2011WR011527

Review of surrogate modeling in water resources

The design and implementation of effective environmental policies need to be informed by a holistic understanding of the system processes (biophysical, social and economic), their complex interactions, and how they respond to various changes. Models, integrating different system processes into a unified framework, are seen as useful tools to help analyse alternatives with stakeholders, assess their outcomes, and communicate results in a transparent way. This paper reviews five common approaches or model types that have the capacity to integrate knowledge by developing models that can accommodate multiple issues, values, scales and uncertainty considerations, as well as facilitate stakeholder engagement. The approaches considered are: systems dynamics, Bayesian networks, coupled component models, agent-based models and knowledge-based models (also referred to as expert systems). We start by discussing several considerations in model development, such as the purpose of model building, the availability of qualitative versus quantitative data for model specification, the level of spatio-temporal detail required, and treatment of uncertainty. These considerations and a review of applications are then used to develop a framework that aims to assist modellers and model users in the choice of an appropriate modelling approach for their integrated assessment applications and that enables more effective learning in interdisciplinary settings. We review five common integrated modelling approaches.Model choice considers purpose, data type, scale and uncertainty treatment.We present a guiding framework for selecting the most appropriate approach.

Selecting among five common modelling approaches for integrated environmental assessment and management

The development and application of evolutionary algorithms (EAs) and other metaheuristics for the optimisation of water resources systems has been an active research field for over two decades. Research to date has emphasized algorithmic improvements and individual applications in specific areas (e.g. model calibration, water distribution systems, groundwater management, river-basin planning and management, etc.). However, there has been limited synthesis between shared problem traits, common EA challenges, and needed advances across major applications. This paper clarifies the current status and future research directions for better solving key water resources problems using EAs. Advances in understanding fitness landscape properties and their effects on algorithm performance are critical. Future EA-based applications to real-world problems require a fundamental shift of focus towards improving problem formulations, understanding general theoretic frameworks for problem decompositions, major advances in EA computational efficiency, and most importantly aiding real decision-making in complex, uncertain application contexts.

/pdf/evolutionary-algorithms-and-other-metaheuristics-in-water-395cpaoum8.pdf

Evolutionary algorithms and other metaheuristics in water resources

Emulation modelling is an effective way of overcoming the large computational burden associated with the process-based models traditionally adopted by the environmental modelling community. An emulator is a low-order, computationally efficient model identified from the original large model and then used to replace it for computationally intensive applications. As the number and forms of the problem that benefit from the identification and subsequent use of an emulator is very large, emulation modelling has emerged in different sectors of science, engineering and social science. For this reason, a variety of different strategies and techniques have been proposed in the last few years. The main aim of the paper is to provide an introduction to emulation modelling, together with a unified strategy for its application, so that modellers from different disciplines can better appreciate how it may be applied in their area of expertise. Particular emphasis is devoted to Dynamic Emulation Modelling (DEMo), a methodological approach that preserves the dynamic nature of the original process-based model, with consequent advantages in a wide variety of problem areas. The different techniques and approaches to DEMo are considered in two macro categories: structure-based methods, where the mathematical structure of the original model is manipulated to a simpler, more computationally efficient form; and data-based approaches, where the emulator is identified and estimated from a data-set generated from planned experiments conducted on the large simulation model. The main contribution of the paper is a unified, six-step procedure that can be applied to most kinds of dynamic emulation problem.

Position Paper: A general framework for Dynamic Emulation Modelling in environmental problems

[1] Although being one of the most popular and extensively studied approaches to design water reservoir operations, Stochastic Dynamic Programming is plagued by a dual curse that makes it unsuitable to cope with large water systems: the computational requirement grows exponentially with the number of state variables considered (curse of dimensionality) and an explicit model must be available to describe every system transition and the associated rewards/costs (curse of modeling). A variety of simplifications and approximations have been devised in the past, which, in many cases, make the resulting operating policies inefficient and of scarce relevance in practical contexts. In this paper, a reinforcement-learning approach, called fitted Q-iteration, is presented: it combines the principle of continuous approximation of the value functions with a process of learning off-line from experience to design daily, cyclostationary operating policies. The continuous approximation, performed via tree-based regression, makes it possible to mitigate the curse of dimensionality by adopting a very coarse discretization grid with respect to the dense grid required to design an equally performing policy via Stochastic Dynamic Programming. The learning experience, in the form of a data set generated combining historical observations and model simulations, allows us to overcome the curse of modeling. Lake Como water system (Italy) is used as study site to infer general guidelines on the appropriate setting for the algorithm parameters and to demonstrate the advantages of the approach in terms of accuracy and computational effectiveness compared to traditional Stochastic Dynamic Programming.

https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2009WR008898

Tree-based reinforcement learning for optimal water reservoir operation

The advent of online services, social networks, crowdsourcing, and serious Web games has promoted the emergence of a novel computation paradigm, where complex tasks are solved by exploiting the capacity of human beings and computer platforms in an integrated way. Water Resources Management systems can take advantage of human and social computation in several ways: collecting and validating data, complementing the analytic knowledge embodied in models with tacit knowledge from individuals and communities, using human sensors to monitor the variation of conditions at a fine grain and in real time, activating human networks to perform search tasks or actuate management actions. This exploratory paper overviews different forms of human and social computation and analyzes how they can be exploited to enhance the effectiveness of ICT-based Water Resources Management.

Putting humans in the loop: Social computing for Water Resources Management

The optimal management of large environmental systems is often limited by the high computational burden associated to the process-based models commonly adopted to describe such systems. In this paper we propose a novel data-driven Dynamic Emulation Modelling approach for the construction of small, computationally efficient models that accurately emulate the main dynamics of the original process-based model, but with less computational requirements. The approach combines the many advantages of data-based modelling in representing complex, non-linear relationships, but preserves the state-space representation, which is both particularly effective in several applications (e.g. optimal management and data assimilation) and facilitates the ex-post physical interpretation of the emulator structure, thus enhancing the credibility of the model to stakeholders and decision-makers. The core mechanism is a novel variable selection procedure that is recursively applied to a data-set of input, state and output variables generated via simulation of the process-based model. The approach is demonstrated on a real-world case study concerning the optimal operation of a selective withdrawal reservoir (Tono Dam, Japan) suffering from downstream water quality problems. The emulator is identified on a data-set generated with a 1D coupled hydrodynamic-ecological model and subsequently used to design the optimal operating policy for the dam. Preliminary results show that the proposed approach significantly simplifies the learning of good operating policies and can highlight interesting properties of the system to be controlled.

Data-driven dynamic emulation modelling for the optimal management of environmental systems

This study presents a new interactive procedure for supporting Decision Makers (DMs) in environmental planning problems involving large, process-based, dynamic models and many (more than two) conflicting objectives. Because of such features of the model, computationally-onerous simulations are the only feasible way of analysis, while the multi-objective nature of the problem entails the combined use of optimization techniques and appropriate tools for the visualization of the associated Pareto frontier. The procedure proposed is based on the iterative improvement of the current best compromise alternative based on interactions with the DM. At each iteration, the DM is informed about the Pareto frontier of a local multi-objective optimization problem, which is generated by linearizing the response surfaces that describe the objectives and constraints of the original planning problem. Interactive visualization of the multi-dimensional Pareto frontier is used to support the DM in choosing the new best compromise alternative. The procedure terminates when the DM is fully satisfied with the current best compromise alternative. The approach is demonstrated in Googong Reservoir (Australia), which is periodically affected by high concentrations of Manganese and Cyanobacteria. Results indicate that substantial improvements could be observed by simply changing the location of the two mixers installed in 2007 and adding another pair of mixers.

R. Soncini-Sessa

Papers

Position Paper: A general framework for Dynamic Emulation Modelling in environmental problems

Tree-based reinforcement learning for optimal water reservoir operation

Putting humans in the loop: Social computing for Water Resources Management

Data-driven dynamic emulation modelling for the optimal management of environmental systems

Visualization-based multi-objective improvement of environmental decision-making using linearization of response surfaces