Bayesian inference has found widespread application and use in science and engineering to reconcile Earth system models with data, including prediction in space (interpolation), prediction in time (forecasting), assimilation of observations and deterministic/stochastic model output, and inference of the model parameters. Bayes theorem states that the posterior probability, p ( H | Y ˜ ) of a hypothesis, H is proportional to the product of the prior probability, p(H) of this hypothesis and the likelihood, L ( H | Y ˜ ) of the same hypothesis given the new observations, Y ˜ , or p ( H | Y ˜ ) ∝ p ( H ) L ( H | Y ˜ ) . In science and engineering, H often constitutes some numerical model, ℱ(x) which summarizes, in algebraic and differential equations, state variables and fluxes, all knowledge of the system of interest, and the unknown parameter values, x are subject to inference using the data Y ˜ . Unfortunately, for complex system models the posterior distribution is often high dimensional and analytically intractable, and sampling methods are required to approximate the target. In this paper I review the basic theory of Markov chain Monte Carlo (MCMC) simulation and introduce a MATLAB toolbox of the DiffeRential Evolution Adaptive Metropolis (DREAM) algorithm developed by Vrugt et al. (2008a, 2009a) and used for Bayesian inference in fields ranging from physics, chemistry and engineering, to ecology, hydrology, and geophysics. This MATLAB toolbox provides scientists and engineers with an arsenal of options and utilities to solve posterior sampling problems involving (among others) bimodality, high-dimensionality, summary statistics, bounded parameter spaces, dynamic simulation models, formal/informal likelihood functions (GLUE), diagnostic model evaluation, data assimilation, Bayesian model averaging, distributed computation, and informative/noninformative prior distributions. The DREAM toolbox supports parallel computing and includes tools for convergence analysis of the sampled chain trajectories and post-processing of the results. Seven different case studies illustrate the main capabilities and functionalities of the MATLAB toolbox.

/pdf/markov-chain-monte-carlo-simulation-using-the-dream-software-2k57zrst2y.pdf

Markov chain Monte Carlo simulation using the DREAM software package: Theory, concepts, and MATLAB implementation

Traffic congestion has become a vexing and complex issue in many urban areas. Of particular interest are the intersections where traffic bottlenecks are known to occur despite being traditionally signalized. Reinforcement learning (RL), which is an artificial intelligence approach, has been adopted in traffic signal control for monitoring and ameliorating traffic congestion. RL enables autonomous decision makers (e.g., traffic signal controllers) to observe, learn, and select the optimal action (e.g., determining the appropriate traffic phase and its timing) to manage traffic such that system performance is improved. This article reviews various RL models and algorithms applied to traffic signal control in the aspects of the representations of the RL model (i.e., state, action, and reward), performance measures, and complexity to establish a foundation for further investigation in this research field. Open issues are presented toward the end of this article to discover new research areas with the objective to spark new interest in this research field.

A Survey on Reinforcement Learning Models and Algorithms for Traffic Signal Control

In this paper, first the travel and delay times in a road ending in a traffic light are determined under different traffic flows and traffic light cycles using a microscopic traffic simulator. The obtained results are analyzed and compared with the results of other models reported in the literature. In addition, the optimal traffic light cycle times under different loads are determined for one road and a method is shown to obtain the optimal green period ratios of a traffic light as well as the cycle time in an intersection. Finally, the generation of a fuzzy model is presented and a methodology is suggested, where the fuzzy system is applied as a surrogate model for the determination of the optimal green period ratios and traffic light cycle times.

Surrogate Model based Optimization of Traffic Lights Cycles and Green Period Ratios using Microscopic Simulation and Fuzzy Rule Interpolation

The problem of adaptive traffic signal control in the multi-intersection system has attracted the attention of researchers Among the existing methods, reinforcement learning has shown to be effective However, the complex intersection features, heterogeneous intersection structures, and dynamic coordination for multiple intersections pose challenges for reinforcement learning-based algorithms This paper proposes a cooperative deep Q-network with Q-value transfer (QT-CDQN) for adaptive multi-intersection signal control In QT-CDQN, a multi-intersection traffic network in a region is modeled as a multi-agent reinforcement learning system Each agent searches the optimal strategy to control an intersection by a deep Q-network that takes the discrete state encoding of traffic information as the network inputs To work cooperatively, the agent considers the influence of the latest actions of its adjacencies in the process of policy learning Especially, the optimal Q-values of the neighbor agents at the latest time step are transferred to the loss function of the Q-network Moreover, the strategy of the target network and the mechanism of experience replay are used to improve the stability of the algorithm The advantages of QT-CDQN lie not only in the effectiveness and scalability for the multi-intersection system but also in the versatility to deal with the heterogeneous intersection structures The experimental studies under different road structures show that the QT-CDQN is competitive in terms of average queue length, average speed, and average waiting time when compared with the state-of-the-art algorithms Furthermore, the experiments of recurring congestion and occasional congestion validate the adaptability of the QT-CDQN to dynamic traffic environments

/pdf/cooperative-deep-q-learning-with-q-value-transfer-for-multi-3rtbglohsi.pdf

Cooperative Deep Q-Learning With Q-Value Transfer for Multi-Intersection Signal Control

SLAM (Simultaneous Localization and Mapping) for underwater vehicles is a challenging research topic due to the limitations of underwater localization sensors and error accumulation over long-term operations. Furthermore, acoustic sensors for mapping often provide noisy and distorted images or low-resolution ranging, while video images provide highly detailed images but are often limited due to turbidity and lighting. This paper presents a review of the approaches used in state-of-the-art SLAM techniques: Extended Kalman Filter SLAM (EKF-SLAM), FastSLAM, GraphSLAM and its application in underwater environments.

Review of underwater SLAM techniques

Traffic flow control optimization in the traffic light systems is studied for improvement in this paper. Traffic light systems are built to control the traffic flows at the intersections to ensure the fluency of traffic flow within the traffic network. The increasing traffic flows that cannot be supported by the current traffic light systems cause lengthen of queue length at the intersection. The effect of queue length, green time, cycle time and amber time in the traffic system is observed and studied through simulations. Longer green time will pass through more vehicles, but it will increase the cycle time at the same time which causes more vehicles to accumulate at the intersection during the waiting time. Genetic algorithm is introduced in this paper for the optimization of the traffic flow control as its ability to find the optimized solution in its self tuning process. Genetic algorithm taking current queue length as its input then it will output the optimized green time for the intersection. The result of Genetic algorithm is further improved with the introduced of the incoming traffic flow during red time of each phase.

Optimization of Traffic Flow within an Urban Traffic Light Intersection with Genetic Algorithm

Traffic flow optimization within traffic networks has been approached through different kinds of methods. One of the methods is to reconfigure the traffic signal timing plan. However, dynamic characteristic of the traffic flow is not able to be resolved by the conventional traffic signal timing plan management. As a result, traffic congestion still remains as an unsolved problem. Thus, in this study, artificial intelligence algorithm has been introduced in the traffic signal timing plan to enable the traffic management systems' learning ability. Q-Learning algorithm acts as the learning mechanism for traffic light intersections to release itself from traffic congestions situation. Adjacent traffic light intersections will work independently and yet cooperate with each other to a common goal of ensuring the fluency of the traffic flows within traffic network. The experimental results show that the Q-Learning algorithm is able to learn from the dynamic traffic flow and optimized the traffic flow.

Q-Learning Traffic Signal Optimization within Multiple Intersections Traffic Network

Image segmentation has been widely applied in image analysis for various areas such as biomedical imaging, intelligent transportation systems and satellite imaging. The main goal of image segmentation is to simplify an image into segments that have a strong correlation with objects in the real world. Homogeneous regions of an image are regions containing common characteristics and are grouped as single segment. One of the graph partitioning methods in image segmentation, normalised cuts, has been recognised producing reliable segmentation result. To date, normalised cuts in image segmentation of various sized images is still lacking of analysis of its performance. In this paper, segmentation on synthetic images and natural images are covered to study the performance and effect of different image complexity towards segmentation process. This study gives some research findings for effective image segmentation using graph partitioning method with computation cost reduced. Because of its cost expensive and it becomes unfavourable in performing image segmentation on high resolution image especially in online image retrieval systems. Thus, a graph-based image segmentation method done in multistage approach is introduced here.

Image segmentation via normalised cuts and clustering algorithm

Image segmentation with low computational burden has been highly regarded as important goal for researchers. Various image segmentation methods are widely discussed and more noble segmentation methods are expected to be developed when there is rapid demand from the emerging machine vision field. One of the popular image segmentation methods is by using normalised cuts algorithm. It is unfavourable for a high resolution image to have its resolution reduced as high detail information is not fully made used when critical objects with weak edges is coarsened undesirably after its resolution reduced. Thus, a graph-based image segmentation method done in multistage manner is proposed here. In this paper, an experimental study based on the method is conducted. This study shows an alternative approach on the segmentation method using k-means clustering and normalised cuts in multistage manner.

Graph-Based Image Segmentation Using K-Means Clustering and Normalised Cuts

Nowadays, vehicle tracking is a vital approach to assist and improve the road traffic control, surveillance and security systems by having the detail of the captured vehicle information. In past, many tracking techniques have been implemented and suffered from the well known 'occlusion' problems. Increasing the accuracy of the tracking algorithm has caused the computational cost due to the inflexibility to adapt the partial and fully occluded situations. Besides occlusion, appearance of new objects and background noises in the captured videos increase the difficulties of continuously tracking the labelled vehicles. In this paper, an adaptive particle filter approach has been proposed as the tracking algorithm to solve the vehicle occlusion problem. In order to solve the common particle filter degeneracy problem, the proposed particle filter is equipped with the adaptive resampling algorithm which is capable of dealing with various occlusion incidents. The experimental results show that enhancement of the particle filter via resampling algorithm has been robustly tracking the vehicles, and significantly improve the accuracy in tracking the occluded vehicles without compromising the processing time.

Wei Yeang Kow

Papers

Optimization of Traffic Flow within an Urban Traffic Light Intersection with Genetic Algorithm

Q-Learning Traffic Signal Optimization within Multiple Intersections Traffic Network

Image segmentation via normalised cuts and clustering algorithm

Graph-Based Image Segmentation Using K-Means Clustering and Normalised Cuts

Enhancement of Particle Filter Approach for Vehicle Tracking Via Adaptive Resampling Algorithm