생체의 모델링과 Simulation

Latest technological improvements increased the quality of transportation. New data-driven approaches bring out a new research direction for all control-based systems, e.g., in transportation, robotics, IoT and power systems. Combining data-driven applications with transportation systems plays a key role in recent transportation applications. In this paper, the latest deep reinforcement learning (RL) based traffic control applications are surveyed. Specifically, traffic signal control (TSC) applications based on (deep) RL, which have been studied extensively in the literature, are discussed in detail. Different problem formulations, RL parameters, and simulation environments for TSC are discussed comprehensively. In the literature, there are also several autonomous driving applications studied with deep RL models. Our survey extensively summarizes existing works in this field by categorizing them with respect to application types, control models and studied algorithms. In the end, we discuss the challenges and open questions regarding deep RL-based transportation applications.

Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey

Traffic congestion has become a vexing and complex issue in many urban areas. Of particular interest are the intersections where traffic bottlenecks are known to occur despite being traditionally signalized. Reinforcement learning (RL), which is an artificial intelligence approach, has been adopted in traffic signal control for monitoring and ameliorating traffic congestion. RL enables autonomous decision makers (e.g., traffic signal controllers) to observe, learn, and select the optimal action (e.g., determining the appropriate traffic phase and its timing) to manage traffic such that system performance is improved. This article reviews various RL models and algorithms applied to traffic signal control in the aspects of the representations of the RL model (i.e., state, action, and reward), performance measures, and complexity to establish a foundation for further investigation in this research field. Open issues are presented toward the end of this article to discover new research areas with the objective to spark new interest in this research field.

A Survey on Reinforcement Learning Models and Algorithms for Traffic Signal Control

The fact that road transportation negatively affects the quality of the environment and deteriorates its bearing capacity has drawn a wide range of concerns among researchers. In order to provide more realistic traffic data for estimations of environmental impacts, dynamic traffic assignment (DTA) models have been adopted in transportation planning and traffic management models concerning environmental sustainability. This review summarizes and examines the recent methodological advances of DTA models in environmentally sustainable road transportation applications including traffic signal control concerning vehicular emissions and emission pricing. A classification of emission estimation models and their integration with DTA models are accordingly reviewed as supplementary to the existing reviews. Finally, a variety of future research prospects of DTA for environmentally sustainable road transportation research are discussed. In particular, this review also points out that at present the research about DTA models in conjunction with noise predictive models is relatively deficient.

/pdf/dynamic-traffic-assignment-a-review-of-the-methodological-3bvlfuxoae.pdf

Dynamic traffic assignment: A review of the methodological advances for environmentally sustainable road transportation applications

https://re.public.polimi.it/bitstream/11311/1122896/1/A_Reinforcement_Learning_Framework_for_Optimal_Operation_and_Maintenance_of_Power_Grids1.pdf

A reinforcement learning framework for optimal operation and maintenance of power grids

Agent-based approaches have gained popularity in engineering applications, but its potential for advanced traffic controls has not been sufficiently explored. This paper presents a multi-agent framework that models traffic control instruments and their interactions with road traffic. A constrained Markov decision process (CMDP) model is used to represent agent decision making in the context of multi-objective policy goals, where the policy goal with the highest priority becomes the single optimization objective and the other goals are transformed as constraints. A reinforcement learning-based computational framework is developed for control applications. To implement the multi-objective decision model, a threshold lexicographic ordering method is introduced and integrated with the learning-based algorithm. Moreover, a two-stage hybrid framework is established to improve the learning efficiency of the model. While the proposed approach is potentially applicable for different road traffic operations, this paper applies the framework for traffic signal control in a network of Stockholm based on traffic simulation. The computational results show that the proposed control approach can handle a complex case of multiple policy requirements. Meanwhile, the agent-based intelligent control has shown superior performance when compared to other optimized signal control methods.

A Multi-Objective Agent-Based Control Approach With Application in Intelligent Traffic Signal System

http://kth.diva-portal.org/smash/get/diva2:1205227/FULLTEXT01

An intelligent control system for traffic lights with simulation-based evaluation

http://kth.diva-portal.org/smash/get/diva2:1183324/FULLTEXT01

Hierarchical multi-agent control of traffic lights based on collective learning

Increasing concerns on environment and natural resources, coupled with increasing demand for transport, put lots of pressure for improved efficiency and performance on transport systems worldwide. New technology nowadays enables fast innovation in transport, but it is the policy for deployment and operation with a systems perspective that often determines success. Smart traffic management has played important roles for continuous development of traffic systems especially in urban areas. There is, however, still lack of effort in current traffic management and planning practice prioritizing policy goals in environment and energy. This paper presents an application of a model-based framework to quantify environmental impacts and fuel efficiency of road traffic, and to evaluate optimal signal plans with respect not only to traffic mobility performance but also other important measures for sustainability. Microscopic traffic simulator is integrated with micro-scale emission model for estimation of emissions and fuel consumption at high resolution. A stochastic optimization engine is implemented to facilitate optimal signal planning for different policy goals, including delay, stop-and-goes, fuel economy etc. In order to enhance the validity of the modeling framework, both traffic and emission models are fine-tuned using data collected in a Chinese city. In addition, two microscopic traffic models are applied, and lead to consistent results for signal optimization. Two control schemes, fixed time and vehicle actuated, are optimized while multiple performance indexes are analyzed and compared for corresponding objectives. Solutions, representing compromise between different policies, are also obtained in the case study by optimizing an integrated performance index.

/pdf/multi-criteria-analysis-of-optimal-signal-plans-using-aum9vjnh9v.pdf

Multi-criteria analysis of optimal signal plans using microscopic traffic models

A paradigm shift towards agile and adaptive traffic signal control empowered with the massive growth of Big Data and Internet of Things (IoT) technologies is emerging rapidly for Intelligent Transportation Systems. Generally, an adaptive signal control system fine-tunes signal timing parameters based on pre-defined control hyperparameters using instantaneous traffic detection information. Once traffic pattern changes, those hyperparameters (e.g., maximum and minimum green times) need to be adjusted according to the evolution of traffic dynamics over a very short-term period. Such adjustment processes are usually conducted by professional and experienced traffic engineers. Here we present a human-in-the-loop parallel learning framework and its utilization in an end-to-end recommendation system that mimics and enhances professional signal control engineers’ behaviors. The system has been deployed into a real-world application for an extended period in Hangzhou, China, where signal control hyperparameters are recommended based on large-scale multidimensional traffic datasets. Experimental evaluations demonstrate significant improvements in traffic efficiency through the use of our signal recommendation system.

Junchen Jin

Papers

A Multi-Objective Agent-Based Control Approach With Application in Intelligent Traffic Signal System

An intelligent control system for traffic lights with simulation-based evaluation

Hierarchical multi-agent control of traffic lights based on collective learning

Multi-criteria analysis of optimal signal plans using microscopic traffic models

An End-to-End Recommendation System for Urban Traffic Controls and Management Under a Parallel Learning Framework