This paper presents an automatic collision avoidance algorithm for ships using a deep reinforcement learning (DRL) in continuous action spaces. Obstacle zone by target (OZT) is used to compute an area where a collision will happen in the future based on dynamic information of ships. Agents of DRL detects the approach of multiple ships using a virtual sensor called the grid sensor. Agents learned collision avoidance maneuvering through Imazu problem, which is a scenario set of ship encounter situations. In this study, we propose a new approach for collision avoidance with a longer safe passing distance using DRL. We develop a novel method named inside OZT that expands OZT to improve the consistency of learning. We redesign the network using the long short-term memory (LSTM) cell and carried out training in continuous action spaces to train a model with longer safe distance than the previous study. The bow cross range in collision detection proposed in this paper is effective to COLREGs-compliant collision avoidance. The trained model has passed all scenarios of Imazu problem. The model is also validated by a test scenario which includes more ships than each scenario of Imazu problem.

/pdf/automatic-ship-collision-avoidance-using-deep-reinforcement-4mpps56bvi.pdf

Automatic ship collision avoidance using deep reinforcement learning with LSTM in continuous action spaces

The automatic collision avoidance system introduced in this paper is a system constantly calculating optimal manoeuvring method from the risk and economic preference in the ship manoeuvring space where the course change and the deceleration are performed. The authors also propose a system that quantitatively evaluates the collision avoidance manoeuvring results. Based on the evaluation results of this system, the authors are setting parameters so that ship manoeuvring that does not give anxiety to target ships to be avoided is also realized in automatic collision avoidance manoeuvring. In addition, comparison between the manoeuvring results of the automatic collision avoidance system and the veteran captain's manoeuvring results was quantitatively compared by the proposed evaluation system. Verification experiments were successfully conducted to verify the effectiveness of the proposed automatic collision avoidance system on the actual ship navigating in congested waters. http://www.transnav.eu the International Journal on Marine Navigation and Safety of Sea Transportation Volume 13 Number 1 March 2019 DOI: 10.12716/1001.13.01.13

https://www.transnav.eu/files/Development of Automatic Collision Avoidance System and Quantitative Evaluation of the Manoeuvring Results,883.pdf

Development of Automatic Collision Avoidance System and Quantitative Evaluation of the Manoeuvring Results

In order to satisfy the demand for highly automated and fully safe ship operation, it is desirable that an Integrated Navigation System (INS) has an intelligent decision-making supporting function. Such a function can propose a proper maneuvering action to operators. Of course, the operators should decide a final action by their judgment supported by other available supporting functions of the INS. To implement such a supporting function, the authors have developed a collision avoidance expert system for such the INS. In this paper, the structure of developed knowledge bases, the inference procedure, the brush-up of the developed knowledge bases using ship handling simulator, and some simulation results are reported. Some concepts and methods concerned to this knowledge base system, such as the classification of the target ships, the unblocking scope of the ship handling space and the prediction of plural actions of target ships, are proposed. The conclusion is that the developed collision avoidance expert system possesses sophisticated problem solving ability for the collision avoidance and it can provide an effective support for operators' decision-making.

A collision avoidance expert system for integrated navigation system and its brush-up

This paper presents an automatic collision avoidance algorithm for multiple ships using reinforcement learning (RL). Obstacle zone by target (OZT) is used to grasp multiple ships’ dynamic information in the form of 2-dimensional areas. OZT shows a dangerous area where collisions may happen. Then a new method using a virtual sensor which is separated in a grid is proposed to detect multiple ships simultaneously. The sensor detects OZTs efficiently and provides information about where OZTs expands as a part of a state vector with a fixed dimension as inputs for a RL algorithm. I applied a deep RL algorithm. An agent of deep RL learned manoeuvre using a set of ship encounter situations called Imazu problem. The learned model can avoid all encounter situations of up to three target ships in simulations. The proposed approach can learn manoeuvre to manage both waypoint navigation and collision avoidance.

Automatic Collision Avoidance Using Deep Reinforcement Learning with Grid Sensor

It is well recognized that the evaluation of the knowledge-based system is very important and difficult in the development of expert systems in the domains such as aviation and navigation. To alleviate some of the difficulties, and to reduce the development cost, the authors proposed an evaluating procedure based on simulation. The proposed procedure was designed to validate the functionality and capability of an expert system that was developed to provide decision-making support in ship navigation. In this paper, the developed expert system for collision avoidance is first outlined, and the procedure to evaluate the developed expert system by using a simulation-based approach is presented in details. It is also concluded that the simulation-based procedure is feasible and effective for evaluating expert systems in a number of different domains.

/pdf/a-simulation-based-procedure-for-expert-system-evaluation-30fdl2kab0.pdf

A simulation-based procedure for expert system evaluation

https://www.jstage.jst.go.jp/article/jin/88/0/88_KJ00004721545/_pdf

A Nagasawa

Papers

The subjective difficulties of the situation of collision avoidance ii - toward the rating by simulation