National University of Defense Technology

About: National University of Defense Technology is a education organization based out in Changsha, China. It is known for research contribution in the topics: Radar & Synthetic aperture radar. The organization has 39430 authors who have published 40181 publications receiving 358979 citations. The organization is also known as: Guófáng Kēxuéjìshù Dàxué & NUDT.

Multiobjective Reinforcement Learning: A Comprehensive Overview

Abstract: Reinforcement learning (RL) is a powerful paradigm for sequential decision-making under uncertainties, and most RL algorithms aim to maximize some numerical value which represents only one long-term objective. However, multiple long-term objectives are exhibited in many real-world decision and control systems, so recently there has been growing interest in solving multiobjective reinforcement learning (MORL) problems where there are multiple conflicting objectives. The aim of this paper is to present a comprehensive overview of MORL. The basic architecture, research topics, and naive solutions of MORL are introduced at first. Then, several representative MORL approaches and some important directions of recent research are comprehensively reviewed. The relationships between MORL and other related research are also discussed, which include multiobjective optimization, hierarchical RL, and multiagent RL. Moreover, research challenges and open problems of MORL techniques are suggested.

Hoaxy: A Platform for Tracking Online Misinformation

Abstract: Massive amounts of misinformation have been observed to spread in uncontrolled fashion across social media. Examples include rumors, hoaxes, fake news, and conspiracy theories. At the same time, several journalistic organizations devote significant efforts to high-quality fact checking of online claims. The resulting information cascades contain instances of both accurate and inaccurate information, unfold over multiple time scales, and often reach audiences of considerable size. All these factors pose challenges for the study of the social dynamics of online news sharing. Here we introduce Hoaxy, a platform for the collection, detection, and analysis of online misinformation and its related fact-checking efforts. We discuss the design of the platform and present a preliminary analysis of a sample of public tweets containing both fake news and fact checking. We find that, in the aggregate, the sharing of fact-checking content typically lags that of misinformation by 10-20 hours. Moreover, fake news are dominated by very active users, while fact checking is a more grass-roots activity. With the increasing risks connected to massive online misinformation, social news observatories have the potential to help researchers, journalists, and the general public understand the dynamics of real and fake news sharing.

Kernel-Based Least Squares Policy Iteration for Reinforcement Learning

Abstract: In this paper, we present a kernel-based least squares policy iteration (KLSPI) algorithm for reinforcement learning (RL) in large or continuous state spaces, which can be used to realize adaptive feedback control of uncertain dynamic systems. By using KLSPI, near-optimal control policies can be obtained without much a priori knowledge on dynamic models of control plants. In KLSPI, Mercer kernels are used in the policy evaluation of a policy iteration process, where a new kernel-based least squares temporal-difference algorithm called KLSTD-Q is proposed for efficient policy evaluation. To keep the sparsity and improve the generalization ability of KLSTD-Q solutions, a kernel sparsification procedure based on approximate linear dependency (ALD) is performed. Compared to the previous works on approximate RL methods, KLSPI makes two progresses to eliminate the main difficulties of existing results. One is the better convergence and (near) optimality guarantee by using the KLSTD-Q algorithm for policy evaluation with high precision. The other is the automatic feature selection using the ALD-based kernel sparsification. Therefore, the KLSPI algorithm provides a general RL method with generalization performance and convergence guarantee for large-scale Markov decision problems (MDPs). Experimental results on a typical RL task for a stochastic chain problem demonstrate that KLSPI can consistently achieve better learning efficiency and policy quality than the previous least squares policy iteration (LSPI) algorithm. Furthermore, the KLSPI method was also evaluated on two nonlinear feedback control problems, including a ship heading control problem and the swing up control of a double-link underactuated pendulum called acrobot. Simulation results illustrate that the proposed method can optimize controller performance using little a priori information of uncertain dynamic systems. It is also demonstrated that KLSPI can be applied to online learning control by incorporating an initial controller to ensure online performance.

A Survey of Traffic Control With Vehicular Communications

Abstract: During the last 60 years, incessant efforts have been made to improve the efficiency of traffic control systems to meet ever-increasing traffic demands. Some recent works attempt to enhance traffic efficiency via vehicle-to-vehicle communications. In this paper, we aim to give a survey of some research frontiers in this trend, identifying early-stage key technologies and discussing potential benefits that will be gained. Our survey focuses on the control side and aims to highlight that the design philosophy for traffic control systems is undergoing a transition from feedback character to feedforward character. Moreover, we discuss some contrasting preferences in the design of traffic control systems and their relations to vehicular communications. The first pair of contrasting preferences are model-based predictive control versus simulation-based predictive control. The second pair are global planning-based control versus local self-organization-based control. The third pair are control using rich information that may be highly redundant versus control using concise information that is necessary. Both the potentials and drawbacks of these control strategies are explained. We hope these comparisons can shed some interesting light on future traffic control studies.