Dimensions of Reinforcement Learning

Recently, the world witnessed a significant increase in the number of used drones, with a global and continuous rise in the demand for their multi-purpose applications. The pervasive aspect of these drones is due to their ability to answer people’s needs. Drones are providing users with a bird’s eye that can be activated and used almost anywhere and at any time. However, recently, the malicious use of drones began to emerge among criminals and cyber-criminals alike. The probability and frequency of these attacks are both high and their impact can be very dangerous with devastating effects. Therefore, the need for detective, protective and preventive counter-measures is highly required. The aim of this survey is to investigate the emerging threats of using drones in cyber-attacks, along the countermeasures to thwart these attacks. The different uses of drones for malicious purposes are also reviewed, along the possible detection methods. As such, this paper analyzes the exploitation of drones vulnerabilities within communication links, as well as smart devices and hardware, including smart-phones and tablets. Moreover, this paper presents a detailed review on the drone/Unmanned Aerial Vehicle (UAV) usage in multiple domains (i.e civilian, military, terrorism, etc.) and for different purposes. A realistic attack scenario is also presented, which details how the authors performed a simulated attack on a given drone following the hacking cycle. This review would greatly help ethical hackers to understand the existing vulnerabilities of UAVs in both military and civilian domains. Moreover, it allows them to adopt and come up with new techniques and technologies for enhanced UAV attack detection and protection. As a result, various civilian and military anti-drones/UAVs (detective and preventive) countermeasures will be reviewed.

Security analysis of drones systems: Attacks, limitations, and recommendations

UAV assistance paradigm: State-of-the-art in applications and challenges

In this paper, we study an intelligent reflecting surface (IRS)-aided wireless secure communication system, where an IRS is deployed to adjust its reflecting elements to secure the communication of multiple legitimate users in the presence of multiple eavesdroppers. Aiming to improve the system secrecy rate, a design problem for jointly optimizing the base station (BS)’s beamforming and the IRS’s reflecting beamforming is formulated considering different quality of service (QoS) requirements and time-varying channel conditions. As the system is highly dynamic and complex, and it is challenging to address the non-convex optimization problem, a novel deep reinforcement learning (DRL)-based secure beamforming approach is firstly proposed to achieve the optimal beamforming policy against eavesdroppers in dynamic environments. Furthermore, post-decision state (PDS) and prioritized experience replay (PER) schemes are utilized to enhance the learning efficiency and secrecy performance. Specifically, a modified PDS scheme is presented to trace the channel dynamic and adjust the beamforming policy against channel uncertainty accordingly. Simulation results demonstrate that the proposed deep PDS-PER learning based secure beamforming approach can significantly improve the system secrecy rate and QoS satisfaction probability in IRS-aided secure communication systems.

Deep Reinforcement Learning-Based Intelligent Reflecting Surface for Secure Wireless Communications

Blockchain for smart cities: A review of architectures, integration trends and future research directions

In this paper, we investigate a secure communication system, where one transmitter sends some confidential message to one receiver, under the smart attack from one unmanned aerial vehicle (UAV). The UAV utilizes the programmable radio devices to execute multiple attack types, such as eavesdropping, jamming, and spoofing. We consider the practical communication scenarios where there are a limited number of pilot signals for channel estimation, and the channel estimation at the receiver side may be imperfect. To deal with the joint impact of smart attack and imperfect channel estimation, we use the non-cooperative game theory to propose a Q-learning-based power control algorithm in order to obtain an adaptive policy for the transmitter. In this algorithm, a Nash equilibrium strategy of the game under imperfect channel estimation is devised to maximize the utility of the transmitter and, meanwhile, suppress the smart attack from the UAV. The simulation results indicate that this strategy can effectively decrease the attack rate of the UAV attacker and increase the system secrecy capacity, regardless of the channel estimation accuracy. Moreover, the impact of the channel estimation error on the attack rate and secrecy capacity is also reflected.

Protecting Secure Communication Under UAV Smart Attack With Imperfect Channel Estimation

In this paper, we proposed an enhanced secure scheme for the wireless communication system threatened by an intelligent attacker, which can work in eavesdropping, jamming, and spoofing modes. The conventional secure scheme is to apply $Q$ -learning-based algorithm to reach a Nash equilibrium (NE) in the framework of a zero-sum game between the transmitter and attacker, which, however, requires the number of antennas at the transmitter to be much larger than that at the attacker. To overcome this limitation, we first consider the scenario where the attacker can flexibly increase the number of antennas in order to increase the attack rate. By adaptively setting the number of antennas at the transmitter and the legitimate receiver equal to that at the attacker, we then apply the beamforming at the transmitter to suppress the eavesdropping and use the filtering at the receiver to prevent the jamming and spoofing. By incorporating the beamforming and filtering, the benefits of the attacker in this game are efficiently restricted. Furthermore, the $Q$ -learning-based power control strategy is used to reach a new NE. The simulation results have been demonstrated to show that the proposed scheme can suppress the intelligent attack efficiently, which outperforms the conventional scheme in the secrecy performance.

/pdf/enhanced-secure-transmission-against-intelligent-attacks-4tthi24tl7.pdf

Enhanced Secure Transmission Against Intelligent Attacks

While Internet-of-Things (IoT) significantly facilitates the convenience of people's daily life, the lack of security practice raises the risk of privacy-sensitive user data leakage. Securing data transmission among IoT devices is therefore a critical capability of IoT environments such as Intelligent Connected Vehicles, Smart Home, Intelligent City and so forth. However, cryptographic communication scheme is challenged by the limited resource of low-cost IoT devices, even negligible extra CPU usage of battery-powered sensors would result in dramatical decrease of the battery life. In this paper, to minimize the resource consumption, we propose a communication protocol involving only the symmetric key-based scheme, which provides ultra-lightweight yet effective encryptions to protect the data transmissions. Symmetric keys generated in this protocol are delegated based on a chaotic system, i.e., Logistic Map, to resist against the key reset and device capture attacks. We semantically model such protocol and analyze the security properties. Moreover, the resource consumption is also evaluated to guarantee runtime efficacy.

A Lightweight Privacy-Preserving Communication Protocol for Heterogeneous IoT Environment

This paper investigates a cache-aided mobile edge computing (MEC) network, where the source offloads the computation task to multiple destinations with computation capacity, with the help of a cache-aided relay. For the proposed cache-aided MEC networks, two destination selection criteria have been proposed to maximize the computation capacity of the selected destination, the channel gain of relay link and the channel gain of direct link, respectively. Similarly, three destination selection criteria have been proposed for the cache-free MEC networks based on the computation capacities of destinations and the channel gains of transmission links, respectively. To evaluate the system performance regarding the latency constraint, we provide the outage probability for the proposed network which is defined based on the transmission-plus-computation time. Our analysis suggests that caching can significantly alleviate the impact of increasing the size of computation task, since only half of the transmission time of cache-free network is required. However, the cache-aided network can not fully exploit the signal from both direct and relay links, thus the improvement by caching is less significant in the high signal-to-noise ratio (SNR) region, compared with the cache-free network employing the destination with maximal channel gain of direct link. Numerical results are given to validate our analysis.

/pdf/cache-aided-mobile-edge-computing-for-b5g-wireless-4nu32jyij5.pdf

Cache-aided mobile edge computing for B5G wireless communication networks

This paper investigates cache-enabled physical-layer secure communication in a no-orthogonal multiple access (NOMA) network with two users, where an intelligent unmanned aerial vehicle (UAV) is equipped with attack module which can perform as multiple attack modes. We present a power allocation strategy to enhance the transmission security. To this end, we propose an algorithm which can adaptively control the power allocation factor for the source station in NOMA network based on reinforcement learning. The interaction between the source station and UAV is regarded as a dynamic game. In the process of the game, the source station adjusts the power allocation factor appropriately according to the current work mode of the attack module on UAV. To maximize the benefit value, the source station keeps exploring the changing radio environment until the Nash equilibrium (NE) is reached. Moreover, the proof of the NE is given to verify the strategy we proposed is optimal. Simulation results prove the effectiveness of the strategy.

/pdf/cache-enabled-physical-layer-secure-game-against-smart-uav-2fbjoaemrb.pdf

Chao Li

Papers

Protecting Secure Communication Under UAV Smart Attack With Imperfect Channel Estimation

Enhanced Secure Transmission Against Intelligent Attacks

A Lightweight Privacy-Preserving Communication Protocol for Heterogeneous IoT Environment

Cache-aided mobile edge computing for B5G wireless communication networks

Cache-enabled physical-layer secure game against smart uAV-assisted attacks in b5G NOMA networks