Reinforcement Learning for Attack Mitigation in SDN-enabled Networks

TLDR: An intelligent defense system implemented as a reinforcement machine learning agent that processes current network state and takes a set of necessary actions in form of software-defined networking flows to redirect certain network traffic to virtual appliances is proposed.

Abstract: With the recent progress in the development of low-budget sensors and machine-to-machine communication, the Internet-of-Things has attracted considerable attention. Unfortunately, many of today's smart devices are rushed to market with little consideration for basic security and privacy protection making them easy targets for various attacks. Unfortunately, organizations and network providers use mostly manual workflows to address malware-related incidents and therefore they are able to prevent neither attack damage nor potential attacks in the future. Thus, there is a need for a defense system that would not only detect an intrusion on time, but also would make the most optimal real-time crisis-action decision on how the network security policy should be modified in order to mitigate the threat. In this study, we are aiming to reach this goal relying on advanced technologies that have recently emerged in the area of cloud computing and network virtualization. We are proposing an intelligent defense system implemented as a reinforcement machine learning agent that processes current network state and takes a set of necessary actions in form of software-defined networking flows to redirect certain network traffic to virtual appliances. We also implement a proof-of-concept of the system and evaluate a couple of state-of-art reinforcement learning algorithms for mitigating three basic network attacks against a small realistic network environment.