Neural networks are increasingly deployed in real-world safety-critical domains such as autonomous driving, aircraft collision avoidance, and malware detection. However, these networks have been shown to often mispredict on inputs with minor adversarial or even accidental perturbations. Consequences of such errors can be disastrous and even potentially fatal as shown by the recent Tesla autopilot crash. Thus, there is an urgent need for formal analysis systems that can rigorously check neural networks for violations of different safety properties such as robustness against adversarial perturbations within a certain L-norm of a given image. An effective safety analysis system for a neural network must be able to either ensure that a safety property is satisfied by the network or find a counterexample, i.e., an input for which the network will violate the property. Unfortunately, most existing techniques for performing such analysis struggle to scale beyond very small networks and the ones that can scale to larger networks suffer from high false positives and cannot produce concrete counterexamples in case of a property violation. In this paper, we present a new efficient approach for rigorously checking different safety properties of neural networks that significantly outperforms existing approaches by multiple orders of magnitude. Our approach can check different safety properties and find concrete counterexamples for networks that are 10x larger than the ones supported by existing analysis techniques. We believe that our approach to estimating tight output bounds of a network for a given input range can also help improve the explainability of neural networks and guide the training process of more robust neural networks.

/pdf/efficient-formal-safety-analysis-of-neural-networks-4wrtspbak4.pdf

Efficient Formal Safety Analysis of Neural Networks

In this paper we assess the feasibility of using formal methods, and model checking in particular, for the certification of Unmanned Aircraft Systems (UAS) within civil airspace. We begin by modelling a basic UAS control system in PROMELA, and verify it against a selected subset of the CAA's Rules of the Air using the SPIN model checker. Next we build a more advanced UAS control system using the autonomous agent language Gwendolen, and verify it against the small subset of the Rules of the Air using the agent model checker AJPF. We introduce more advanced autonomy into the UAS agent and show that this too can be verified. Finally we compare and contrast the various approaches, discuss the paths towards full certification, and present directions for future research.

/pdf/formal-methods-for-the-certification-of-autonomous-unmanned-1cw30ptnos.pdf

Formal methods for the certification of autonomous unmanned aircraft systems

The Traffic Alert and Collision Avoidance System (TCAS) is a world-wide accepted last-resort means of reducing the probability and frequency of mid-air collisions between aircraft. Unfortunately, it is widely known that in congested airspace, the use of the TCAS may actually lead to induced collisions. Therefore, further research regarding TCAS logic is required. In this paper, an encounter model is formalised to identify all of the potential collision scenarios that can be induced by a resolution advisory that was generated previously by the TCAS without considering the downstream consequences in the surrounding traffic. The existing encounter models focus on checking and validating the potential collisions between trajectories of a specific scenario. In contrast, the innovative approach described in this paper concentrates on quantitative analysis of the different induced collision scenarios that could be reached for a given initial trajectory and a rough specification of the surrounding traffic. This approach provides valuable information at the operational level. Furthermore, the proposed encounter model can be used as a test-bed to evaluate future TCAS logic changes to mitigate potential induced collisions in hot spot volumes. In addition, the encounter model is described by means of the coloured Petri net (CPN) formalism. The resulting state space provides a deep understanding of the cause-and-effect relationship that each TCAS action proposed to avoid an actual collision with a potential new collision in the surrounding traffic. Quantitative simulation results are conducted to validate the proposed encounter model, and the resulting collision scenarios are summarised as valuable information for future Air Traffic Management (ATM) systems.

/pdf/coloured-petri-net-based-traffic-collision-avoidance-system-13lb1cgmzb.pdf

Coloured Petri net-based traffic collision avoidance system encounter model for the analysis of potential induced collisions

The engineering vision of relying on the ``smart sky" for supporting air traffic and the ``Internet above the clouds" for in-flight entertainment has become imperative for the future aircraft industry. Aeronautical ad hoc Networking (AANET) constitutes a compelling concept for providing broadband communications above clouds by extending the coverage of Air-to-Ground (A2G) networks to oceanic and remote airspace via autonomous and self-configured wireless networking amongst commercial passenger airplanes. The AANET concept may be viewed as a new member of the family of Mobile ad hoc Networks (MANETs) in action above the clouds. However, AANETs have more dynamic topologies, larger and more variable geographical network size, stricter security requirements and more hostile transmission conditions. These specific characteristics lead to more grave challenges in aircraft mobility modeling, aeronautical channel modeling and interference mitigation as well as in network scheduling and routing. This paper provides an overview of AANET solutions by characterizing the associated scenarios, requirements and challenges. Explicitly, the research addressing the key techniques of AANETs, such as their mobility models, network scheduling and routing, security and interference are reviewed. Furthermore, we also identify the remaining challenges associated with developing AANETs and present their prospective solutions as well as open issues. The design framework of AANETs and the key technical issues are investigated along with some recent research results. Furthermore, a range of performance metrics optimized in designing AANETs and a number of representative multi-objective optimization algorithms are outlined.

Aeronautical Ad Hoc Networking for the Internet-Above-The-Clouds

In view of the SESAR and NextGEN objectives of increasing both the capacity and the safety of the Air Traffic Management (ATM) system, there is a need to conduct safety risk analysis of current or new operations, covering the joint effect of airborne and ground-based safety nets in ATM. The subject of the research presented in this paper is Airborne Collision Avoidance System (ACAS) which presents an airborne safety net within an ATM context, for current practices as well as advanced concepts. The aim of the research described in this paper is fivefold: (a) to verify existing ACAS models regarding their coverage of evaluation needs of ACAS operations; (b) to develop a stochastic and dynamical model of ACAS II including interactions with pilots and air traffic control; (c) to develop a systematic validation process that allows building model confidence; (d) to initially apply this validation process to the newly-developed ACAS model; and (e) to use the ACAS model to assess the potential collision risk reduction by ACAS II for a historical en-route mid-air collision event. The specific modelling formalism used for this is Stochastically and Dynamically Coloured Petri Nets (SDCPN). The developed SDCPN-based ACAS model contains the technical, human and procedural elements of ACAS operations and fully supports mathematical analysis as well as rare event Monte Carlo simulation of aircraft encounters. In order to build confidence into the developed model and to judge model credibility, a systematic multilevel validation process is defined and is successfully applied. The SDCPN-based ACAS model is demonstrated to work well for a historical en-route mid-air collision event and is very powerful in determining the most critical elements contributing to the non-zero collision risk of ACAS operation.

Development, validation and application of stochastically and dynamically coloured Petri net model of ACAS operations for safety assessment purposes

The paper provides an overview of the development and operational deployment of the Traffic alert Collision Avoidance system (TCAS). TCAS was one of the first software based "safety of life" systems deployed in aircraft.

Ed Williams

Papers

Airborne collision avoidance system