scispace - formally typeset
Search or ask a question

Showing papers on "Situation awareness published in 2019"


Journal ArticleDOI
TL;DR: This paper provides a survey of prediction, and forecasting methods used in cyber security, and discusses machine learning and data mining approaches, that have gained a lot of attention recently and appears promising for such a constantly changing environment, which is cyber security.
Abstract: This paper provides a survey of prediction, and forecasting methods used in cyber security. Four main tasks are discussed first, attack projection and intention recognition, in which there is a need to predict the next move or the intentions of the attacker, intrusion prediction, in which there is a need to predict upcoming cyber attacks, and network security situation forecasting, in which we project cybersecurity situation in the whole network. Methods and approaches for addressing these tasks often share the theoretical background and are often complementary. In this survey, both methods based on discrete models, such as attack graphs, Bayesian networks, and Markov models, and continuous models, such as time series and grey models, are surveyed, compared, and contrasted. We further discuss machine learning and data mining approaches, that have gained a lot of attention recently and appears promising for such a constantly changing environment, which is cyber security. The survey also focuses on the practical usability of the methods and problems related to their evaluation.

171 citations


Journal ArticleDOI
TL;DR: A multiclass traffic assignment model, where HDV users and CAV users follow different route choice principles, characterized by the cross-nested logit (CNL) model and user equilibrium (UE) model, respectively is proposed, which can assist transportation decision-makers to design effective planning and operational strategies to leverage the advantages of CAVs and manage traffic congestion under mixed traffic flows.
Abstract: Compared to existing human-driven vehicles (HDVs), connected and autonomous vehicles (CAVs) offer users the potential for reduced value of time, enhanced quality of travel experience, and seamless situational awareness and connectivity. Hence, CAV users can differ in their route choice behavior compared to HDV users, leading to mixed traffic flows that can significantly deviate from the single-class HDV traffic pattern. However, due to the lack of quantitative models, there is limited knowledge on the evolution of mixed traffic flows in a traffic network. To partly bridge this gap, this study proposes a multiclass traffic assignment model, where HDV users and CAV users follow different route choice principles, characterized by the cross-nested logit (CNL) model and user equilibrium (UE) model, respectively. The CNL model captures HDV users’ uncertainty associated with limited knowledge of traffic conditions while overcoming the route overlap issue of logit-based stochastic user equilibrium. The UE model characterizes the CAV's capability for acquiring accurate information on traffic conditions. In addition, the multiclass model can capture the characteristics of mixed traffic flow such as the difference in value of time between HDVs and CAVs and the asymmetry in their driving interactions, thereby enhancing behavioral realism in the modeling. The study develops a new solution algorithm labeled RSRS-MSRA, in which a route-swapping based strategy is embedded with a self-regulated step size choice technique, to solve the proposed model efficiently. Sensitivity analysis of the proposed model is performed to gain insights into the effects of perturbations on the mixed traffic equilibrium, which facilitates the estimation of equilibrium traffic flow and identification of critical elements under expected or unexpected events. The study results can assist transportation decision-makers to design effective planning and operational strategies to leverage the advantages of CAVs and manage traffic congestion under mixed traffic flows.

100 citations


Journal ArticleDOI
TL;DR: In this article, the authors reviewed peer-reviewed research publications investigating automated visual inspection technologies following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

94 citations


Journal ArticleDOI
TL;DR: A model is established that describes the statistical dependencies between channel state information and the position, orientation, and clock offset of a user equipment along with the locations of features in the propagation environment and introduces COMPAS (COncurrent Mapping, Positioning, And Synchronization); an inference engine that can provide accurate and reliable situational awareness in millimeter wave massive multiple-input multiple-output communication systems.
Abstract: Situational awareness in wireless networks refers to the availability of position information on transmitters and receivers as well as information on their propagation environments to aid wireless communications. In millimeter wave massive multiple-input multiple-output communication systems, situational awareness can significantly improve the quality and robustness of communications. In this paper, we establish a model that describes the statistical dependencies between channel state information and the position, orientation, and clock offset of a user equipment along with the locations of features in the propagation environment. Based on this model, we introduce COMPAS ( CO ncurrent M apping, P ositioning, A nd S ynchronization); an inference engine that can provide accurate and reliable situational awareness in millimeter wave massive multiple-input multiple-output communication systems. Numerical results show that COMPAS is able to infer the positions of an unknown and time-varying number of features in the propagation environment and, at the same time, estimate the position, orientation, and clock offset of a user equipment.

91 citations


Journal ArticleDOI
TL;DR: An overview of recent measurement models and approaches to establishing and enhancing SA in aviation environments and future research directions regarding SA assessment approaches are raised to deal with shortcomings of the existing state-of-the-art methods in the literature.
Abstract: Situation awareness (SA) is an important constituent in human information processing and essential in pilots’ decision making processes. Acquiring and maintaining appropriate levels of SA is critical in aviation environments as it affects all decisions and actions taking place in flights and air traffic control. This paper provides an overview of recent measurement models and approaches to establishing and enhancing SA in aviation environments. Many aspects of SA are examined including the classification of SA techniques into six categories, and different theoretical SA models from individual, to shared or team, and to distributed or system levels. Quantitative and qualitative perspectives pertaining to SA methods and issues of SA for unmanned vehicles are also addressed. Furthermore, future research directions regarding SA assessment approaches are raised to deal with shortcomings of the existing state-of-the-art methods in the literature.

83 citations


Journal ArticleDOI
TL;DR: The numerical results show that situational awareness-assisted beam selection using machine learning is able to provide beam prediction, with accuracy that increases with more complete knowledge of the environment.
Abstract: Establishing and tracking beams in millimeter-wave (mmWave) vehicular communication is a challenging task. Large antenna arrays and narrow beams introduce significant system overhead configuring the beams using exhaustive beam search. In this paper, we propose to learn the optimal beam pair index by exploiting the locations and types of the receiver vehicle and its neighboring vehicles (situational awareness), leveraging machine learning classification and past beam training data. We formulate the mmWave beam selection as a multi-class classification problem based on hand-crafted features that capture the situational awareness in different coordinates. We then provide a comprehensive comparison of the different classification models and various levels of situational awareness. Furthermore, we examine several practical issues in the implementation: localization is susceptible to inaccuracy; situational awareness at the base station (BS) can be outdated due to vehicle mobility and limited location reporting frequencies; the situational awareness may be incomplete since vehicles could be invisible to the BS if they are not connected. To demonstrate the scalability of the proposed beam selection solution in the large antenna array regime, we propose two solutions to recommend multiple beams and exploit an extra phase of beam sweeping among the recommended beams. The numerical results show that situational awareness-assisted beam selection using machine learning is able to provide beam prediction, with accuracy that increases with more complete knowledge of the environment.

81 citations


Journal ArticleDOI
TL;DR: A system based on electroencephalography (EEG) and eye-tracking (ET) techniques aimed to assess in real time the vigilance level of an ATCo dealing with a highly automated human–machine interface and to use this measure to adapt the level of automation of the interface itself.
Abstract: Increasing the level of automation in air traffic management is seen as a measure to increase the performance of the service to satisfy the predicted future demand. This is expected to result in new roles for the human operator: he will mainly monitor highly automated systems and seldom intervene. Therefore, air traffic controllers (ATCos) would often work in a supervisory or control mode rather than in a direct operating mode. However, it has been demonstrated how human operators in such a role are affected by human performance issues, known as Out-Of-The-Loop (OOTL) phenomenon, consisting in lack of attention, loss of situational awareness and de-skilling. A countermeasure to this phenomenon has been identified in the adaptive automation (AA), i.e., a system able to allocate the operative tasks to the machine or to the operator depending on their needs. In this context, psychophysiological measures have been highlighted as powerful tool to provide a reliable, unobtrusive and real-time assessment of the ATCo’s mental state to be used as control logic for AA-based systems. In this paper, it is presented the so-called “Vigilance and Attention Controller”, a system based on electroencephalography (EEG) and eye-tracking (ET) techniques, aimed to assess in real time the vigilance level of an ATCo dealing with a highly automated human–machine interface and to use this measure to adapt the level of automation of the interface itself. The system has been tested on 14 professional ATCos performing two highly realistic scenarios, one with the system disabled and one with the system enabled. The results confirmed that (i) long high automated tasks induce vigilance decreasing and OOTL-related phenomena; (ii) EEG measures are sensitive to these kinds of mental impairments; and (iii) AA was able to counteract this negative effect by keeping the ATCo more involved within the operative task. The results were confirmed by EEG and ET measures as well as by performance and subjective ones, providing a clear example of potential applications and related benefits of AA.

80 citations


Journal ArticleDOI
TL;DR: It is concluded that the outlined limitations of the SAGAT impede measurement of situation awareness, which can be computed more effectively from eye movement measurements in relation to the state of the task environment.
Abstract: The topic of situation awareness has received continuing interest over the last decades Freeze-probe methods, such as the Situation Awareness Global Assessment Technique (SAGAT), are commonly employed for measuring situation awareness The aim of this paper was to review validity issues of the SAGAT and examine whether eye movements are a promising alternative for measuring situation awareness First, we outlined six problems of freeze-probe methods, such as the fact that freeze-probe methods rely on what the operator has been able to remember and then explicitly recall We propose an operationalization of situation awareness based on the eye movements of the person in relation to their task environment to circumvent shortfalls of memory mediation and task interruption Next, we analyzed experimental data in which participants (N = 86) were tasked to observe a display of six dials for about 10 min, and press the space bar if a dial pointer crossed a threshold value Every 90 s, the screen was blanked and participants had to report the state of the dials on a paper sheet We assessed correlations of participants’ task performance (% of threshold crossing detected) with visual sampling scores (% of dials glanced at during threshold crossings) and freeze-probe scores Results showed that the visual-sampling score correlated with task performance at the threshold-crossing level (r = 031) and at the individual level (r = 078) Freeze-probe scores were low and showed weak associations with task performance We conclude that the outlined limitations of the SAGAT impede measurement of situation awareness, which can be computed more effectively from eye movement measurements in relation to the state of the task environment The present findings have practical value, as advances in eye-tracking cameras and ubiquitous computing lessen the need for interruptive tests such as SAGAT Eye-based situation awareness is a predictor of performance, with the advantage that it is applicable through real-time feedback technologies

64 citations


Journal ArticleDOI
TL;DR: Wang et al. as mentioned in this paper introduced blockchain-enabled, decentralized, capability-based access control (BlendCAC), a decentralized authentication, and capability based access control mechanism to enable effective protection for devices, services, and information in SSA networks.
Abstract: Space situation awareness (SSA) includes tracking of active and inactive resident space objects and assessing the space environment through sensor data collection and processing. To enhance SSA, the dynamic data-driven application systems framework couples online data with offline models to enhance performance by using feedback control, sensor management, and communications reliability. For information management, there is a need for identity authentication and access control (AC) to ensure the integrity of exchanged data as well as to grant authorized entities access right to data and services. Due to decentralization and heterogeneity of SSA systems, it is challenging to build an efficient centralized AC system, which can either be a performance bottleneck or the single point of failure. Inspired by the blockchain and smart contract technology, we introduce blockchain-enabled, decentralized, capability-based access control (BlendCAC), a decentralized authentication, and capability-based AC mechanism to enable effective protection for devices, services, and information in SSA networks. To achieve secure identity authentication, the BlendCAC leverages the blockchain to create virtual trust zones, in which distributed components can identify and update each other in a trustless network environment. A robust identity-based capability token management strategy is proposed, which takes advantage of the smart contract for registration, propagation, and revocation of the access authorization. A proof-of-concept prototype has been implemented on both resources-constrained devices (i.e., Raspberry Pi nodes emulating satellites with sensor observations) and more powerful computing devices (i.e., laptops emulating a ground network) and is tested on a private Ethereum blockchain network. The experimental results demonstrate the feasibility of the BlendCAC scheme to offer a decentralized, scalable, lightweight, and fine-grained AC solution for space system toward SSA.

62 citations


Proceedings ArticleDOI
16 Jun 2019
TL;DR: A dedicated Aerial Image Database for Emergency Response (AIDER) applications is introduced and a lightweight convolutional neural network (CNN) architecture is developed, capable of running efficiently on an embedded platform achieving ~3x higher performance compared to existing models with minimal memory requirements with less than 2% accuracy drop compared to the state-of-the-art.
Abstract: Unmanned Aerial Vehicles (UAVs), equipped with camera sensors can facilitate enhanced situational awareness for many emergency response and disaster management applications since they are capable of operating in remote and difficult to access areas. In addition, by utilizing an embedded platform and deep learning UAVs can autonomously monitor a disaster stricken area, analyze the image in real-time and alert in the presence of various calamities such as collapsed buildings, flood, or fire in order to faster mitigate their effects on the environment and on human population. To this end, this paper focuses on the automated aerial scene classification of disaster events from on-board a UAV. Specifically, a dedicated Aerial Image Database for Emergency Response (AIDER) applications is introduced and a comparative analysis of existing approaches is performed. Through this analysis a lightweight convolutional neural network (CNN) architecture is developed, capable of running efficiently on an embedded platform achieving ~3x higher performance compared to existing models with minimal memory requirements with less than 2% accuracy drop compared to the state-of-the-art. These preliminary results provide a solid basis for further experimentation towards real-time aerial image classification for emergency response applications using UAVs.

60 citations


Posted Content
TL;DR: This research was supported in part by the Automotive Research Center at the University of Michigan, with funding from government contract Department of the Army W56HZV-14-2-0001 through the U. S. Army Tank Automotive Development, Development, and Engineering Center (TARDEC).
Abstract: Driver assistance systems, also called automated driving systems, allow drivers to immerse themselves in non-driving-related tasks. Unfortunately, drivers may not trust the automated driving system, which prevents either handing over the driving task or fully focusing on the secondary task. We assert that enhancing situational awareness can increase trust in automation. Situational awareness should increase trust and lead to better secondary task performance. This study manipulated situational awareness by providing them with different types of information: the control condition provided no information to the driver, the low condition provided a status update, while the high condition provided a status update and a suggested course of action. Data collected included measures of trust, trusting behavior, and task performance through surveys, eye-tracking, and heart rate data. Results show that situational awareness both promoted and moderated the impact of trust in the automated vehicle, leading to better secondary task performance. This result was evident in measures of self-reported trust and trusting behavior.

Journal ArticleDOI
TL;DR: In this paper, the authors identify interactions among the most important individual, situational, and organizational variables affecting situation awareness in industrial workplaces, and their interactions are key factors in preventing accidents.

Journal ArticleDOI
TL;DR: A closer examination of system and driver responsibility is examined, independent of but includes SAE levels with respect to specific handover situations, which identifies gaps between the current legal liability for accidents when compared to aspects such as the situational awareness requirements placed on driver under different driving conditions.
Abstract: This paper provides a taxonomy of different forms of autonomous vehicle handover situations. It covers scheduled, emergency and non-emergency handovers and it differentiates between system and driver initiated handovers. The purpose is to examine how the system and driver are responsible for different stages in the transition timeline, i.e., first alert, handover phase, and return to automated control (handback). This is examined from the perspective of SAE levels in comparison to aspects drawn from situational awareness. The work is complemented by analysis drawn from current practice within the insurance industry and interviews with insurers. The result is a closer examination of system and driver responsibility which is independent of but includes SAE levels with respect to specific handover situations. It also identifies gaps between the current legal liability for accidents when compared to aspects such as the situational awareness requirements placed on driver under different driving conditions.

Journal ArticleDOI
TL;DR: Several design implications are suggested for improving performance including adding features to the automation that will allow the operator to use common strategies and providing necessary information using multiple sensory channels.

Journal ArticleDOI
TL;DR: This framework uses text mining, text classification, named entity recognition, and stemming techniques to extract the intelligence needed from Arabic social media feeds, for effective incident and emergency management in smart cities.

Journal ArticleDOI
TL;DR: In this article, the authors explored the concept of situation awareness within the maritime domain, identifying the situation awareness information requirements of navigators and factors affecting their SA, and discussed with their potential implications for the procedures and practices which better support situation awareness in maritime navigation.

Journal ArticleDOI
TL;DR: A method based on granular computing to support decision makers in analysing and protecting large-scale infrastructures or urban areas from external attacks by identifying a suitable partition of the infrastructure or the area under analysis by providing approximate solutions with the advantages of supporting reasoning at different levels of abstraction.
Abstract: We present a method based on granular computing to support decision makers in analysing and protecting large-scale infrastructures or urban areas from external attacks by identifying a suitable partition of the infrastructure or the area under analysis. The method works on a very limited set of information relating to the vulnerabilities of components, and probability information regarding how vulnerabilities can impact meaningful partitions. These aspects make the method very useful as a reasoning mechanism to improve awareness and support rapid decision making at early stages of intelligence analysis, when information is scarce and contains a high degree of uncertainty. The results of the case study, which are based on the hypothesis of a terrorist attack on a subway, show that the method provides approximate solutions with the advantages of supporting reasoning at different levels of abstraction and providing simplicity of threat scenario analysis. We also discuss the limitations of the applicability of our approach.

Journal ArticleDOI
TL;DR: This work presents an integrated set of functions for the presentation of and interaction with information for a mobile augmented reality application for military applications and refined the user interface architecture to conform to requirements from subject matter experts.
Abstract: Designing a user interface for military situation awareness presents challenges for managing information in a useful and usable manner. We present an integrated set of functions for the presentation of and interaction with information for a mobile augmented reality application for military applications. Our research has concentrated on four areas. We filter information based on relevance to the user (in turn based on location), evaluate methods for presenting information that represents entities occluded from the user's view, enable interaction through a top-down map view metaphor akin to current techniques used in the military, and facilitate collaboration with other mobile users and/or a command center. In addition, we refined the user interface architecture to conform to requirements from subject matter experts. We discuss the lessons learned in our work and directions for future research.

Journal ArticleDOI
TL;DR: The Person-Action-Locator (PAL), a novel UAV-based situational awareness system that relies on Deep Learning models to automatically detect people and recognize their actions in near real-time, was developed and successfully tested in the field.
Abstract: Situational awareness by Unmanned Aerial Vehicles (UAVs) is important for many applications such as surveillance, search and rescue, and disaster response. In those applications, detecting and locating people and recognizing their actions in near real-time can play a crucial role for preparing an effective response. However, there are currently three main limitations to perform this task efficiently. First, it is currently often not possible to access the live video feed from a UAV’s camera due to limited bandwidth. Second, even if the video feed is available, monitoring and analyzing video over prolonged time is a tedious task for humans. Third, it is typically not possible to locate random people via their cellphones. Therefore, we developed the Person-Action-Locator (PAL), a novel UAV-based situational awareness system. The PAL system addresses the first issue by analyzing the video feed onboard the UAV, powered by a supercomputer-on-a-module. Specifically, as a support for human operators, the PAL system relies on Deep Learning models to automatically detect people and recognize their actions in near real-time. To address the third issue, we developed a Pixel2GPS converter that estimates the location of people from the video feed. The result – icons representing detected people labeled by their actions – is visualized on the map interface of the PAL system. The Deep Learning models were first tested in the lab and demonstrated promising results. The fully integrated PAL system was successfully tested in the field. We also performed another collection of surveillance data to complement the lab results.

Proceedings ArticleDOI
02 May 2019
TL;DR: This work evaluates mental models that experts and non-expert users have of autonomous driving to provide an explanation of the vehicle's past driving behavior to identify a target mental model that enhances the user's mental model by adding key components from the mental model experts have.
Abstract: Driving in autonomous cars requires trust, especially in case of unexpected driving behavior of the vehicle. This work evaluates mental models that experts and non-expert users have of autonomous driving to provide an explanation of the vehicle's past driving behavior. We identified a target mental model that enhances the user's mental model by adding key components from the mental model experts have. To construct this target mental model and to evaluate a prototype of an explanation visualization we conducted interviews (N=8) and a user study (N=16). The explanation consists of abstract visualizations of different elements, representing the autonomous system's components. We explore the relevance of the explanation's individual elements and their influence on the user's situation awareness. The results show that displaying the detected objects and their predicted motion was most important to understand a situation. After seeing the explanation, the user's level of situation awareness increased significantly.

Journal ArticleDOI
TL;DR: The proposed framework which is based on the multi-agent system manages to mitigate potential traffic congestions and minimize drivers’ average travel time in metropolitan areas and can be achieved by the utilization of a closed-loop management system.
Abstract: Transportation infrastructure is undergoing major revolutions in most metropolitan areas, which demands for improved operational strategies to meet requirements of smart cities. Such requirements include more convenience more travelers, and higher levels of security, reliability, economics, and societal sustainability in our communities. Given that the wide-area situational awareness is enabled by advanced information and communication technologies, this paper develops a hierarchical operation framework for regulating traffic signals effectively and flexibly in dynamic traffic conditions. The proposed framework which is based on the multi-agent system manages to mitigate potential traffic congestions and minimize drivers’ average travel time in metropolitan areas. Further traffic efficiency improvements can be achieved by the utilization of a closed-loop management system. Interactive simulations are conducted in this paper to examine the performance of the proposed framework in a real-world transportation system.

Journal ArticleDOI
01 Apr 2019
TL;DR: The overall performance of the operators in terms of control efficiency and task completion is significantly improved with the proposed framework, and a suitable motion-scaling ratio can be obtained and adjusted online.
Abstract: Master–slave control is a common form of human–robot interaction for robotic surgery. To ensure seamless and intuitive control, a mechanism of self-adaptive motion scaling during teleoperaton is proposed in this letter. The operator can retain precise control when conducting delicate or complex manipulation, while the movement to a remote target is accelerated via adaptive motion scaling. The proposed framework consists of three components: 1) situation awareness, 2) skill level awareness, and 3) task awareness. The self-adaptive motion scaling ratio allows the operators to perform surgical tasks with high efficiency, forgoing the need of frequent clutching and instrument repositioning. The proposed framework has been verified on a da Vinci Research Kit to assess its usability and robustness. An in-house database is constructed for offline model training and parameter estimation, including both the kinematic data obtained from the robot and visual cues captured through the endoscope. Detailed user studies indicate that a suitable motion-scaling ratio can be obtained and adjusted online. The overall performance of the operators in terms of control efficiency and task completion is significantly improved with the proposed framework.

Proceedings ArticleDOI
21 Sep 2019
TL;DR: A driving simulator study was conducted to compare a digital uncertainty display located in the instrument cluster with a peripheral awareness display consisting of a light strip and vibro-tactile seat feedback, and results indicate that the latter display affords users flexibility to direct more attention towards the road prior to critical situations and leads to lower workload scores while improving takeover performance.
Abstract: As a consequence of insufficient situation awareness and inappropriate trust, operators of highly automated driving systems may be unable to safely perform takeovers following system failures. The communication of system uncertainties has been shown to alleviate these issues by supporting trust calibration. However, the existing approaches rely on information presented in the instrument cluster and therefore require users to regularly shift their attention between road, uncertainty display, and non-driving related tasks. As a result, these displays have the potential to increase workload and the likelihood of missed signals. A driving simulator study was conducted to compare a digital uncertainty display located in the instrument cluster with a peripheral awareness display consisting of a light strip and vibro-tactile seat feedback. The results indicate that the latter display affords users flexibility to direct more attention towards the road prior to critical situations and leads to lower workload scores while improving takeover performance.

Journal ArticleDOI
TL;DR: The design and development of an adaptive and immersive interface using virtual reality to bring operators into scenarios and allow an intuitive commanding of robots, as well as a complete set of experiments carried out to establish comparisons with a conventional one.
Abstract: Multiple robot missions imply a series of challenges for single human operators, such as managing high workloads or maintaining a correct level of situational awareness. Conventional interfaces are not prepared to face these challenges; however, new concepts have arisen to cover this need, such as adaptive and immersive interfaces. This paper reports the design and development of an adaptive and immersive interface, as well as a complete set of experiments carried out to establish comparisons with a conventional one. The interface object of study has been developed using virtual reality to bring operators into scenarios and allow an intuitive commanding of robots. Additionally, it is able to recognize the mission’s state and show hints to the operators. The experiments were performed in both outdoor and indoor scenarios recreating an intervention after an accident in critical infrastructure. The results show the potential of adaptive and immersive interfaces in the improvement of workload, situational awareness and performance of operators in multi-robot missions.

Journal ArticleDOI
TL;DR: This paper proposes a unique learning-based system architecture that allows operator control of larger numbers of rescue robots in a team as well as effective sharing of information between these robots, and shows that the learning- based approach can provide more scene coverage during robot exploration when compared to a non-learning based method.
Abstract: Teams of semi-autonomous robots can provide valuable assistance in Urban Search and Rescue (USAR) by efficiently exploring cluttered environments and searching for potential victims. Their advantage over solely teleoperated robots is that they can address the task handling and situation awareness limitations of human operators by providing some level of autonomy to the multi-robot team. Our research focuses on developing learning-based semi-autonomous controllers for rescue robot teams. In this paper, we specifically investigate the influence of the operator-to-robot ratio on the performance of our proposed MAXQ hierarchical reinforcement learning based semi-autonomous controller for USAR missions. In particular, we propose a unique learning-based system architecture that allows operator control of larger numbers of rescue robots in a team as well as effective sharing of information between these robots. A rigorous comparative study of our learning-based semi-autonomous controller versus a fully teleoperation-based approach was conducted in a 3D simulation environment. The results, as expected, show that, for both semi-autonomous and teleoperation modes, the total scene exploration time increases as the number of robots utilized increases. However, when using the proposed learning-based semi-autonomous controller, the rate of exploration-time increase and operator-interaction effort are significantly lower, while task performance is significantly higher. Furthermore, an additional case study showed that our learning-based approach can provide more scene coverage during robot exploration when compared to a non-learning based method.

Journal ArticleDOI
TL;DR: A joint cognitive system approach is applied to explore a socio-technical system to understand work systems and performance variability in relation to the transformation of information within a flight deck for a specific phase of flight.
Abstract: In a socio-technical work domain, humans, device interfaces and artefacts all affect transformations of information flow. Such transformations, which may involve a change of auditory to visual information & vice versa or alter semantic approximations into spatial proximities from instruments readings, are generally not restricted to solely human cognition. This paper applies a joint cognitive system approach to explore a socio-technical system. A systems ergonomics perspective is achieved by applying a multi-layered division to transformations of information between, and within, human and technical agents. The approach uses the Functional Resonance Analysis Method (FRAM), but abandons the traditional boundary between medium and agent in favour of accepting aircraft systems and artefacts as agents, with their own functional properties and relationships. The joint cognitive system perspective in developing the FRAM model allows an understanding of the effects of task and information propagation, and eventual distributed criticalities, taking advantage of the functional properties of the system, as described in a case study related to the cockpit environment of a DC-9 aircraft. Practitioner Summary: This research presents the application of one systemic method to understand work systems and performance variability in relation to the transformation of information within a flight deck for a specific phase of flight. By using a joint cognitive systems approach both retrospective and prospective investigation of cockpit challenges will be better understood. Abbreviations: ATC: air traffic control; ATCO: air traffic controller; ATM: air traffic management; CSE: cognitive systems engineering; DSA: distributed situation awareness; FMS: flight management system; FMV: FRAM model visualize; FRAM: functional resonance analysis method; GF: generalised function; GW: gross weight; HFACS: human factors analysis and classification system; JCS: joint cognitive systems; PF: pilot flying; PNF: pilot not flying; SA: situation awareness; SME: subject matter expert; STAMP: systems theoretic accident model and processes; VBA: visual basic for applications; WAD: work-as-done; WAI: work-as-imagined; ZFW: zero fuel weight.

Journal ArticleDOI
TL;DR: A framework for the situation awareness system based on multi-sensor fusion in the open-pit mine Internet of Things and information entropy theory is introduced to weight the data varying with attributes.
Abstract: Disasters that are uncertain and destructive pose severe threats to life and property of miners. One of the major precautious measures is to set up real-time monitoring of disaster with a number of different sensors. Single sensor which features weak, unstable, and noisy signal is prone to raise misjudgment leading to non-linearly correlated data coming from different sensors. This paper unfolds with a theoretical introduction to the situation awareness of data from sensors in the Internet of Things, covering theories including the Internet of Things, multi-sensor data fusion, and situation awareness. Subsequently, we construct a framework for the situation awareness system based on multi-sensor fusion in the open-pit mine Internet of Things. The data coming from multiple sensors are pre-processed with wavelet transform, data filling, and normalization. In addition, information entropy theory is introduced to weight the data varying with attributes. An RF-SVM-based model is constructed to accomplish data fusion and determine situation levels as well. The output of the RF-SVM-based model is input as an ELM model. The fusion results at the first 10 time points are used to forecast the situation level at next point, so that the proposed disaster forecast approach in this paper is practiced. To test the stationarity and validity of the approach, MATALAB is employed to run a simulation of the data of a given open-pit mine. The results show that the RMSE of the model remains below 0.2 and TSQ is no greater than 1.691 after we run 50 times, 100 times, and 200 times iteration. It convinces that forecast results made by the model are valid, indicating that the multi-sensor signal fusion which is effective and efficient provides support to disaster situation forecast and emergency management in the mine.

Journal ArticleDOI
20 Mar 2019
TL;DR: A prediction framework is presented that is able to infer a driver’s maneuver intention via a hybrid Bayesian network whose hidden layers represent a driver's lane contentedness and which is predicted by solving an optimal control problem.
Abstract: Today the automotive industry faces a robust trend toward assisted and automated driving. The technology to accomplish this ambition has evolved rapidly over the last few years, and yet there are still a lot of algorithmical challenges left to make an automation of the driving task a safe and comfortable experience. One of the main remaining challenges is the comprehension of the current traffic situation and the anticipation of all traffic participants’ future driving behavior, which is needed for the technical system to obtain situation awareness: an indispensable foundation for successful decision-making. In this paper, a prediction framework is presented that is able to infer a driver's maneuver intention. This is achieved via a hybrid Bayesian network whose hidden layers represent a driver's lane contentedness. A pre-training of the network's parameters with simulated data provides for human interpretable parameters even after running the expectation maximization algorithm based on data gathered on German highways. Moreover, the future driving path of any traffic participant is predicted by solving an optimal control problem, whereby the parameters of the optimal control formulation are found via inverse reinforcement learning.

Journal ArticleDOI
TL;DR: This paper determines that message content should be concentrated on mapped objects that are located farther away from the sender, but near the edge of local sensor range, and finds that optimized combination of message length and transmit rate ensures the optimal channel utilization for cooperative vehicular communication.
Abstract: By providing information about the objects that are non-line of sight and/or beyond the detection range of the local sensors, inter-vehicle communication compensates for the limitations of vehicle tracking subsystem in automated driving systems that relies on on-board sensing devices. Tracking capability in such systems can further be improved by making optimal use of the communication channel through sharing of locally created map data instead of transmitting only beacon messages. Message length adaptation, together with transmit rate control can address the scalability issue inherent in the vehicular network. The content of the exchanged information is another important aspect that has significant impact on the map accuracy in cooperative driving systems. In this paper, we study different congestion and content control schemes for a communication architecture aimed at map sharing, and evaluate their performance in terms of a situational awareness metric, namely position tracking error. This paper determines that message content should be concentrated on mapped objects that are located farther away from the sender, but near the edge of local sensor range. This paper also finds that optimized combination of message length and transmit rate ensures the optimal channel utilization for cooperative vehicular communication, which in turn improves the situational awareness of the whole system.

Proceedings ArticleDOI
01 Sep 2019
TL;DR: A cyber situational awareness framework based on digital twins is developed that provides a profound, holistic, and current view on the cyber situation that CPSs are in and enables a thorough, repeatable investigation process on a logic and network level.
Abstract: Operators of cyber-physical systems (CPSs) need to maintain awareness of the cyber situation in order to be able to adequately address potential issues in a timely manner. For instance, detecting early symptoms of cyber attacks may speed up the incident response process and mitigate consequences of attacks (e.g., business interruption, safety hazards). However, attaining a full understanding of the cyber situation may be challenging, given the complexity of CPSs and the ever-changing threat landscape. In particular, CPSs typically need to be continuously operational, may be sensitive to active scanning, and often provide only limited in-depth analysis capabilities. To address these challenges, we propose to utilize the concept of digital twins for enhancing cyber situational awareness. Digital twins, i.e., virtual replicas of systems, can run in parallel to their physical counterparts and allow deep inspection of their behavior without the risk of disrupting operational technology services. This paper reports our work in progress to develop a cyber situational awareness framework based on digital twins that provides a profound, holistic, and current view on the cyber situation that CPSs are in. More specifically, we present a prototype that provides real-time visualization features (i.e., system topology, program variables of devices) and enables a thorough, repeatable investigation process on a logic and network level. A brief explanation of technological use cases and outlook on future development efforts completes this work.