Showing papers presented at "Analysis, Design, and Evaluation of Human-Machine Systems in 2016"

Shared control is the sharp end of cooperation: Towards a common framework of joint action, shared control and human machine cooperation

Abstract: As an introduction to the session of shared and cooperative control, this article will briefly look into the history, start with definitions and sketch a common framework of shared and cooperative control that sees the two phrases not as different concepts, but as different perspectives or foci on a common design space of shared intentionality, control and cooperation between humans and machines. One working hypothesis which the session will explore is that shared control can be understood as cooperation at the control level, while human machine cooperation can include shared control, but can also extend towards cooperation at higher levels, e.g. of guidance and navigation, of maneuvers and goals. We propose to view the relationship between shared control and human-machine cooperation as being similar to the relationship between the sharp, pointy tip and the (blunt) shaft of a spear. Shared control is where cooperation comes sharply into effect at the control level, but to be truly effective it should be supported by cooperation on all levels of interaction beyond the control level, e.g. on the guidance and navigation level.

Situation Awareness Training for General Aviation Pilots using Eye Tracking

Abstract: Numerous studies in high-risk work settings reveal the great amount of human related causes in accident and incidents occurrences (e.g., Flin, 2008; Helmreich, 2000; Reason, 1990). Moreover, Endsley (1999) points out that about 88% of all accident causes can be traced back to situation awareness (L1 perception [76.3%], L2 comprehension [20.3%], and L3 anticipation [3.4%]). As there is no common and standardized training content on scanning behavior and situation awareness in today’s flight school education in Switzerland, an effective training concept is necessary a) to support pilots in achieving and maintaining appropriate scanning skills; b) to develop personal mechanisms in handling situation awareness and c) to regain situation awareness after an unexpected event (e.g., LOC-I). The following paper aims at demonstrating a new training design for general aviation pilots. The training involves theoretical information about scanning techniques and situation awareness and is combined with practical exercises - either in a flight simulator or in real flight sessions. During the practical training session, the pilot wears an eye tracking device, which provides the flight instructor (trainer) with immediate information on the pilot’s scanning patterns and state of situation awareness.

Smartwatch-Enhanced Interaction with an Advanced Troubleshooting System for Industrial Machines

Abstract: Smartwatches are unobtrusive everyday devices which can be also exploited for effective gesture-based human-machine interaction. In this paper, we propose the use of a smartwatch to interact with an advanced troubleshooting application to be used in industrial environment. The application is a hypermedia information system aiming at assisting workers in performing preventive and corrective machine maintenance. The smartwatch allows a handsfree interaction, thus facilitating the use of the whole system when wearing personal protective equipment such as gloves or having fingers greased with oil or dust, which prevent operating touch screens. The algorithm for gesture recognition we have devised, which is based on template matching, is described in the paper, together with its experimental validation. Finally, we present a preliminary usability assessment of the overall system, meant as integration of the smartwatch with the hypermedia system.

Insights into Interaction - Effects of Human-Human Interaction in Pedestrian Crossing Situations using a linked Simulator Environment

Abstract: The classical and common approach of driving simulation as a tool in traffic research is usually limited to one driving simulator. The behavior of the driver in specific traffic situations is of interest while driving through programmed scenarios, encountering the situations addressing the research questions and reacting on the more or less sufficient programmed other road users. Especially the latter is, due to programming skills and limitations of the used software frameworks, the bottleneck when it comes to the crucial aspect of social interaction. This mutual behavior adaption is a vivid, complex, short-timed and multi-directional process between road users and is of even higher interest when focusing on urban traffic. In this paper, an approach is presented which addresses the aspect of social interaction in driving simulation in case of pedestrian crossing situations. A multi simulator setting is used to bring both participants in the same virtual environment and perform the given driving respectively walking task while encountering each other for example at a zebra crossing or in an occlusion situation. Driving data like time-to-arrival, braking pressure and average speed in these situations reveal the effects of the human-human interaction in contrast to human-bot interaction.

The Effects of Yaw and Sway Motion Cues in Curve Driving Simulation

Abstract: This paper investigates the importance of yaw and sway motion cues in curve driving simulation. While such motion cues are known to enhance simulation realism, their function in supporting realistic driver behavior in simulators is still largely unknown. A human-in-the-loop curve driving experiment was performed in the SIMONA Research Simulator at TU Delft, in which eight participants were asked to follow a winding road’s center-line, while being subject to wind disturbances. Four motion conditions were tested: 1) no motion, 2) yaw only, 3) sway only, and 4) both yaw and sway; each was tested with 5 m and 100 m road preview for correspondence with earlier work. Results show that visual road preview is essential for adequate road-following. Although the effects of yaw and sway cues are much smaller, sway motion feedback allows for improved disturbance-rejection performance, while yaw motion feedback results in reduced control activity. These distinctly different effects suggest that both motion cues are important for evoking realistic driving behavior in simulators.

Human Crossfeed in Dual-Axis Manual Control with Motion Feedback

Abstract: While many realistic manual control tasks require human operators to control multiple degrees-of-freedomsimultaneously, our understanding of such multi-axis manual control has not moved far beyond considering it simply as the control of multiple fully-independent axes. This investigation aims to further our understanding of multi-axis control by focusing on one phenomenon that is known to occur in such tasks: crossfeed. Crossfeed occurs when operators’ inputs in one controlled axis feed into another controlled degree-of-freedom, thereby affecting overall control performance. A human-in-the-loop experiment, in which operators performed a dual-axis aircraft roll and pitch tracking task with physical motion feedback, was conducted in the SIMONA Research Simulator at TU Delft. Three conditions were tested: the full dual-axis control task, supplemented with reference single-axis roll and pitch tasks. Through the use of independent target and disturbance forcing function signals in both controlled axes, we were able to detect the presence of crossfeed in this dual-axis task from spectral analysis. Furthermore, these signals facilitated the objective identification of the dynamics of the crossfeed contribution, in parallel with estimating operators visual and motion responses. The crossfeed dynamics were found to resemble the well-known dynamics of human operators’ visual responses. The crossfeed contribution was found to explain up to 20% of the measured control inputs, thereby indicating that crossfeed can be a factor of significance in multi-axis manual control.

Interdisciplinary Communication and Comprehension in Factory Automation Engineering - A Concept for an Immersive Virtual Environment

Abstract: In engineering of factory automation systems, understanding and discussing complex systems is important for a competitive development process. Understanding complex connections within these systems while working with interdisciplinary teams scattered around the globe can be a very challenging task. An intuitive way for comprehension and communication within this environment would be beneficial. For this reason, we propose a concept for a virtual environment for gaining a quicker understanding of complex systems and their connections, as well as a way for presentation and discussion using the concept of virtual reality. This is approached by using an immersive way of visualizing and interacting with complex information. The environment itself as well as different ways of immersing in it are presented. To highlight the properties of the approach, an application example is given.

Design and evaluation of a Flight Envelope Protection haptic feedback system

Abstract: This paper describes the design and evaluation of a shared control, haptic feedback system to communicate Flight Envelope Protection System intent. The concept uses a combination of stiffness feedback and vibration to communicate proximity of the aircraft state to flight envelope boundaries. In addition, a stick center shift can be applied by the envelope protection system to cooperatively perform corrective actions in case of severe excursions of the envelope margins. Results from the evaluation experiment show improved performance with haptic feedback in both scenarios. Workload ratings were unaffected. Pilot opinion was unanimously positive, especially with regard to the combination of stiffness feedback and vibration cues.

The Predictability of a Target Signal Affects Manual Feedforward Control

Abstract: In the manual control of a dynamic system, the human controller (HC) is often required to follow a visible and predictable reference path. Using the predictable aspect of a reference signal, through applying feedforward control, the HC can significantly improve performance as compared to a purely feedback control strategy. A proper definition of a signal’s predictability, however, is never given in literature. This paper investigates the predictability of a sum-of-sinusoids target signal, as a function of the number of sinusoid components and the fact whether the sinusoid frequencies are harmonic, or not. A human-in-the-loop experiment was done, with target signals varying for these two signal characteristics. A combined feedback-feedforward HC model was identified and parameters were estimated. It was found that for all experimental conditions, subjects used a feedforward strategy. Results further showed thatsubjects were able to perform better for harmonic signals as compared to non-harmonic signals, for signals with roughly the same frequency content.

A dynamic closed-looped and multidimensional model for Mental Workload evaluation

Abstract: The interest in mental workload (MWL) has strongly increased over the last 40 years in ergonomics and engineering studies. However, the subject is still very controversial and there is a debate on the dimensions to be considered for representing and estimating the workload. This paper reinvestigates the modeling of the mental workload concept, by combining the model of Hart and Staveland (1988), which considers this concept as a multidimensional construction, and the models of Sperandio (1971) and Leplat (2006), which focus on the regulation of the activity by the operator. This combination reveals new relationships between the different dimensions of MWL, and especially points out a means of measuring and understanding the effect of regulation on the workload. To validate this proposed approach to the modeling of MWL, an experiment is conducted on the dynamic activity of area monitoring based on the management of a swarm of drones.

Multifactor interactions and the air traffic controller: The interaction of situation awareness and workload in association with automation

Abstract: Air traffic controllers (ATCOs) must maintain a consistently high level of human performance in order to maintain flight safety and efficiency. In current control environments, performance-influencing factors such as workload, fatigue and situation awareness can co-occur and interact to affect performance. However, multifactor influences and the association with performance are under-researched. This study utilized a high fidelity human in the loop en-route air traffic control simulation to investigate the relationship between workload, situation awareness and ATCO performance. The current study aimed to replicate Edwards, Sharples, Wilson and Kirwan’s (2012) previous research, and extend the study by using a ex-controllers as participants, and comparing multifactor relationships across four levels of automation. Results suggest that workload and situation awareness may interact to produce a compound impact on controller performance. In addition, the effect of the interaction on performance may be dependent on the context and level of automation. Findings have implications for human-automation teaming in air traffic control, and the potential prediction, and therefore support, of ATCO performance.

MyAID: a Troubleshooting Application for Supporting Human Operators in Industrial Environment

Abstract: Nowadays manufacturing systems are increasing in sophistication and complexity. In this scenario, human operators experience many diffculties to interact effciently with the machine. To tackle this problem, in this paper we present a novel interactive troubleshooting tool to be used in industrial environment. The application, called MyAID , relies on a hypermedia information system and aims at assisting shopoor workers in a factory to perform preventive and corrective machine maintenance. The main advantages of MyAID are the following. First, it overcomes limitations of conventional printed documentation. Second, it can be easily updated and adapted to other machines or uses, other than troubleshooting, since it enjoys a modular structure. Third, it can be connected directly to the control unit of the machine to read active alarms or verify whether the user is following correctly the troubleshooting procedure. Finally, the plan for usability assessment of the proposed application, which is organized in a preliminary heuristic evaluation according to Nielsen's heuristics and field tests with users, is introduced.

Impact of Automation Support on the Conflict Resolution Task in a Human-in-the-Loop Air Traffic Control Simulation

Abstract: To determine the capabilities and limitations of human operators and automation in separation assurance roles, the second of three Human-in-the-Loop (HITL) part-task studies investigated air traffic controllers’ ability to detect and resolve conflicts under varying task sets, traffic densities, and run lengths. Operations remained within a single sector, staffed by a single controller, and explored, among other things, the controller’s responsibility for conflict resolution with or without their involvement in the conflict detection task. Furthermore, these conditions were examined across two different traffic densities; 1x (current-day traffic) and a 20% increase above current-day traffic levels (1.2x). Analyses herein offer an examination of the conflict resolution strategies employed by controllers. In particular, data in the form of elapsed time between conflict detection and conflict resolution are used to assess if, and how, the controllers’ involvement in the conflict detection task affected the way in which they resolved traffic conflicts.

Formal and Joint Verification of Control Programs and Supervision Interfaces for Socio-technical Systems Components

Abstract: The Anaxagore Project (Bignon et al., 2013) provides a component-based design ow for reconfigurable socio-technical systems. Each component integrates a control program and a supervision interface and it has been validated by empirical testing. The purpose of this paper is the use of formal methods for the verification of the whole component control-command chain. Different component features (the control program, the supervision interface, the physical device) and the human tasks are modeled using timed automata. These timed automata are then checked by model checking (UPPAAL) with a set of safety and usability properties written in CTL. Our approach is presented through an industrial case study: the supervised control of a 2-way motorized valve. The results show that the use of formal techniques enables to successfully detect control program and supervision interface design errors.

Blind driving by means of auditory feedback

Abstract: Driving is a safety-critical task that predominantly relies on vision. However, visual information from the environment is sometimes degraded or absent. In other cases, visual information is available, but the driver fails to use it due to distraction or impairment. Providing drivers with real-time auditory feedback about the state of the vehicle in relation to the environment may be an appropriate means of support when visual information is compromised. In this study, we explored whether driving can be performed solely by means of artificial auditory feedback. We focused on lane keeping, a task that is vital for safe driving. Three auditory parameter sets were tested: (1) predictor time, where the volume of a continuous tone was a linear function of the predicted lateral error from the lane centre 0 s, 1 s, 2 s, or 3 s into the future; (2) feedback mode (volume feedback vs. beep-frequency feedback) and mapping (linear vs. exponential relationship between predicted error and volume/beep frequency); and (3) corner support, in which in addition to volume feedback, a beep was offered upon entering/leaving a corner, or alternatively when crossing the lane centre while driving in a corner. A dead-zone was used, whereby the volume/beep-frequency feedback was provided only when the vehicle deviated more than 0.5 m from the centre of the lane. An experiment was conducted in which participants (N = 2) steered along a track with sharp 90-degree corners in a simulator with the visual projection shut down. Results showed that without predictor feedback (i.e., 0 s prediction), participants were more likely to depart the road compared to with predictor feedback. Moreover, volume feedback resulted in fewer road departures than beep-frequency feedback. The results of this study may be used in the design of in-vehicle auditory displays. Specifically, we recommend that feedback be based on anticipated error rather than current error.

Ecological Interface for Collaboration of Multiple UAVs in Remote Areas

Abstract: Unmanned Aerial Vehicles (UAVs) can be used to access remote areas that were otherwise inaccessible, for example, for surveillance missions. Collaboration between them can help overcome communication constraints by building airborne relay networks that allow beyond line of sight communication. This research investigates if a single operator can supervise multiple UAVs in a collaborative surveillance task under communication constraints. For this purpose and ecological interface was designed to support operators in their task and to bring flexibility in the system. A human-in-the-loop evaluation study was performed to investigate the successfulness of operators in the control task of such a mission including an analysis of individual components of the interface. It was shown that operators are able to successfully operate surveillance missions under communication- and battery constraints. Participants did however not completely do this without separation conflicts and communication losses, which indicates that the interface lacks elements representing endurance and separation assurance. To an extent the interface design turned out to be scalable, with a few remaining visualizations that still suffered from this problem. More advanced ways of displaying information on request and grouping of select information is thought to offer opportunities to improve ground control interface on this matter.

A Cybernetic Approach to Assess the Training of Manual Control Skills

Abstract: This paper presents a cybernetic approach to assess the training of manual control skills in simulators. The approach uses multi-channel pilot models that separate pilots’ responses to visual and motion stimuli. This allows for a quantitative analysis of pilots’ use of visual and motion cues for manual aircraft control, as well as the evolution of these control skills during training and after transfer. The cybernetic approach was applied to data from a quasi-transfer-of-training experiment performed in the SIMONA Research Simulator at Delft University of Technology. In this experiment, fully task-naive participants were trained to perform an aircraft pitch attitude tracking task in a fixed-base simulator environment. After training, participants were transferred to a motion-base simulator environment. Results indicate that the cybernetic approach is successful in revealing progressive changes in participants’ utilization of visual and motion cues – i.e., their equalization dynamics – during training and after transfer. Furthermore, the results show that convergence to a final skill-based manual control strategy requires significant training.

Seeing the woods for the trees: The problem of information inefficiency and information overload on operator performance

Abstract: One of the recurring questions in designing dynamic control environments is whether providing more information leads to better operational decisions. The idea of having every piece of information and increasing situation awareness is so tempting (and in safety critical domains often mandatory) that has become an obstacle for designers and operators. This research examined this challenge within a railway control setting. A laboratory study was conducted to investigate the presentation of different levels of information (taken from data processing framework, Dadashi et al., 2014) and the association with, and potential prediction of, the performance of a human operator when completing a cognitively demanding problem solving scenario within railways. Results indicated that presenting users with information corresponding to their cognitive task (and no more) improves the performance of their problem solving/alarm handling. Knowing the key features of interest to various agents (machine or human) and using the data processing framework to guide the optimal level of information required by each of these agents could potentially lead to safer and more usable designs.

Constraints in Identification of Multi-Loop Feedforward Human Control Models

Abstract: The human controller (HC) can greatly improve target-tracking performance by utilizing a feedforward operation on the target signal, in addition to a feedback response. System identification methods are used to determine the correct HC model structure: purely feedback or a combined feedforward/feedback model. In this paper, we investigate three central issues that complicate this objective. First, the identification method should not require prior assumptions regarding the dynamics of the feedforward and feedback components. Second, severe biases might be introduced by high levels of noise in the data measured under closed-loop conditions. To address the first two issues, we will consider two identification methods that make use of linear ARX models: the classic direct method and the two-stage indirect method of van den Hof and Schrama (1993). Third, model complexity should be considered in the selection of the ‘best’ ARX model to prevent ‘false-positive’ feedforward identification. Various model selection criteria, that make an explicit trade-off between model quality and model complexity, are considered. Based on computer simulations with a HC model, we conclude that 1) the direct method provides more accurate estimates in the frequency range of interest, and 2) existing model selection criteria do not prevent false-positive feedforward identification. Copyright ©2016 IFAC

Ecological Interface Design: Sensor Failure Diagnosis in Air Traffic Control

Abstract: Future air traffic control will have to rely on more advanced automation in order to support controllers in their job of safely controlling increased traffic volumes. A prerequisite for the success of such automation is that the underlying data driving it is reliable. Current technology, however, still warrants human supervision in coping with (data) uncertainties and consequently in judging the validity of machine decisions. In this paper the Ecological Interface Design (EID) framework is explored to assist controllers in fault diagnosis using a prototype ecological interface (called the Solution Space Diagram) for tactical conict detection and resolution in the horizontal plane. Results from a human-in-the-loop experiment with sixteen participants indicate that the ecological interface with explicit presentation of the means-ends relations between higher-level functional goals and lower-level physical objects (i.e., aircraft) enables improved sensor failure detection. Especially in high complexity scenarios, this feature had a positive impact on failure detection performance.

Biodynamic Feedthrough: Current Status and Open Issues

Abstract: Biodynamic feedthrough (BDFT) occurs when vehicle accelerations feed through the body of a human operator, causing involuntary limb motions, which in turn result in involuntary control inputs. Manual control of many different vehicles is known to be vulnerable to BDFT effects, such as that of helicopters, aircraft, electric wheelchairs and hydraulic excavators. This paper provides a brief review of BDFT literature, which serves as a basis for identifying the fundamental challenges that remain to be addressed in future BDFT research. One of these challenges, time-variant BDFT identification, is discussed in more detail. Currently, it is often assumed that BDFT dynamics are (quasi)linear and time-invariant. This assumption can only be justified when measuring BDFT under carefully crafted experimental conditions, which are very different from real-world situations. As BDFT dynamics depend on neuromuscular dynamics, they are typically time-varying. This paper investigates the suitability of a recently developed time-variant identification approach, based on a recursive least-squares algorithm, which has been successfully used to identify time-varying neuromuscular dynamics.

Design, Realization and Experimental Evaluation of a Haptic Stick for Shared Control Studies

Abstract: Shared control is becoming widely used in many manual control tasks as a mean for improving performance and safety. Designing an effective shared control system requires extensive testing and knowledge of how operators react to the haptic sensations provided by the control device shared with the support system. Commercial general purpose haptic devices may be unfit to reproduce the operational situation typical of the control task under study, like car driving or airplane flying. Thus specific devices are needed for research on specific task; this market niche exists but is characterized by expensive products. This paper presents the development of a complete low cost haptic stick, of its initial characterization and inner loop and impedance control systems design, and finally proposes an evaluation with two test cases: pilot admittance identification with the classical tasks, and an entire haptic experiment. In particular this latter experiment tries to study what happens when a system failure happens in a pilot support system built using a classical embedded controller, compared to a system built following the haptic shared control paradigm.

Cross-sector transferability of metrics for air traffic controller workload

Abstract: Air traffc controller workload is an important impediment to air transport growth. Several approaches exist that aim to better understand the causes for workload, and models have been derived to predict workload in new operational settings. These methods often relate workload to the diffculty, or complexity, that an average controller would have to safely manage all traffc in a sector with a particular traffc demand. In this paper, several of these complexity-based metrics for workload will be compared. Of special interest is whether the complexity measures transfer from one sector design to another. That is, does a metric that is well-tuned to predict workload for controllers working in one sector, also predict the workload for another group of controllers active in a different sector? Results from a human-in-the-loop experiment show that a solution space-based metric, which requires no tuning or weighing at all, has the highest correlations with subjectively reported workload, and also yields the best workload predictions across different controller groups and sectors.

Design of test signals for identification of neuromuscular admittance

Abstract: The human neuromuscular system can be seen as a versatile and extremely adaptive actuator. Through co-contraction and reex modulation, the properties of the neuromuscular system can be modified, leading to a change in movement response to externally applied forces. These properties are normally expressed in the form of the neuromuscular admittance. In a series of standard tasks, the force-, relax-, and position-task admittance of the neuromuscular system can be identified. However, the test signals used in these tasks can also limit the range of reex adaptation possible and wrong choice can create a phenomenon analogous to cross-over regression in manual control tasks, and force the human to use only a limited range of the possible reex adaptation. This paper presents a systematic investigation, through a model study, of the inuence of test signals on the range of reex adaptation. For this, criteria for test signal acceptability have been developed. The method is applied to the currently used test signals consisting of a high and a low shelf, and enables the selection of the high shelf bandwidth.