Showing papers presented at "Analysis, Design, and Evaluation of Human-Machine Systems in 2019"

Haptic shared steering control with an adaptive level of authority based on time-to-line crossing

Abstract: Traditional driver-automation interaction trades control over the vehicle back and forth between driver and automation. Haptic shared control offers an alternative by continuously sharing the control through torques on the steering wheel and pedals. When designing additional feedback torques, part of the design choice lies in the stiffness around the neutral steering point: also called the Level of Haptic Authority (LoHA), which is usually static and tuned to balance safety benefits (better at high LoHA) with conflicts torques in case of different intentions between automation and driver (higher conflict torques with increased LoHA). In this paper we explore the idea of situation-adaptive LoHA: in this case during lane-keeping by changing the LoHA based on time to lane crossing (TLC). Consequently, when safety margins are high (e.g., when driving on a wide road) the LoHA is low, but the LoHA would only increase when safety margins decrease. We propose two alternative design approaches to apply the LoHA: symmetrically and asymmetrically (i.e., only increase of LoHA in the direction of the low TLC). We compared these design in an explorative driving simulator study (n=14) to driving with two static LoHA designs (low and high). We found that compared to the high LoHA controller, both adaptive LoHA controllers designs resulted in similar safety margins, but at decreased conflict torques. Hence, a TLC-based adaptive LoHA controller seems to be an effective approach to mitigate conflicts while maintaining the safety benefits associated with HSC.

Manual Control with Pursuit Displays New Insights, New Models, New Issues

Abstract: Mathematical control models are widely used in tuning manual control systems and understanding human performance. The most common model, the crossover model, is severely limited, however, in describing realistic human control behaviour in relevant control tasks as it is only valid for tracking with a compensatory display. This paper first discusses the state-of-the-art in modelling human control in tracking with pursuit displays. It is shown that, although both tasks seem very similar, the separate presentation of target and system output signals allows operators to adopt a huge variety in control strategies, which makes the development of a universal model for pursuit control a challenge. Two recent models are then described which can act as precursors to such a universal model. Third, system identification choices and issues are discussed for pursuit tracking tasks. Finally, it is argued that it is inevitable that time-varying rather than time-invariant methods are needed to properly describe human behaviour in the pursuit tracking task, as skilled operators will learn to characterize the probabilistic nature of the task, which cannot be captured in a single, linear, time-invariant model.

New trends in the design of human-machine interaction for CNC machines

Abstract: The development of numerical control technology, the increasing efficiency of production processes and the high automation level have allowed a great improvement of Computer Numerical Control (CNC) machines. Although CNC machines allow short working times and the precise repeatability of the processing, they suffer from some major disadvantages: they need to be controlled and managed by an expert operator who has to interact with the user interface to complete her/his tasks and set-up the machine. Unfortunately, to the best of our knowledge, there are no thorough studies on how human operators manage and perceive the interaction with these machines. As a result, the interfaces of most current CNC machines present several limitations. Considering the advantages in adding ergonomics and usability analysis to increase quality, productivity and market competitiveness, the industrial world is moving to develop new interfaces for CNC machines, to improve users’ experience. Starting from our experience with woodworking CNC machines, this paper reports, first, an analysis of the CNC process from the point of view of human-machine interaction; specifically, main issues and criticalities are outlined. Then, major new trends in modern interfaces for CNC machines are presented, highlighting the tendency to integrate user experience in the design of interfaces.

Using Real-time Feedback in a Training System for Manual Procedures

Abstract: Human workers remain a crucial part of production environments for conducting manual assembly or maintenance procedures, despite increasing automation. These procedures cannot be automated due to small lot sizes and high product variability. Performing manual procedures requires the application of procedural knowledge and motor skills, such as bimanual coordination and complex hand movements. Many training systems for manual procedures have been proposed. However, these systems focus on declarative knowledge about the sequence of work steps. The inherent haptic characteristics and sense for correct tool and component application gets lost. This paper proposes a training system that introduces haptic components for the training of assembly procedures. The proposed training system instructs the user in mounting two physical components by employing haptic and visual interaction. Augmentations and real-time feedback assist the user during the training and enable the assessment of applying accurate torque on screw connections. An evaluation compared the training system against video-based instructions and indicated advantages for the proposed system in terms of objective measures (time on task, precision) and in terms of subjective measures such as usability.

Supporting Humans in Solving Multi-UAV Dynamic Vehicle Routing Problems

Abstract: Real-time optimization of Vehicle Routing Problems (VRP) during mission operations raises concerns regarding obtaining a solution within a reasonable timeframe, especially in domains where operations cannot easily be paused and the number of control parameters is high. Humans, however, are heuristic problem solvers and could potentially complement VRP algorithms in providing quickly a workable and safe solution from which the algorithms can further find the optimum. In this study, a visual interface was developed and evaluated aiming to support humans in manually solving a dynamic VRP in which they needed to solve various simulated payload delivery missions, featuring multiple Unmanned Aerial Vehicles, under failure conditions. Experiment results (n = 16) indicate that the interface enabled the majority of participants to quickly solve the perturbed scenarios, although not always in the most efficient way. Interestingly, participants experienced most difficulty in solving the seemingly easier scenarios, featuring less customers and a relatively low number of vehicles compared to the more complex scenarios.

Complementing Haptic Shared Control with visual feedback for obstacle avoidance

Abstract: For automated vehicles (SAE Level 2-3) part of the challenge lies in communicating to the driver what control actions the automation is taking and will take, and what its capabilities are. A promising approach is haptic shared control (HSC), which uses continuous torques on the steering wheel to communicate the automation’s current control actions. However, torques on the steering wheel cannot communicate future spatiotemporal constraints, that might be required to judge appropriate overtaking or obstacle avoidance. A visualisation of predicted vehicle trajectory, along with velocity-dependent constraints with respect to achievable trajectories is proposed. The goal of this paper is to experimentally compare obstacle avoidance behaviour while driving with the designed visualisation against driving with a previously designed HSC, as well as the two support systems combined. It is expected that adding visual feedback improves obstacle avoidance and user acceptance, and reduces control effort with respect to HSC only. In a driving simulator experiment, 26 participants drove three trials with each feedback condition (visual, HSC, and combination) and had to avoid obstacles that appeared with a Time to collision of either 1.85 s (critical) or 4.7 s (non-criticall). Results showed that, compared to HSC only, the HSC and visual combination yielded slightly smaller safety margins to the obstacle, a significant reduction of control activity on straights, and increased subjective acceptance rating. Visual and HSC offered a beneficial synergy, as it seemed the visual feedback allowed drivers to anticipate the effect of their steering actions on the car’s trajectory more accurately, and the HSC reduced the intra-subject variability. Future research should investigate the effects of added visual feedback in more detail, specifically in terms of the effectiveness to communicate automation capabilities and driver gaze behavior.

Interactive visualization of model dependencies in a transdisciplinary environment

Abstract: Innovation processes are influenced by cyclical factors and involve many stakeholders from different disciplines within a company. Working together in transdisciplinary teams can save time and therefore improve the innovation process. However, the stakeholders have a completely different understanding of the processes based on their domain specific knowledge and experience and the requirements of their individual task. To avoid conflicts between the disciplines, this paper presents an interactive two-level graph-based approach to visualize cross-discipline model dependencies. This visualization can help to support the transdisciplinary communication and comprehension.

Survey on usability assessment for industrial user interfaces

Abstract: Modern production systems have become highly complex, given technological progress and competitive market needs. However, the role of human operators keeps being fundamental since they are in charge of supervising machines and take proper action in the presence of alarms and faults. As a consequence, the design of effective and easy to use human-machine interfaces (HMIs) is a key component of overall production efficiency. In this paper we provide an extensive overview of different methods to assess the usability of HMIs for industrial operators. In particular, the methods covered by the survey range from subjective assessment to objective quantitative analysis that takes into account user’s mental fatigue, user’s interaction strategies and production efficiency.

Synthesis of Valve and Pump Operations in Complex Plants by Using Functional Modeling

Abstract: It is crucial to provide operators supports for operation planning in complex process plants. This paper presents an operation searching approach used for synthesis of valve and pump operations to establish specific pipe routes. Instead of modeling pipeline fragments, a functional modeling methodology called Multilevel Flow Modeling (MFM) is adopted to represent plant knowledge. Relying on causality feature of MFM, one can identify required state changes of specific functions in the model, and accordingly operations. Since relationships between functions, components, and operations are independent to the modeling object, these generic principles can be implemented into a rule-based reasoning system for inferring operational conditions of pipe routes, namely, operations. An example in the literature has been used to demonstrate the presented approach and application of the software.

Evaluation of a 3D Solution Space-based ATC Workload Metric

Abstract: Air Traffic Control (ATC) workload is a limiting factor for air traffic growth, creating a need for objective ATC workload metrics. Previous research has shown that the solution space diagram can be a basis for a workload prediction metric. The current solution space metric however, does not incorporate altitude. In this paper, a 3D solution space metric is described and evaluated. An experiment has been conducted to test the relation of the 3D solution space metric with workload and compare it to other workload metrics; the aircraft count, and a quasi-3D metric: the 2D layered solution space and the Instantaneous Self Assessment-based method. Weak correlations with workload were found for all tested metrics and no significant differences were found between them. Although no significant differences were found, the 2D layered metric showed better results than the 3D solution space-based metric, indicating that air traffic controllers might think in 2D layers over fixed altitude ranges rather than considering the complete 3D physical solution space.

UKF-based Identification of Time-Varying Manual Control Behaviour

Abstract: This paper describes a novel method for time-varying identification of Human Controller (HC) manual control parameters (called UKF-FPV), based on a steady-state (constant state covariance) Unscented Kalman Filter (UKF). This approach requires no a priori assumptions on the shape of HO parameter variations, which is expected to be an advantage over other state-of-the-art methods, such as the recently proposed MLE-APV approach, for which a sigmoid-shaped parameter variation is assumed. For a scenario where an HO performs a single-loop compensatory tracking task with time-varying controlled system dynamics, both identification methods are compared using Monte Carlo simulations and human-in-the-loop experiment data. Despite some lag in the HO parameter traces of UKF-FPV, the identification results and the HC model quality-of-fit obtained with both methods were found to match well for both the simulation and experiment data. For the experiment data, UKF-FPV even revealed clear “local” changes in HC parameters not captured by theMLE-APV approach, which confirms that HCs adapt unpredictably even in time-invariant conditions. Overall, the results thus show that an identification method that requires no a priori assumptions on HC parameter variations is of critical importance for a complete analysis of time-varying HC behaviour.

Concept of Reskilling for Automation Collaboration in Maritime Piloting

Abstract: Advanced automation has been highlighted as contributory to several accidents involving modern bridge support systems and automation aiding maritime pilots for maneuvering and navigation. This paper argues for reskilling for automation collaboration, that operators need training that provides an understanding of what data the automation uses and how, and to transfer this skill to their working environment and be able to make full use of the automation even under influence of inaccurate data. As a case, this paper explores the predictor automation, which is an advanced navigation aid that visualizes an estimation of the ship’s future trajectory on an electronic chart display. Field studies and a literature review of maritime accidents were carried out to determine difficulties maritime pilots have with understanding the predictor. This research provides valuable guidance for how automation transparency can be an important part of reskilling and how to achieve it.

Analysis of Human Skill Development in Manual Ramp-Tracking Tasks

Abstract: Human modelling approaches are typically limited to feedback-only, compensatory tracking tasks. Advances in system identification techniques allow us to consider more realistic tasks that involve feedforward and even precognitive control. In this paper we study the human development of a feedforward control response while learning to accurately follow a ramp-shaped target signal in the presence of a disturbance acting on the controlled element. An experiment was conducted in which two groups of eight subjects each tracked ramps of different steepnesses in a random or ordered fashion. In addition, ordered runs were followed by a 'surprise' run with a random ramp steepness. Results show that operators learn rapidly, continue to learn during the entire experiment, and can adapt very quickly to surprise situations. Experiments involving learning operators are challenging, as it is difficult to balance-out all experimental conditions and control for inevitable differences between (groups of) subjects.

Generalizability of Manual Control Skills Between Control Tasks of Varying Difficulty

Abstract: This paper presents the results of an experiment that was performed at NASA Ames Research Center using 18 participants in two different groups who trained a task for ten days, with the goal of identifying how skill generalization would occur between two similar tasks of varying difficulty. A cybernetic approach was used. The first group was trained in a simple one-dimensional tracking task and transferred to a difficult two-dimensional tracking task. For the second group, this was reversed. Training with a simple task before transferring to the difficult task resulted in a slower convergence to final performance. However, it did allow participants to start with a better initial performance in the difficult task. Furthermore, after training with a simple task, participants controlled with a higher gain and generated lower lead time constants. However, possibly due to the number of participants, this experiment did not find any statistical evidence to support the conclusion that training with a simple task version helps in learning a more complex task.

Effects of Multisine Signal Bandwidth on Eye Movement Dynamics

Abstract: In the analysis of human motor skills, tracking tasks with multisine target signals are often performed as they allow for quantitative measurement, identification, and modeling of human control dynamics. In this paper, the same "cybernetic" approach is taken to analyze eye movement dynamics in gaze tracking tasks, where participants had to track a moving target marker across the screen) with their eyes (i.e., a eye-only task) and 2) with their dominant hand (i.e., a eye-hand task). A human-in-the-loop experiment with 10 participants was performed to measure the eye movement dynamics. These two different conditions were performed with four different bandwidths of the multisine target signal driving the movement of the visual stimulus. The results show that the measured eye movement dynamics can be identified from the data of all experiment conditions and can be accurately modeled as an underdamped mass-spring-damper system with a time delay. Furthermore, with increased target signal bandwidth the bandwidth of participants' eye movements also increases. Future development of gaze tracking tasks into a new tool for assessment of altered gaze behavior due to neurological diseases should take the balance between saccadic and smooth pursuit eye movements into account and avoid very high (i.e., too difficult) bandwidths to warrant accurate modelling of gaze dynamics.

Approximating Road Geometry with Multisine Signals for Driver Identification

Abstract: The understanding of human responses to visual information in car driving tasks requires the use of system identification tools that put constraints on the design of data collection experiments. Most importantly, multisine perturbation signals are required, including a multisine road geometry, to separately identify the different driver steering responses in the frequency domain. It is as of yet unclear, however, to what extent drivers steer differently along such multisine roads than they do for real roads. This paper presents a method for approximating real-world road geometries with multisine signals, and applies it to a stretch of road used in an earlier investigation into driver steering. In addition, a human-in-the-loop experiment is performed to collect driver steering data for both the realistic real-world road and its multisine approximation. Overall, the analysis of driver performance metrics and driver identification data shows that drivers adopt equivalent control behaviour when steering along both roads. Hence, the use of such multisine approximations allows for the realization of realistic roads and driver behaviour in car driving experiments, in addition to supporting the application of quantitative driver identification techniques for data analysis.