Showing papers in "IEEE Transactions on Human-Machine Systems in 2014"

Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues

Abstract: The human factors literature on intelligent systems was reviewed in relation to the following: efficient human supervision of multiple robots, appropriate human trust in the automated systems, maintenance of human operator's situation awareness, individual differences in human-agent (H-A) interaction, and retention of human decision authority. A number of approaches-from flexible automation to autonomous agents-were reviewed, and their advantages and disadvantages were discussed. In addition, two key human performance issues (trust and situation awareness) related to H-A teaming for multirobot control and some promising user interface design solutions to address these issues were discussed. Some major individual differences factors (operator spatial ability, attentional control ability, and gaming experience) were identified that may impact H-A teaming in the context of robotics control.

A Hand Gesture Recognition Framework and Wearable Gesture-Based Interaction Prototype for Mobile Devices

Abstract: An algorithmic framework is proposed to process acceleration and surface electromyographic (SEMG) signals for gesture recognition. It includes a novel segmentation scheme, a score-based sensor fusion scheme, and two new features. A Bayes linear classifier and an improved dynamic time-warping algorithm are utilized in the framework. In addition, a prototype system, including a wearable gesture sensing device (embedded with a three-axis accelerometer and four SEMG sensors) and an application program with the proposed algorithmic framework for a mobile phone, is developed to realize gesture-based real-time interaction. With the device worn on the forearm, the user is able to manipulate a mobile phone using 19 predefined gestures or even personalized ones. Results suggest that the developed prototype responded to each gesture instruction within 300 ms on the mobile phone, with the average accuracy of 95.0% in user-dependent testing and 89.6% in user-independent testing. Such performance during the interaction testing, along with positive user experience questionnaire feedback, demonstrates the utility of the framework.

An Ontology-Based Hybrid Approach to Activity Modeling for Smart Homes

Abstract: Activity models play a critical role for activity recognition and assistance in ambient assisted living. Existing approaches to activity modeling suffer from a number of problems, e.g., cold-start, model reusability, and incompleteness. In an effort to address these problems, we introduce an ontology-based hybrid approach to activity modeling that combines domain knowledge based model specification and data-driven model learning. Central to the approach is an iterative process that begins with “seed” activity models created by ontological engineering. The “seed” models are deployed, and subsequently evolved through incremental activity discovery and model update. While our previous work has detailed ontological activity modeling and activity recognition, this paper focuses on the systematic hybrid approach and associated methods and inference rules for learning new activities and user activity profiles. The approach has been implemented in a feature-rich assistive living system. Analysis of the experiments conducted has been undertaken in an effort to test and evaluate the activity learning algorithms and associated mechanisms.

Image-Based and Sensor-Based Approaches to Arabic Sign Language Recognition

Abstract: Sign language continues to be the preferred method of communication among the deaf and the hearing-impaired. Advances in information technology have prompted the development of systems that can facilitate automatic translation between sign language and spoken language. More recently, systems translating between Arabic sign and spoken language have become popular. This paper reviews systems and methods for the automatic recognition of Arabic sign language. Additionally, this paper highlights the main challenges characterizing Arabic sign language as well as potential future research directions.

Application of Adaptive Controllers in Teleoperation Systems: A Survey

Abstract: A survey of the adaptive controllers deployed to address major inherent control issues in robotic teleoperation systems is carried out. The study in particular explores the application of adaptive controllers in dealing with master and slave model uncertainties, operator and environment force model uncertainties, unknown external disturbances, and communication delay. The reviewed literature is structured according to the objectives envisaged for the adaptive controllers. Meanwhile, some adaptive methods deployed in human–robot interaction, where robots collaborate with people and actively support them, and local robot control, where robot manipulators are controlled at the same location as the operator, are also considered in the review as they can be used in teleoperation with some minor adjustment. A comparison of the strengths, deficiencies, and requirement of methods in each category is carried out. The study indicates that the majority of the proposed methods either require additional hardware such as sensors, or assume an accurate model of the system under study. The possible future research directions are outlined based on the gaps identified in the survey.

Human Action Recognition With Video Data: Research and Evaluation Challenges

Abstract: Given a video sequence, the task of action recognition is to identify the most similar action among the action sequences learned by the system. Such human action recognition is based on evidence gathered from videos. It has wide application including surveillance, video indexing, biometrics, telehealth, and human-computer interaction. Vision-based human action recognition is affected by several challenges due to view changes, occlusion, variation in execution rate, anthropometry, camera motion, and background clutter. In this survey, we provide an overview of the existing methods based on their ability to handle these challenges as well as how these methods can be generalized and their ability to detect abnormal actions. Such systematic classification will help researchers to identify the suitable methods available to address each of the challenges faced and their limitations. In addition, we also identify the publicly available datasets and the challenges posed by them. From this survey, we draw conclusions regarding how well a challenge has been solved, and we identify potential research areas that require further work.

Recent Advances on Singlemodal and Multimodal Face Recognition: A Survey

Abstract: High performance for face recognition systems occurs in controlled environments and degrades with variations in illumination, facial expression, and pose. Efforts have been made to explore alternate face modalities such as infrared (IR) and 3-D for face recognition. Studies also demonstrate that fusion of multiple face modalities improve performance as compared with singlemodal face recognition. This paper categorizes these algorithms into singlemodal and multimodal face recognition and evaluates methods within each category via detailed descriptions of representative work and summarizations in tables. Advantages and disadvantages of each modality for face recognition are analyzed. In addition, face databases and system evaluations are also covered.

Designing and Evaluating a Social Gaze-Control System for a Humanoid Robot

Abstract: This paper describes a context-dependent social gaze-control system implemented as part of a humanoid social robot. The system enables the robot to direct its gaze at multiple humans who are interacting with each other and with the robot. The attention mechanism of the gaze-control system is based on features that have been proven to guide human attention: nonverbal and verbal cues, proxemics, the visual field of view, and the habituation effect. Our gaze-control system uses Kinect skeleton tracking together with speech recognition and SHORE-based facial expression recognition to implement the same features. As part of a pilot evaluation, we collected the gaze behavior of 11 participants in an eye-tracking study. We showed participants videos of two-person interactions and tracked their gaze behavior. A comparison of the human gaze behavior with the behavior of our gaze-control system running on the same videos shows that it replicated human gaze behavior 89% of the time.

Using Smart-Phones and Floor Plans for Indoor Location Tracking

Abstract: We implement pedestrian dead reckoning (PDR) for indoor localization. With a waist-mounted PDR based system on a smart-phone, we estimate the user's step length that utilizes the height change of the waist based on the Pythagorean Theorem. We propose a zero velocity update (ZUPT) method to address sensor drift error: Simple harmonic motion and a low-pass filtering mechanism combined with the analysis of gait characteristics. This method does not require training to develop the step length model. Exploiting the geometric similarity between the user trajectory and the floor map, our map matching algorithm includes three different filters to calibrate the direction errors from the gyro using building floor plans. A sliding-window-based algorithm detects corners. The system achieved 98% accuracy in estimating user walking distance with a waist-mounted phone and 97% accuracy when the phone is in the user's pocket. ZUPT improves sensor drift error (the accuracy drops from 98% to 84% without ZUPT) using 8 Hz as the cut-off frequency to filter out sensor noise. Corner length impacted the corner detection algorithm. In our experiments, the overall location error is about 0.48 meter.

Power-Aware Activity Monitoring Using Distributed Wearable Sensors

Abstract: Monitoring human movements using wireless wearable sensors finds applications in a variety of domains including healthcare and wellness. In these systems, sensory devices are tightly integrated with the human body and infer status of the user through signal and information processing. Typically, highly accurate observations can be made at the cost of deploying a sufficiently large number of sensors, which in turn results in increased energy consumption of the system and reduced adherence to using the system. Therefore, optimizing power consumption of the system while maintaining acceptable accuracy plays a crucial role in realizing these stringent resource constraint systems. In this paper, we present an activity monitoring approach that minimizes power consumption of the system subject to a lower bound on the classification accuracy. The system utilizes computationally simple template-matching blocks that perform classifications on individual sensor nodes. The system further employs a boosting approach to enhance accuracy of the distributed classifier by selecting a subset of sensors optimized in terms of power consumption and capable of achieving a given lower bound accuracy criterion. A proof-of-concept evaluation with three participants performing 14 transitional actions was conducted, where collected signals were segmented and labeled manually for each action. The results indicated that the proposed approach provides more than a 65% reduction in the power consumption of the signal processing, while maintaining 80% sensitivity in classifying human movements.

Augmentation Of Stiffness Perception With a 1-Degree-of-Freedom Skin Stretch Device

Abstract: During tool-mediated interactions with objects, we experience force and fingerpad skin stretch resulting from shear forces caused by friction between the fingerpad skin and the stylus. When probing an object, for the same penetration distance, a stiffer object causes a larger load force and, thus, greater fingerpad skin stretch. We hypothesized that rendering additional artificial skin stretch together with force will increase perceived stiffness. We created a Skin Stretch Stylus that renders skin stretch through tactor displacement, attached it to a force-feedback device, and performed a study to characterize the effect of tactor displacement-induced skin stretch on stiffness perception. Results showed that adding artificial skin stretch causes additive augmentation of perceived stiffness across a range of surface stiffness, and the addition is a linear function of tactor displacement gain. However, intersubject variability in the estimated slope coefficient was large. We propose a model that explains the additive effect and suggests potential sources for the intersubject variability. We conclude that augmenting force feedback with skin stretch can increase users' perception of stiffness, but the effect is user-specific. Such augmentation may be useful in virtual environment and teleoperation scenarios when force feedback gains must be kept low to prevent feedback-induced instabilities, or when force feedback is limited due to actuator force limits.

Cognitive Workload and Learning Assessment During the Implementation of a Next-Generation Air Traffic Control Technology Using Functional Near-Infrared Spectroscopy

Abstract: Neuroimaging technologies, such as functional near- infrared spectroscopy (fNIR), could provide performance metrics directly from brain-based measures to assess safety and perfor- mance of operators in high-risk fields. In this paper, we objectively and subjectively examine the cognitive workload of air traffic con- trol specialists utilizing a next-generation conflict resolution ad- visory. Credible differences were observed between continuously increasing workload levels that were induced by increasing the number of aircraft under control. In higher aircraft counts, a pos- sible saturation in brain activity was realized in the fNIR data. A learning effect was also analyzed across a three-day/nine-session training period. The difference between Day 1 and Day 2 was cred- ible, while there was a noncredible difference between Day 2 and Day 3. The results presented in this paper indicate some advantages in objective measures of cognitive workload assessment with fNIR cortical imaging over the subjective workload assessment keypad.

Assistive Clothing Pattern Recognition for Visually Impaired People

Abstract: Choosing clothes with complex patterns and colors is a challenging task for visually impaired people. Automatic clothing pattern recognition is also a challenging research problem due to rotation, scaling, illumination, and especially large intraclass pattern variations. We have developed a camera-based prototype system that recognizes clothing patterns in four categories (plaid, striped, patternless, and irregular) and identifies 11 clothing colors. The system integrates a camera, a microphone, a computer, and a Bluetooth earpiece for audio description of clothing patterns and colors. A camera mounted upon a pair of sunglasses is used to capture clothing images. The clothing patterns and colors are described to blind users verbally. This system can be controlled by speech input through microphone. To recognize clothing patterns, we propose a novel Radon Signature descriptor and a schema to extract statistical properties from wavelet subbands to capture global features of clothing patterns. They are combined with local features to recognize complex clothing patterns. To evaluate the effectiveness of the proposed approach, we used the CCNY Clothing Pattern dataset. Our approach achieves 92.55% recognition accuracy which significantly outperforms the state-of-the-art texture analysis methods on clothing pattern recognition. The prototype was also used by ten visually impaired participants. Most thought such a system would support more independence in their daily life but they also made suggestions for improvements.

EEG Analysis for Olfactory Perceptual-Ability Measurement Using a Recurrent Neural Classifier

Abstract: A recurrent neural network model is designed to classify (pretrained) aromatic stimuli and discriminate noisy stimuli of both similar and different genres, using EEG analysis of the experimental subjects. The design involves determining the weights of the selected recurrent dynamics so that for a given base stimulus, the dynamics converges to one of several optima (local attractors) on the given Lyapunov energy surface. Experiments undertaken reveal that for small noise amplitude below a selected threshold, the dynamics essentially converges to fixed stable attractor. However, with a slight increase in noise amplitude above the selected threshold, the local attractor of the dynamics shifts in the neighborhood of the attractor obtained for the noise-free standard stimuli. The other important issues undertaken in this paper include a novel algorithm for evolutionary feature selection and data-point reduction from multiple experimental EEG trials using principal component analysis. The confusion matrices constructed from experimental results show a marked improvement in classification accuracy in the presence of data point reduction algorithm. Statistical tests undertaken indicate that the proposed recurrent classifier outperforms its competitors with classification accuracy as the comparator. The importance of this paper is illustrated with a tea-taster selection problem, where an olfactory perceptual-ability measure is used to rank the tasters.

Formal Detection of Attentional Tunneling in Human Operator–Automation Interactions

Abstract: The allocation of visual attention is a key factor for the humans when operating complex systems under time pressure with multiple information sources. In some situations, attentional tunneling is likely to appear and leads to excessive focus and poor decision making. In this study, we propose a formal approach to detect the occurrence of such an attentional impairment that is based on machine learning techniques. An experiment was conducted to provoke attentional tunneling during which psycho-physiological and oculomotor data from 23 participants were collected. Data from 18 participants were used to train an adaptive neuro-fuzzy inference system (ANFIS). From a machine learning point of view, the classification performance of the trained ANFIS proved the validity of this approach. Furthermore, the resulting classification rules were consistent with the attentional tunneling literature. Finally, the classifier was robust to detect attentional tunneling when performing over test data from four participants.

Estimating 3-D Point-of-Regard in a Real Environment Using a Head-Mounted Eye-Tracking System

Abstract: Unlike conventional portable eye-tracking methods that estimate the position of the mounted camera using 2-D image coordinates, the techniques that are proposed here present richer information about person's gaze when moving over a wide area. They also include visualizing scanpaths when the user with a head-mounted device makes natural head movements. We employ a Visual SLAM technique to estimate the head pose and extract environmental information. When the person's head moves, the proposed method obtains a 3-D point-of-regard. Furthermore, scanpaths can be appropriately overlaid on image sequences to support quantitative analysis. Additionally, a 3-D environment is employed to detect objects of focus and to visualize an attention map.

Anticipation in Driving: The Role of Experience in the Efficacy of Pre-event Conflict Cues

Abstract: Anticipation of future events is recognized to be a significant element of driver competence. Surely, guiding one's behavior through the anticipation of future traffic states provides potential gains in recognition and reaction times. However, the role of anticipation in driving has not been systematically studied. In this paper, we identify the characteristics of anticipation in driving and provide a working definition. In particular, we distinguish it from driving goals such as eco or defensive driving and define it as a high-level competence for efficient positioning of the vehicle to facilitate these goals. We also present a driving simulator study assessing the relation between driver experience and anticipation. Thirty drivers from three different experience categories (low, medium, and high) completed five scenarios, each involving several pre-event cues designed to allow the anticipation of an event. The results showed that more experienced drivers demonstrated more pre-event actions compared with less experienced drivers. While pre-event actions resulted in improved safety on certain occasions, the effects were often not significant. Future research should further investigate the mechanisms underlying anticipation, particularly how drivers make use of temporal and spatial gains obtained through the recognition of pre-event cues.

A Gesture Learning Interface for Simulated Robot Path Shaping With a Human Teacher

Abstract: Recognition of human gestures is an active area of research integral for the development of intuitive human-machine interfaces for ubiquitous computing and assistive robotics. In particular, such systems are key to effective environmental designs that facilitate aging in place. Typically, gesture recognition takes the form of template matching in which the human participant is expected to emulate a choreographed motion as prescribed by the researchers. A corresponding robotic action is then a one-to-one mapping of the template classification to a library of distinct responses. In this paper, we explore a recognition scheme based on the growing neural gas (GNG) algorithm that places no initial constraints on the user to perform gestures in a specific way. Motion descriptors extracted from sequential skeletal depth data are clustered by GNG and mapped directly to a robotic response that is refined through reinforcement learning. A simple good/bad reward signal is provided by the user. This paper presents results that show that the topology-preserving quality of GNG allows generalization between gestured commands. Experimental results using an automated reward are presented that compare learning results involving single nodes versus results involving the influence of node neighborhoods. Although separability of input data influences the speed of learning convergence for a given neighborhood radius, it is shown that learning progresses toward emulation of an associative memory that maps input gesture to desired action.

Task Versus Vehicle-Based Control Paradigms in Multiple Unmanned Vehicle Supervision by a Single Operator

Abstract: There has recently been a significant amount of activity in developing supervisory control algorithms for multiple unmanned aerial vehicle operation by a single operator. While previous work has demonstrated the favorable impacts that arise in the introduction of increasingly sophisticated autonomy algorithms, little work has performed an explicit comparison of different types of multiple unmanned vehicle control architectures on operator performance and workload. This paper compares a vehicle-based paradigm (where a single operator individually assigns tasks to unmanned assets) to a task-based paradigm (where the operator generates a task list, which is then given to the group of vehicles that determine how to best divide the tasks among themselves.) The results demonstrate significant advantages in using a task-based paradigm for both overall performance and robustness to increased workload. This effort also demonstrated that while previous video gaming experience mattered for performance, the degree of experience that demonstrated benefit was minimal. Further work should focus on designing a flexible automated system that allows operators to focus on a primary goal, but also facilitate lower level control when needed without degradation in performance.

Improving Smart Conference Participation Through Socially Aware Recommendation

Abstract: This paper addresses recommending presentation sessions at smart conferences to participants. We propose a venue recommendation algorithm: socially aware recommendation of venues and environments (SARVE). SARVE computes correlation and social characteristic information of conference participants. In order to model a recommendation process using distributed community detection, SARVE further integrates the current context of both the smart conference community and participants. SARVE recommends presentation sessions that may be of high interest to each participant. We evaluate SARVE using a real-world dataset. In our experiments, we compare SARVE with two related state-of-the-art methods, namely context-aware mobile recommendation services and conference navigator (recommender) model. Our experimental results show that in terms of the utilized evaluation metrics, i.e., precision, recall, and f-measure, SARVE achieves more reliable and favorable social (relations and context) recommendation results.

Shared Experiences of Technology and Trust: An Experimental Study of Physiological Compliance Between Active and Passive Users in Technology-Mediated Collaborative Encounters

Abstract: —The aim of this study is to examine the utility of phys-iological compliance (PC) to understand shared experience in amultiuser technological environment involving active and passiveusers. Common ground is critical for effective collaboration andimportant for multiuser technological systems that include passiveusers since this kind of user typically does not have control overthe technology being used. An experiment was conducted with 48participants who worked in two-person groups in a multitask envi-ronment under varied task and technology conditions. Indicatorsof PC were measured from participants’ cardiovascular and elec-trodermal activities. The relationship between these PC indicatorsand collaboration outcomes, such as performance and subjectiveperception of the system, was explored. Results indicate that PC isrelated to group performance after controlling for task/technologyconditions. PC is also correlated with shared perceptions of trustin technology among group members. PC is a useful tool for moni-toring group processes and, thus, can be valuable for the design ofcollaborative systems. This study has implications for understand-ing effective collaboration.

Social Cognitive and Affective Neuroscience in Human–Machine Systems: A Roadmap for Improving Training, Human–Robot Interaction, and Team Performance

Abstract: This paper augments recent advances in social cognitive and affective neuroscience (SCAN) and illustrates their relevance to the development of novel human–machine systems. Advances in this area are crucial for understanding and exploring the social, cognitive, and neural processes that arise during human interactions with complex sociotechnological systems. Overviews of the major areas of SCAN research, including emotion, theory of mind, and joint action, are provided as the basis for describing three applications of SCAN to human–machine systems research and development. Specifically, this paper provides three examples to demonstrate the broad interdisciplinary applicability of SCAN and the ways it can contribute to improving a number of human–machine systems with the pursuit of further research in this vein. These include applying SCAN to learning and training, informing the field of human–robot interaction (HRI), and, finally, for enhancing team performance. The goal is to draw attention to the insights that can be gained by integrating SCAN with ongoing human–machine system research and to provide guidance to foster collaborations of this nature. Toward this end, we provide a systematic set of notional research questions for each detailed application within the context of the three major emphases of SCAN research. In turn, this study serves as a roadmap for preliminary investigations that integrate SCAN and human–machine system research.

Regression-Based Facial Expression Optimization

Abstract: This paper presents an approach for reproducing optimal 3-D facial expressions based on blendshape regression. It aims to improve fidelity of facial expressions but maintain the efficiency of the blendshape method, which is necessary for applications such as human–machine interaction and avatars. The method intends to optimize the given facial expression using action units (AUs) based on the facial action coding system recorded from human faces. To help capture facial movements for the target face, an intermediate model space is generated, where both the target and source AUs have the same mesh topology and vertex number. The optimization is conducted interactively in the intermediate model space through adjusting the regulating parameter. The optimized facial expression model is transferred back to the target facial model to produce the final facial expression. We demonstrate that given a sketched facial expression with rough vertex positions indicating the intended facial expression, the proposed method approaches the sketched facial expression through automatically selecting blendshapes with corresponding weights. The sketched expression model is finally approximated through AUs representing true muscle movements, which improves the fidelity of facial expressions.

Experimental Performance Analysis of a Homotopy-Based Shared Autonomy Framework

Abstract: This paper describes and experimentally demonstrates a new approach to shared-adaptive control of human-machine systems. Motivated by observed human proclivity toward fields of safe travel rather than specific trajectories, our approach is rooted in the planning and enforcement of constraints rather than the more traditional reference paths. This approach identifies path homotopies, bounds a desired homotopy with constraints, and allocates control as necessary to ensure that these constraints remain satisfied without unduly restricting the human operator. We present a summary of this framework's technical background and analyze its effect both with and without driver feedback on the performance and confidence of 20 different drivers teleoperating an unmanned (teleoperated) vehicle through an outdoor obstacle course. In 1200 trials, constraint-based semiautonomy was shown to increase the operator speed by 26% while reducing the occurrence of collisions by 78%, and improving overall user confidence and sense of control by 44% and 12%, respectively-all the while assuming less than 43% control of the vehicle.

Effects of Multivantage Point Systems on the Teleoperation of Spacecraft Docking

Abstract: Rendezvous and docking with uncooperative target objects are driving capabilities for future robotic on-orbit servicing and space debris removal systems. A teleoperation system augments a robotic system with the perception, cognition, and decision capabilities of a human operator, which can lead to a more capable and more flexible telerobotic system. The ThirdEye system was developed in order to support the human operator in the complex relative navigation task of final approach and docking. It provides the operator with a flexible camera vantage point which can be positioned freely in the relevant space around and between the chaser and target spacecraft. The primary and secondary camera views, an attitude head-up display, and a trajectory prediction display are integrated into an intuitive graphical user interface. A validation study was conducted to evaluate the effects of this ThirdEye system on the performance of the teleoperation system during final approach and docking with uncooperative, rotating targets. The results of this study show that the ThirdEye system increases the overall task success rate by 15% and improves operator situation awareness, without having negative impact on the usage of system resources. The partial failure rates are decreased by 20-30%. In high-difficulty scenarios, the operator task load is increased due to the dual task of teleoperating the camera arm and the spacecraft in tandem, which leads to a minor increase in failure rate in these scenarios.

Automatically Generating Specification Properties From Task Models for the Formal Verification of Human–Automation Interaction

Abstract: Human-automation interaction (HAI) is often a con- tributor to failures in complex systems. This is frequently due to system interactions that were not anticipated by designers and an- alysts. Model checking is a method of formal verification analysis that automatically proves whether or not a formal system model adheres to desirable specification properties. Task analytic mod- els can be included in formal system models to allow HAI to be evaluated with model checking. However, previous work in this area has required analysts to manually formulate the properties to check. Such a practice can be prone to analyst error and oversight which can result in unexpected dangerous HAI conditions not be- ing discovered. To address this, this paper presents a method for automatically generating specification properties from task models that enables analysts to use formal verification to check for system HAI problems they may not have anticipated. This paper describes the design and implementation of the method. An example (a pilot performing a before landing checklist) is presented to illustrate its utility. Limitations of this approach and future research directions are discussed.

Human Movement Modeling and Activity Perception Based on Fiber-Optic Sensing System

Abstract: This paper presents a flexible fiber-optic sensor-based pressure sensing system for human activity analysis and situation perception in indoor environments. In this system, a binary sensing technology is applied to reduce the data workload, and a bipedal movement-based space encoding scheme is designed to capture people's geometric information. We also develop a nonrepetitive encoding scheme to eliminate the ambiguity caused by the two-foot structure of bipedal movements. Furthermore, we propose an invariant activity representation model based on trajectory segments and their statistical distributions. In addition, a mixture model is applied to represent scenarios. The number of subjects is finally determined by Bayesian information criterion. The Bayesian network and region of interests are employed to facilitate the perception of interactions and situations. The results are obtained using distribution divergence estimation, expectation-maximization, and Bayesian network inference methods. In the experiments, we simulated an office environment and tested walk, work, rest, and talk activities for both one and two person cases. The experiment results have demonstrated that the average individual activity recognition is higher than 90%, and the situation perception rate can achieve 80%.

Driving Assistance by Deictic Control for a Smart Wheelchair: The Assessment Issue

Abstract: In this paper, an assessment of a driving assistance by a deictic command for a smart wheelchair is proposed. This equipment enables the user to move with a series of indications on an interface displaying a view of the environment and bringing about automatic movement of the wheelchair. Two sets of tests were implemented to assess the advantages of this type of assistance compared with conventional wheelchair control. The first set evaluated the performance of the human-machine system that is based on a course time analysis, an observation of users' actions, and an estimation of driving comfort. The second test was implemented to assess the cognitive requirements of the driving task, specifically the attentional and executive processes required when driving in assisted mode. A dual-task method was used to achieve this. The results show that driving assistance brings about a decrease in physical load for the same level of comfort as manual driving, but requires an additional cognitive effort for the user, especially in terms of executive abilities.

Dynamic Operator Overload: A Model for Predicting Workload During Supervisory Control

Abstract: Crandall and Cummings & Mitchell introduced fan-out as a measure of the maximum number of robots a single human operator can supervise in a given single-human-multiple-robot system. Fan-out is based on the time constraints imposed by limitations of the robots and of the supervisor, e.g., limitations in attention. Adapting their work, we introduced a dynamic model of operator overload that predicts failures in supervisory control in real time, based on fluctuations in time constraints and in the supervisor's allocation of attention, as assessed by eye fixations. Operator overload was assessed by damage incurred by unmanned aerial vehicles when they traversed hazard areas. The model generalized well to variants of the baseline task. We then incorporated the model into the system where it predicted in real time, when an operator would fail to prevent vehicle damage and alerted the operator to the threat at those times. These model-based adaptive cues reduced the damage rate by one-half relative to a control condition with no cues.

Enhancing Memory Recall via an Intelligent Social Contact Management System

Abstract: Human memory often fails. People are frequently beset with questions like “Who is that person? I think I met him in Tokyo last year.” Existing memory aid tools cannot well support the recall of names effectively. This paper explores the memory recall enhancement issue from the perspective of memory cue extraction and associative search, and proposes a generic methodology to extract memory cues from heterogeneous, multimodal, physical/virtual data sources. Specifically, we use the contact name recall in the academic community as the target application to showcase our proposed methodology. We further develop an intelligent social contact manager that supports 1) autocollection of rich contact data from a combination of pervasive sensors and Web data sources, and 2) associative search of contacts when human memory fails. The system is validated by testing the performance of contact data collection techniques. An empirical user study on contact memory recall is also conducted, through which several findings about contact memorizing and recall are presented. Classic cognitive psychology theories are used to interpret these findings.