Showing papers in "IEEE Transactions on Human-Machine Systems in 2016"

The GRASP Taxonomy of Human Grasp Types

Abstract: In this paper, we analyze and compare existing human grasp taxonomies and synthesize them into a single new taxonomy (dubbed “The GRASP Taxonomy” after the GRASP project funded by the European Commission). We consider only static and stable grasps performed by one hand. The goal is to extract the largest set of different grasps that were referenced in the literature and arrange them in a systematic way. The taxonomy provides a common terminology to define human hand configurations and is important in many domains such as human–computer interaction and tangible user interfaces where an understanding of the human is basis for a proper interface. Overall, 33 different grasp types are found and arranged into the GRASP taxonomy. Within the taxonomy, grasps are arranged according to 1) opposition type, 2) the virtual finger assignments, 3) type in terms of power, precision, or intermediate grasp, and 4) the position of the thumb. The resulting taxonomy incorporates all grasps found in the reviewed taxonomies that complied with the grasp definition. We also show that due to the nature of the classification, the 33 grasp types might be reduced to a set of 17 more general grasps if only the hand configuration is considered without the object shape/size.

Human Interaction With Robot Swarms: A Survey

Abstract: Recent advances in technology are delivering robots of reduced size and cost. A natural outgrowth of these advances are systems comprised of large numbers of robots that collaborate autonomously in diverse applications. Research on effective autonomous control of such systems, commonly called swarms, has increased dramatically in recent years and received attention from many domains, such as bioinspired robotics and control theory. These kinds of distributed systems present novel challenges for the effective integration of human supervisors, operators, and teammates that are only beginning to be addressed. This paper is the first survey of human–swarm interaction (HSI) and identifies the core concepts needed to design a human–swarm system. We first present the basics of swarm robotics. Then, we introduce HSI from the perspective of a human operator by discussing the cognitive complexity of solving tasks with swarm systems. Next, we introduce the interface between swarm and operator and identify challenges and solutions relating to human–swarm communication, state estimation and visualization, and human control of swarms. For the latter, we develop a taxonomy of control methods that enable operators to control swarms effectively. Finally, we synthesize the results to highlight remaining challenges, unanswered questions, and open problems for HSI, as well as how to address them in future works.

Action Recognition From Depth Maps Using Deep Convolutional Neural Networks

Abstract: This paper proposes a new method, i.e., weighted hierarchical depth motion maps (WHDMM) + three-channel deep convolutional neural networks (3ConvNets), for human action recognition from depth maps on small training datasets. Three strategies are developed to leverage the capability of ConvNets in mining discriminative features for recognition. First, different viewpoints are mimicked by rotating the 3-D points of the captured depth maps. This not only synthesizes more data, but also makes the trained ConvNets view-tolerant. Second, WHDMMs at several temporal scales are constructed to encode the spatiotemporal motion patterns of actions into 2-D spatial structures. The 2-D spatial structures are further enhanced for recognition by converting the WHDMMs into pseudocolor images. Finally, the three ConvNets are initialized with the models obtained from ImageNet and fine-tuned independently on the color-coded WHDMMs constructed in three orthogonal planes. The proposed algorithm was evaluated on the MSRAction3D, MSRAction3DExt, UTKinect-Action, and MSRDailyActivity3D datasets using cross-subject protocols. In addition, the method was evaluated on the large dataset constructed from the above datasets. The proposed method achieved 2–9% better results on most of the individual datasets. Furthermore, the proposed method maintained its performance on the large dataset, whereas the performance of existing methods decreased with the increased number of actions.

Personalized Travel Package With Multi-Point-of-Interest Recommendation Based on Crowdsourced User Footprints

Abstract: Location-based social networks (LBSNs) provide people with an interface to share their locations and write reviews about interesting places of attraction. The shared locations form the crowdsourced digital footprints, in which each user has many connections to many locations, indicating user preference to locations. In this paper, we propose an approach for personalized travel package recommendation to help users make travel plans. The approach utilizes data collected from LBSNs to model users and locations, and it determines users’ preferred destinations using collaborative filtering approaches. Recommendations are generated by jointly considering user preference and spatiotemporal constraints. A heuristic search-based travel route planning algorithm was designed to generate travel packages. We developed a prototype system, which obtains users’ travel demands from mobile client and generates travel packages containing multiple points of interest and their visiting sequence. Experimental results suggest that the proposed approach shows promise with respect to improving recommendation accuracy and diversity.

Using Wireless EEG Signals to Assess Memory Workload in the $n$ -Back Task

Abstract: Assessment of mental workload using physiological measures, especially electroencephalography (EEG) signals, is an active area. Recently, a number of wireless acquisition systems to measure EEG and other physiological signals have become available. Few studies have applied such wireless systems to assess cognitive workload and evaluate their performance. This paper presents an initial step to explore the feasibility of a popular wireless system (Emotiv EPOC headset) to assess memory workload levels in a well-known $n$ -back task. We developed a signal processing and classification framework, which integrated an automatic artifact removal algorithm, a broad spectrum of feature extraction techniques, a personalized feature scaling method, an information-theory-based feature selection approach, and a proximal-support-vector-machine-based classification model. The experimental results show that the wirelessly collected EEG signals can be used to classify different memory workload levels for nine participants. The classification accuracies between the lowest workload level (0-back) and active workload levels (1-, 2-, 3-back) were close to 100%. The best classification accuracy for 1- versus 2-back was 80%, and 1- versus 3-back was 84%. This study indicates that the wireless acquisition system and the advanced data analytics and pattern recognition techniques are promising to achieve real-time monitoring and identification of mental workload levels for humans engaged in a wide variety of cognitive activities in the modern society.

Discrete Event Simulation and Virtual Reality Use in Industry: New Opportunities and Future Trends

Abstract: This paper reviews the area of combined discrete event simulation (DES) and virtual reality (VR) use within industry. While establishing a state of the art for progress in this area, this paper makes the case for VR DES as the vehicle of choice for complex data analysis through interactive simulation models, highlighting both its advantages and current limitations. This paper reviews active research topics such as VR and DES real-time integration, communication protocols, system design considerations, model validation, and applications of VR and DES. While summarizing future research directions for this technology combination, the case is made for smart factory adoption of VR DES as a new platform for scenario testing and decision making. It is put that in order for VR DES to fully meet the visualization requirements of both Industry 4.0 and Industrial Internet visions of digital manufacturing, further research is required in the areas of lower latency image processing, DES delivery as a service, gesture recognition for VR DES interaction, and linkage of DES to real-time data streams and Big Data sets.

Air-Writing Recognition—Part I: Modeling and Recognition of Characters, Words, and Connecting Motions

Abstract: Air-writing refers to writing of linguistic characters or words in a free space by hand or finger movements. Air-writing differs from conventional handwriting; the latter contains the pen-up-pen-down motion, while the former lacks such a delimited sequence of writing events. We address air-writing recognition problems in a pair of companion papers. In Part I, recognition of characters or words is accomplished based on six-degree-of-freedom hand motion data. We address air-writing on two levels: motion characters and motion words. Isolated air-writing characters can be recognized similar to motion gestures although with increased sophistication and variability. For motion word recognition in which letters are connected and superimposed in the same virtual box in space, we build statistical models for words by concatenating clustered ligature models and individual letter models. A hidden Markov model is used for air-writing modeling and recognition. We show that motion data along dimensions beyond a 2-D trajectory can be beneficially discriminative for air-writing recognition. We investigate the relative effectiveness of various feature dimensions of optical and inertial tracking signals and report the attainable recognition performance correspondingly. The proposed system achieves a word error rate of 0.8% for word-based recognition and 1.9% for letter-based recognition. We also subjectively and objectively evaluate the effectiveness of air-writing and compare it with text input using a virtual keyboard. The words per minute of air-writing and virtual keyboard are 5.43 and 8.42, respectively.

Toward Personalized Activity Recognition Systems With a Semipopulation Approach

Abstract: Activity recognition is a key component of context-aware computing to support people's physical activity, but conventional approaches often lack in their generalizability and scalability due to problems of diversity in how individuals perform activities, overfitting when building activity models, and collection of a large amount of labeled data from end users. To address these limitations, we propose a semipopulation-based approach that exploits activity models trained from other users; therefore, a new user does not need to provide a large volume of labeled activity data. Instead of relying on any additional information from users like their weight or height, our approach directly measures the fitness of others’ models on a small amount of labeled data collected from the new user. With these shared activity models among users, we compose a hybrid model of Bayesian networks and support vector machines to accurately recognize the activity of the new user. On activity data collected from 28 people with a diversity in gender, age, weight, and height, our approach produced an average accuracy of 83.4% (kappa: 0.852), compared with individual and (standard) population models that had accuracies of 77.3% (kappa: 0.79) and 77.7% (kappa: 0.743), respectively. Through an analysis on the performance of our approach and users’ demographic information, our approach outperforms others that rely on users’ demographic information for recognizing their activities, which may contradict the commonly held belief that physically similar people would have similar activity patterns.

“Teach Me–Show Me”—End-User Personalization of a Smart Home and Companion Robot

Abstract: Care issues and costs associated with an increasing elderly population are becoming a major concern for many countries. The use of assistive robots in “smart-home” environments has been suggested as a possible partial solution to these concerns. A challenge is the personalization of the robot to meet the changing needs of the elderly person over time. One approach is to allow the elderly person, or their carers or relatives, to make the robot learn activities in the smart home and teach it to carry out behaviors in response to these activities. The overriding premise being that such teaching is both intuitive and “nontechnical.” To evaluate these issues, a commercially available autonomous robot has been deployed in a fully sensorized but otherwise ordinary suburban house. We describe the design approach to the teaching, learning, robot, and smart home systems as an integrated unit and present results from an evaluation of the teaching component with 20 participants and a preliminary evaluation of the learning component with three participants in a human–robot interaction experiment. Participants reported findings using a system usability scale and ad-hoc Likert questionnaires. Results indicated that participants thought that this approach to robot personalization was easy to use, useful, and that they would be capable of using it in real-life situations both for themselves and for others.

Driver Assistance System With a Dual Control Scheme: Effectiveness of Identifying Driver Drowsiness and Preventing Lane Departure Accidents

Abstract: Driver drowsiness is a common cause of fatal traffic accidents. In this study, a driver assistance system with a dual control scheme is developed; it attempts to perform simultaneously the safety control of the vehicle and identification of the driver's state. The assistance system implements partial control in the event of lane departure and gives the driver the chance to voluntarily take the action needed. If the driver fails to implement the steering action needed within a limited time, the assistance system judges that “the driver's understanding of the given situation is incorrect” and executes the remaining control. We used a driving simulator equipped with the assistance system to investigate the effectiveness of identifying driver drowsiness and preventing lane departure accidents. Twenty students participated in three trials on a straight expressway, and they were required to implement only lateral control. We hypothesized that a participant cannot implement the action needed to maintain safety when he/she falls asleep, and that in such a case, the assistance system will implement the safety control repeatedly. The assistance system assisted the participants only when almost really needed, such as when their eyelids were closed. The results validated the hypothesis, showing that the assistance system implemented the safety control repeatedly when a participant fell asleep. In addition, the algorithms used by the assistance system to determine whether the driver can continue driving were evaluated through a leave-one-out cross-validation, and they were proven to be effective for identifying driver drowsiness.

Air-Writing Recognition—Part II: Detection and Recognition of Writing Activity in Continuous Stream of Motion Data

Abstract: Air-writing refers to writing of characters or words in the free space by hand or finger movements. We address air-writing recognition problems in two companion papers. Part 2 addresses detecting and recognizing air-writing activities that are embedded in a continuous motion trajectory without delimitation. Detection of intended writing activities among superfluous finger movements unrelated to letters or words presents a challenge that needs to be treated separately from the traditional problem of pattern recognition. We first present a dataset that contains a mixture of writing and nonwriting finger motions in each recording. The LEAP from Leap Motion is used for marker-free and glove-free finger tracking. We propose a window-based approach that automatically detects and extracts the air-writing event in a continuous stream of motion data, containing stray finger movements unrelated to writing. Consecutive writing events are converted into a writing segment. The recognition performance is further evaluated based on the detected writing segment. Our main contribution is to build an air-writing system encompassing both detection and recognition stages and to give insights into how the detected writing segments affect the recognition result. With leave-one-out cross validation, the proposed system achieves an overall segment error rate of 1.15% for word-based recognition and 9.84% for letter-based recognition.

Movement Primitive Segmentation for Human Motion Modeling: A Framework for Analysis

Abstract: Movement primitive segmentation enables long sequences of human movement observation data to be segmented into smaller components, termed movement primitives , to facilitate movement identification, modeling, and learning. It has been applied to exercise monitoring, gesture recognition, human–machine interaction, and robot imitation learning. This paper proposes a segmentation framework to categorize and compare different segmentation algorithms considering segment definitions, data sources, application-specific requirements, algorithm mechanics, and validation techniques. The framework is applied to human motion segmentation methods by grouping them into online, semionline, and offline approaches. Among the online approaches, distance-based methods provide the best performance, while stochastic dynamic models work best in the semionline and offline settings. However, most algorithms to date are tested with small datasets, and algorithm generalization across participants and to movement changes remains largely untested.

Optimized Bi-Objective EEG Channel Selection and Cross-Subject Generalization With Brain–Computer Interfaces

Abstract: Electroencephalography (EEG) signal processing to decode motor imagery (MI) involves high-dimensional features, which increases the computational complexity. To reduce this computational burden due to the large number of channels, an iterative multiobjective optimization for channel selection (IMOCS) is proposed in this paper. For a given MI classification task, the proposed method initializes a reference candidate solution and subsequently finds a set of the most relevant channels in an iterative manner by exploiting both the anatomical and functional relevance of EEG channels. The proposed approach is evaluated on the Wadsworth dataset for the right fist versus left fist MI tasks, while considering the cross-validation accuracy as the performance evaluation criteria. Furthermore, 12 other dimension reduction and channel selection algorithms are used for benchmarking. The proposed approach (IMOCS) achieved an average classification accuracy of about 80% when evaluated using 35 best-performing subjects. One-way analysis of variance revealed the statistical significance of the proposed approach with at least 7% improvement over other benchmarking algorithms. Furthermore, a cross-subject generalization of channel selection on untrained subjects shows that the subject-independent channels perform as good as using all channels achieving an average classification accuracy of 61%. These results are promising for the online brain–computer interface (BCI) paradigm that requires low computational complexity and also for reducing the preparation time while conducting multiple session BCI experiments for a larger pool of subjects.

Toward Reliable Autonomous Robotic Assistants Through Formal Verification: A Case Study

Abstract: It is essential for robots working in close proximity to people to be both safe and trustworthy. We present a case study on formal verification for a high-level planner/scheduler for the Care-O-bot, an autonomous personal robotic assistant. We describe how a model of the Care-O-bot and its environment was developed using Brahms, a multiagent workflow language. Formal verification was then carried out by automatically translating this model to the input language of an existing model checker. Four sample properties based on system requirements were verified. We then refined the environment model three times to increase its accuracy and the persuasiveness of the formal verification results. The first refinement uses a user activity log based on real-life experiments, but is deterministic. The second refinement uses the activities from the user activity log nondeterministically. The third refinement uses “conjoined activities” based on an observation that many user activities can overlap. The four samples properties were verified for each refinement of the environment model. Finally, we discuss the approach of environment model refinement with respect to this case study.

A Multimodal Gaze-Controlled Virtual Keyboard

Abstract: New portable and noninvasive eye-trackers allow the creation of robust virtual keyboards that aim to improve the life of disabled people who are unable to communicate. This paper presents a novel multimodal virtual keyboard and evaluates the performance changes that occur with the use of different modalities. The virtual keyboard is based on a menu selection with eight main commands that allow us to spell 30 different characters and correct errors with a delete button. The system has been evaluated with 18 adult participants in three conditions corresponding to three modalities: direct selection using a mouse, with the eye-tracker to point at the desired command and a switch to select it, and with only the eye-tracker for command selection. The performance of the proposed virtual keyboard was evaluated by the speed and information transfer rate (ITR) at both the command and application levels. The average speed across subjects was 18.43 letters/min with the mouse only, 15.26 letters/min with the eye-tracker and the switch, and 9.30 letters/min with only the eye-tracker. The later provided an ITR of 44.96 and 57.46 bits/min at the letter and command levels, respectively. The results show to what extent a drop of performance can occur when switching between several modalities. While the speed decreases when controlling the virtual keyboard with the eye-tracker only, the system's performance remains functioning for severely disabled people who have their gaze as one of their only means of communication.

Palmprint Recognition Using Neighboring Direction Indicator

Abstract: Orientation features are successfully used in coding-based palmprint recognition methods. In this paper, we propose a discriminative neighboring direction indicator to represent the orientation feature of the palmprint. The neighboring direction indicator feature not only represents the most dominant orientation feature of the palmprint, but also better describes the orientation feature of those points which have double dominant orientations. In addition, the neighboring direction indicator shows good robustness to noise and rotation. Using the neighboring direction indicator, we propose a novel palmprint recognition method. Extensive experiments conducted on three types of palmprint databases demonstrate that the proposed method gives better performance than the existing state-of-the-art orientation-based methods. By using the proposed method, the equal error rate is improved by about 10% for palmprint verification, and the average error rate is reduced by 2.7–14% for palmprint identification with a single training sample.

Badminton Stroke Recognition Based on Body Sensor Networks

Abstract: A badminton training system based on body sensor networks has been proposed. The system may recognize different badminton strokes of badminton players. A two-layer hidden Markov model (HMM) classification algorithm is proposed to recognize 14 types of badminton strokes. In the first layer, we use acceleration magnitude of the right wrist to determine a threshold to detect strokes, and then, the HMM is applied to filter out nonstroke motions. In the second layer, we adopt the HMM to classify all the strokes into 14 categories. Experimental results show that the two-layer HMM can achieve good recognition accuracy. The effectiveness and feasibility of the two-layer HMM classification algorithm have been verified in a comparison.

Eye-Gaze Tracking Analysis of Driver Behavior While Interacting With Navigation Systems in an Urban Area

Abstract: With the advent of global positioning system technology, smart phones are used as portable navigation systems. Guidelines that ensure driving safety while using conventional on-board navigation systems have already been published but do not extend to portable navigation systems; therefore, this study focused on the analysis of the eye-gaze tracking of drivers interacting with portable navigation systems in an urban area. Combinations of different display sizes and positions of portable navigation systems were adopted by 20 participants in a driving simulator experiment. An expectation maximum algorithm was proposed to classify the measured eye-gaze points; furthermore, three measures of glance frequency, glance time, and total glance time as a percentage were calculated. The results indicated that the convenient display position with a small visual angle can provide a significantly shorter glance time but a significantly higher glance frequency; however, the small-size display will bring on significantly longer glance time that may result in the increasing of visual distraction for drivers. The small-size portable display received significantly lower scores for subjective evaluation of acceptability and fatigue; moreover, the small-size portable display on the conventional built-in position received significantly lower subjective evaluation scores than that of the big-size one on the upper side of the dashboard. In addition, it indicated an increased risk of rear-end collision that the proportion of time that the time-to-collision was less than 1 s was significantly shorter for the portable navigation than that of traditional on-board one.

Smoothness of Surgical Tool Tip Motion Correlates to Skill in Endovascular Tasks

Abstract: Current performance assessment techniques in endovascular surgery are subjective or limited to grading scales based solely on an expert's observation of a novice's task execution. Since most endovascular procedures involve performing fine motor control tasks that require complex dexterous movements, this paper evaluates objective and quantitative metrics of performance that capture movement quality through the computation of tool tip movement smoothness. An experiment was designed that involved recording the catheter tip movement from 20 subjects performing four fundamental endovascular tasks in each of three sessions using manual catheterization on a physical model and in a simulation environment. Several motion-based performance measures that have been shown to reliably assess skill in other domains were computed and tested for correlation with subjective data that were simultaneously obtained from the global rating scale assessment tool. Metrics that captured movement smoothness produced statistically significant correlations with the observation-based assessment metrics and were able to differentiate skill among participants. In particular, submovement analysis led to metrics that captured statistically significant differences across ability group, session, experimental platform, and task. Objective and quantitative metrics that capture movement smoothness could be incorporated into future training protocols to provide detailed feedback on trainee performance.

iLeg —A Lower Limb Rehabilitation Robot: A Proof of Concept

Abstract: In this paper, a robot, namely iLeg , is designed for the purpose of rehabilitation of patients with hemiplegia or paraplegia. The iLeg is composed of one reclining seat and two leg orthoses, and each leg orthosis has three degrees of freedom, which correspond to the hip, knee, and ankle. Based on this robotic system, two controllers, i.e., passive training controller and active training controller, are proposed. The former takes advantage of the proportional-integral control method to solve the trajectory tracking problem, and the latter employs the surface electromyography signals to achieve active training. Two simplified impedance controllers, i.e., damping-type velocity controller and spring-type position controller, are designed for active training. A perceptron neural network detects movement intentions. The performance of the controllers was investigated with one able-bodied male. The results showed that the leg orthosis tracked the predefined trajectory based on the passive training controller, with the error rates of $0.45\%$ , $0.44\%$ , and $0.27\%$ , respectively, for the hip, knee, and ankle. The active training controller whose loop rate is 6.67 Hz can move the leg orthosis smoothly, and the average recognition error of the perceptron neural network is less than $5\%$ .

MobiGroup: Enabling Lifecycle Support to Social Activity Organization and Suggestion With Mobile Crowd Sensing

Abstract: This paper presents a group-aware mobile crowd sensing system called MobiGroup, which supports group activity organization in real-world settings. Acknowledging the complexity and diversity of group activities, this paper introduces a formal concept model to characterize group activities and classifies them into four organizational stages. We then present an intelligent approach to support group activity preparation, including a heuristic rule-based mechanism for advertising public activity and a context-based method for private group formation. In addition, we leverage features extracted from both online and offline communities to recommend ongoing events to attendees with different needs. Compared with the baseline method, people preferred public activities suggested by our heuristic rule-based method. Using a dataset collected from 45 participants, we found that the context-based approach for private group formation can attain a precision and recall of over 80%, and the usage of spatial–temporal contexts and group computing can have more than a 30% performance improvement over considering the interaction frequency between a user and related groups. A case study revealed that, by extracting the features such as dynamic intimacy and static intimacy, our cross-community approach for ongoing event recommendation can meet different user needs.

Human Mobility Modeling for Robot-Assisted Evacuation in Complex Indoor Environments

Abstract: A large number of injuries or deaths may occur when an emergency happens in a crowded public place. The congestion at exits may slow down the egress rate due to the effect of “faster-is-slower”. This inspires us to study how human behavior dynamically changes over time at an emergency in a complex indoor environment. In this paper, we refer the panic of evacuees to their perception of the threat and propose a panic propagation model to model how crowd panic changes during evacuation at an emergency. Combined with the existing social force model, our panic model interprets the self-driven force and interactive forces with others in human mobility. To improve evacuation efficiency, robots are introduced to guide evacuees to escape. Using dynamic environment information, we design an evacuation exit selection algorithm where the optimal exit is automatically selected by the robot with the minimum escape time. In our experiments, a real shopping mall is examined, and the dynamic behavior of panicked evacuees is simulated with the proposed panic model. The evacuation performance of using emergency evacuation robots is evaluated. The improvement of evacuation efficiency validates the effectiveness of our robot-assisted evacuation system.

Exploring Latent Preferences for Context-Aware Personalized Recommendation Systems

Abstract: Context-aware recommendations offer the potential of exploiting social contents and utilize related tags and rating information to personalize the search for content considering a given context. Recommendation systems tackle the problem of trying to identify relevant resources from the vast number of choices available online. In this study, we propose a new recommendation model that personalizes recommendations and improves the user experience by analyzing the context when a user wishes to access multimedia content. We conducted empirical analysis on a dataset from last.fm to demonstrate the use of latent preferences for ranking items under a given context. Additionally, we use an optimization function to maximize the mean average precision measure of the resulted recommendation. Experimental results show a potential improvement to the quality of the recommendation in terms of accuracy when compared with state-of-the-art algorithms.

Biometric-Enabled Authentication Machines: A Survey of Open-Set Real-World Applications

Abstract: This paper revisits the concept of an authentication machine (A-machine) that aims at identifying/verifying humans. Although A-machines in the closed-set application scenario are well understood and commonly used for access control utilizing human biometrics (face, iris, and fingerprints), open-set applications of A-machines have yet to be equally characterized. This paper presents an analysis and taxonomy of A-machines, trends, and challenges of open-set real-world applications. This paper makes the following contributions to the area of open-set A-machines: 1) a survey of applications; 2) new novel life cycle metrics for theoretical, predicted, and operational performance evaluation; 3) a new concept of evidence accumulation for risk assessment; 4) new criteria for the comparison of A-machines based on the notion of a supporting assistant; and 5) a new approach to border personnel training based on the A-machine training mode. It offers a technique for modeling A-machines using belief (Bayesian) networks and provides an example of this technique for biometric-based e-profiling.

Gaze Estimation From Color Image Based on the Eye Model With Known Head Pose

Abstract: This paper proposes a novel method of gaze estimation based on an eye model with known head pose. The most crucial factors in the eye-model-based approach to gaze estimation are the 3-D positions of the eyeball and iris centers. In the proposed method, an RGB-D camera, Kinect sensor, is used to obtain the head pose as well as the eye region of the color image. The 3-D position of the eyeball center is determined in the calibration phase by gazing at the center of the color image camera. Then, to estimate the 3-D position of the iris center, the 3-D contour of the iris is projected onto the color image with the known head pose obtained from color and depth cues of an RGB-D camera. Thus, the ellipse of the iris in the image can be described using only two parameters: the yaw and pitch angles of the eyeball in the iris coordinate system, rather than the conventional five parameters of an ellipse. The proposed method can fit an iris that is not complete due to eyelid occlusion. The average errors of vertical and horizontal angles of the gaze estimation for seven subjects are 5.9° and 4.4°, respectively. The processing speed is as high as 330 ms per frame. However, for lower resolution and poor illumination images, as tested on the public database EYEDIAP, the performance of the proposed eye-model-based method is inferior to that of the-state-of-the-art appearance-based method.

Graphical Password-Based User Authentication With Free-Form Doodles

Abstract: User authentication using simple gestures is now common in portable devices. In this work, authentication with free-form sketches is studied. Verification systems using dynamic time warping and Gaussian mixture models are proposed, based on dynamic signature verification approaches. The most discriminant features are studied using the sequential forward floating selection algorithm. The effects of the time lapse between capture sessions and the impact of the training set size are also studied. Development and validation experiments are performed using the DooDB database, which contains passwords from 100 users captured on a smartphone touchscreen. Equal error rates between 3% and 8% are obtained against random forgeries and between 21% and 22% against skilled forgeries. High variability between capture sessions increases the error rates.

A Multisensor Technique for Gesture Recognition Through Intelligent Skeletal Pose Analysis

Abstract: Recent advances in smart sensor technology and computer vision techniques have made the tracking of unmarked human hand and finger movements possible with high accuracy and at sampling rates of over 120 Hz. However, these new sensors also present challenges for real-time gesture recognition due to the frequent occlusion of fingers by other parts of the hand. We present a novel multisensor technique that improves the pose estimation accuracy during real-time computer vision gesture recognition. A classifier is trained offline, using a premeasured artificial hand, to learn which hand positions and orientations are likely to be associated with higher pose estimation error. During run-time, our algorithm uses the prebuilt classifier to select the best sensor-generated skeletal pose at each time step, which leads to a fused sequence of optimal poses over time. The artificial hand used to establish the ground truth is configured in a number of commonly used hand poses such as pinches and taps. Experimental results demonstrate that this new technique can reduce total pose estimation error by over 30% compared with using a single sensor, while still maintaining real-time performance. Our evaluations also demonstrate that our approach significantly outperforms many other alternative approaches such as weighted averaging of hand poses. An analysis of our classifier performance shows that the offline training time is insignificant, and our configuration achieves about 90.8% optimality for the dataset used. Our method effectively increases the robustness of touchless display interactions, especially in high-occlusion situations by analyzing skeletal poses from multiple views.

Predictive Communication Quality Control in Haptic Teleoperation With Time Delay and Packet Loss

Abstract: Teleoperation in extreme environments may suffer from communication delay and packet loss during the transmission of command signals and sensory feedback. This study investigates whether online control of communication time delay by using quality of service (QoS) techniques can improve operator task performance in a virtual teleoperated collision avoidance task. We first introduce the framework of predictive communication quality control based on a dynamic performance model of a human handling the teleoperation system. We then apply the framework to a virtual collision avoidance scenario and evaluate it with two behavioral studies. Study 1 identifies that prolonging time delay significantly increases the frequency of collisions and completion time. We develop a model for predicting the probability of the operator causing collisions with the wall and fit its parameters with the experimental data. In Study 2, we compare the completion time and the number of collisions with and without the predictive QoS control. It is shown that the predictive QoS control is capable of reducing the number of collisions, but it does not affect task completion time. The prediction model and empirical validation provide a successful proof of concept for a human-centered system design, in which the dynamic model of the operator is the center of the control architecture.

Hierarchical Sample-Based Joint Probabilistic Data Association Filter for Following Human Legs Using a Mobile Robot in a Cluttered Environment

Abstract: Human-following in a cluttered environment is one of the challenging issues for mobile service robot applications. Since a laser range finder (LRF) is commonly installed for autonomous navigation, it is advantageous to adopt an LRF for detection and tracking humans. In this paper, we aim at the reliable human tracking performances in a dynamic cluttered environment. The key idea is to develop a hierarchical sample-based joint probabilistic data association filter (HSJPDAF) by focusing on the leg positions as well as human positions. The proposed HSJPDAF consists of two levels in order to consider the interdependence between targets. Possible locations of multiple human targets can be simultaneously estimated on the basis of Bayesian filtering. Comparison with the general technique was carried out to verify the performance of HSJPDAF in both artificial indoor and real-world environments. Owing to the hierarchical framework, the proposed method shows the improved robustness by reducing the target loss rate significantly in a dynamic cluttered environment.

Strategic Conformance: Overcoming Acceptance Issues of Decision Aiding Automation?

Abstract: Cognitive engineering researchers have long studied the complexity and reliability of human–automation interaction. Historically, though, the area of human–automation decision-making compatibility has received less attention. Paradoxically, this could in the future become one of the most critical issues of all, as mismatches between human and automation problem-solving styles could threaten the adoption of automation. This paper presents the concept of strategic conformance as a potential key factor influencing initial acceptance of automation, specifically decision aiding systems capable of guiding decision and action. Here, strategic conformance represents the match in problem-solving style between decision aiding automation and the individual operator. The theoretical foundation builds on the compatibility construct found in technology acceptance theories such as the innovation diffusion and technology acceptance models. The paper concludes with a critical discussion on the limitations and drawbacks of strategic conformance. It is proposed that the construct would be most applicable at the introductory phase of new decision aiding automation, in helping to foster operators’ initial acceptance of such automation.