We exploit the falloff of acuity in the visual periphery to accelerate graphics computation by a factor of 5-6 on a desktop HD display (1920x1080). Our method tracks the user's gaze point and renders three image layers around it at progressively higher angular size but lower sampling rate. The three layers are then magnified to display resolution and smoothly composited. We develop a general and efficient antialiasing algorithm easily retrofitted into existing graphics code to minimize "twinkling" artifacts in the lower-resolution layers. A standard psychophysical model for acuity falloff assumes that minimum detectable angular size increases linearly as a function of eccentricity. Given the slope characterizing this falloff, we automatically compute layer sizes and sampling rates. The result looks like a full-resolution image but reduces the number of pixels shaded by a factor of 10-15.We performed a user study to validate these results. It identifies two levels of foveation quality: a more conservative one in which users reported foveated rendering quality as equivalent to or better than non-foveated when directly shown both, and a more aggressive one in which users were unable to correctly label as increasing or decreasing a short quality progression relative to a high-quality foveated reference. Based on this user study, we obtain a slope value for the model of 1.32-1.65 arc minutes per degree of eccentricity. This allows us to predict two future advantages of foveated rendering: (1) bigger savings with larger, sharper displays than exist currently (e.g. 100 times speedup at a field of view of 70° and resolution matching foveal acuity), and (2) a roughly linear (rather than quadratic or worse) increase in rendering cost with increasing display field of view, for planar displays at a constant sharpness.

Foveated 3D graphics

We investigate the potential of the Kinect depth-mapping camera for sign language recognition and verification for educational games for deaf children. We compare a prototype Kinect-based system to our current CopyCat system which uses colored gloves and embedded accelerometers to track children's hand movements. If successful, a Kinect-based approach could improve interactivity, user comfort, system robustness, system sustainability, cost, and ease of deployment. We collected a total of 1000 American Sign Language (ASL) phrases across both systems. On adult data, the Kinect system resulted in 51.5% and 76.12% sentence verification rates when the users were seated and standing respectively. These rates are comparable to the 74.82% verification rate when using the current(seated) CopyCat system. While the Kinect computer vision system requires more tuning for seated use, the results suggest that the Kinect may be a viable option for sign verification.

American sign language recognition with the kinect

Today's health care is difficult to imagine without the possibility to objectively measure various physiological parameters related to patients' symptoms (from temperature through blood pressure to complex tomographic procedures). Psychiatric care remains a notable exception that heavily relies on patient interviews and self-assessment. This is due to the fact that mental illnesses manifest themselves mainly in the way patients behave throughout their daily life and, until recently there were no “behavior measurement devices.” This is now changing with the progress in wearable activity recognition and sensor enabled smartphones. In this paper, we introduce a system, which, based on smartphone-sensing is able to recognize depressive and manic states and detect state changes of patients suffering from bipolar disorder. Drawing upon a real-life dataset of ten patients, recorded over a time period of 12 weeks (in total over 800 days of data tracing 17 state changes) by four different sensing modalities, we could extract features corresponding to all disease-relevant aspects in behavior. Using these features, we gain recognition accuracies of 76% by fusing all sensor modalities and state change detection precision and recall of over 97%. This paper furthermore outlines the applicability of this system in the physician–patient relations in order to facilitate the life and treatment of bipolar patients.

/pdf/smartphone-based-recognition-of-states-and-state-changes-in-4gf5ehghq3.pdf

Smartphone-Based Recognition of States and State Changes in Bipolar Disorder Patients

Order picking (OP) activities, essential to logistics operations, are laborious and time-intensive. Humans are central actors in the OP process and determine both OP effectiveness and efficiency. Many researchers have developed models for planning OP activities and increasing the efficiencies of such systems by suggesting different warehouse layouts, OP routes or storage assignments. These studies have, however, ignored workers’ characteristics, or human factors, suggesting that they cannot be substantiated, which led to only partially realistic results. This paper proposes a conceptual framework for integrating human factors into planning models of OP activities and hypothesises that doing so improves the performance of an OP system and workers’ welfare. The framework is based on a systematic literature review that synthesises findings documented in the OP and human factors literature. The results of the paper may assist researchers and practitioners in designing OP systems by developing planning models ...

Incorporating human factors in order picking planning models: framework and research opportunities

Mental disorders can have a significant, negative impact on sufferers' lives, as well as on their friends and family, healthcare systems and other parts of society. Approximately 25 % of all people in Europe and the USA experience a mental disorder at least once in their lifetime. Currently, monitoring mental disorders relies on subjective clinical self-reporting rating scales, which were developed more than 50 years ago. In this paper, we discuss how mobile phones can support the treatment of mental disorders by (1) implementing human---computer interfaces to support therapy and (2) collecting relevant data from patients' daily lives to monitor the current state and development of their mental disorders. Concerning the first point, we review various systems that utilize mobile phones for the treatment of mental disorders. We also evaluate how their core design features and dimensions can be applied in other, similar systems. Concerning the second point, we highlight the feasibility of using mobile phones to collect comprehensive data including voice data, motion and location information. Data mining methods are also reviewed and discussed. Based on the presented studies, we summarize advantages and drawbacks of the most promising mobile phone technologies for detecting mood disorders like depression or bipolar disorder. Finally, we discuss practical implementation details, legal issues and business models for the introduction of mobile phones as medical devices.

/pdf/mobile-phones-as-medical-devices-in-mental-disorder-1la3im6s54.pdf

Mobile phones as medical devices in mental disorder treatment: an overview

Previous research has established the connection between the way in which children interact with objects and the potential early identification of children with autism. Those findings motivate our own work to develop "smart toys," objects embedded with wireless sensors that are safe and enjoyable for very small children, that allow detailed interaction data to be easily recorded. These sensor-enabled toys provide opportunities for autism research by reducing the effort required to collect and analyze a child's interactions with objects. In the future, such toys may be a useful part of clinical and in-home assessment tools. In this paper, we discuss the design of a collection of smart toys that can be used to automatically characterize the way in which a child is playing. We use statistical models to provide objective, quantitative measures of object play interactions. We also developed a tool to view rich forms of annotated play data for later analysis. We report the results of recognition experiments on more than fifty play sessions conducted with adults and children as well as discuss the opportunities for using this approach to support video annotation and other applications.

https://static.springer.com/sgw/documents/1389130/application/pdf/10.1007_s00779-011-0386-0_PUC_16%2C2.pdf

Monitoring children's developmental progress using augmented toys and activity recognition

Evaluations of task guidance systems often focus on evaluations of new technologies rather than comparing the nuances of interaction across the various systems. One common domain for task guidance systems is warehouse order picking. We present a method involving an easily reproducible ecologically motivated order picking environment for quantitative user studies designed to reveal differences in interactions. Using this environment, we perform a 12 participant within-subjects experiment demonstrating the advantages of a head-mounted display based picking chart over a traditional text-based pick list, a paper-based graphical pick chart, and a mobile pick-by-voice system. The test environment proved sufficiently sensitive, showing statistically significant results along several metrics with the head-mounted display system performing the best. We also provide a detailed analysis of the strategies adopted by our participants.

An empirical task analysis of warehouse order picking using head-mounted displays

Browsing a collection of information on a mobile device is a common task, yet it can be difficult due to the small size of mobile displays. A common trade-off offered by many current mobile interfaces is to allow users to switch between an overview and detailed views of particular items. An open question is how much preview of each item to include in the overview. Using a mobile email processing task, we attempted to answer that question. We investigated participants' email processing behaviors under differing preview conditions in a semi-controlled, naturalistic study. We collected log data of participants' actual behaviors as well as their subjective impressions of different conditions. Our results suggest that a moderate level of two to three lines of preview should be the default. The overall benefit of a moderate amount of preview was supported by both positive subjective ratings and fewer transitions between the overview and individual items.

/pdf/understanding-information-preview-in-mobile-email-processing-4t5ly2uftc.pdf

Understanding information preview in mobile email processing

Language immersion from birth is crucial to a child's language development. However, language immersion can be particularly challenging for hearing parents of deaf children to provide as they may have to overcome many difficulties while learning American Sign Language (ASL). We are in the process of creating a mobile application to help hearing parents learn ASL. To this end, we have interviewed members of our target population to gain understanding of their motivations and needs when learning sign language. We found that the most common motivation for parents learning ASL is better communication with their children. Parents are most interested in acquiring more fluent sign language skills through learning to read stories to their children.

/pdf/we-need-to-communicate-helping-hearing-parents-of-deaf-4v6qlfiac8.pdf

We need to communicate!: helping hearing parents of deaf children learn american sign language

This paper addresses the issue of automatic word/sentence boundary detection in both quiet and noisy environments. We propose to use an entropy based contrast function between the speech segments and the background noise. A simplified data based scheme of computing the entropy of the speech data is presented. The entropy-based contrast exhibits better-behaved characteristics as compared to the energy-based methods. An adaptive threshold is used to determine the candidate speech segments, which are subjected to word/sentence constraints. Experimental. results show that this algorithm outperforms energy-based algorithms. The improved detection accuracy of speech segments results in at least 25% improvement of recognition performance for isolated speech and more than 16% for connected speech. For continuous speech, a preprocessing stage comprising of the proposed speech segment detection makes the overall HMM based scheme more computationally efficient by rejection of silence periods.

/pdf/a-robust-algorithm-for-detecting-speech-segments-using-an-3rnce7h1vx.pdf

Kimberly A. Weaver

Papers

Monitoring children's developmental progress using augmented toys and activity recognition

An empirical task analysis of warehouse order picking using head-mounted displays

Understanding information preview in mobile email processing

We need to communicate!: helping hearing parents of deaf children learn american sign language

A robust algorithm for detecting speech segments using an entropic contrast