A method for controlling vehicle systems includes receiving monitoring information from one or more monitoring systems and determining a plurality of driver states based on the monitoring information from the one or more monitoring systems. The method includes determining a combined driver state based on the plurality of driver states and modifying control of one or more vehicle systems based on the combined driver state.

System and method for responding to driver state

This paper proposes a novel machine learning architecture, specifically designed for radio-frequency based gesture recognition. We focus on high-frequency (60]GHz), short-range radar based sensing, in particular Google's Soli sensor. The signal has unique properties such as resolving motion at a very fine level and allowing for segmentation in range and velocity spaces rather than image space. This enables recognition of new types of inputs but poses significant difficulties for the design of input recognition algorithms. The proposed algorithm is capable of detecting a rich set of dynamic gestures and can resolve small motions of fingers in fine detail. Our technique is based on an end-to-end trained combination of deep convolutional and recurrent neural networks. The algorithm achieves high recognition rates (avg 87%) on a challenging set of 11 dynamic gestures and generalizes well across 10 users. The proposed model runs on commodity hardware at 140 Hz (CPU only).

https://dl.acm.org/doi/pdf/10.1145/2984511.2984565

Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum

Touche proposes a novel Swept Frequency Capacitive Sensing technique that can not only detect a touch event, but also recognize complex configurations of the human hands and body. Such contextual information significantly enhances touch interaction in a broad range of applications, from conventional touchscreens to unique contexts and materials. For example, in our explorations we add touch and gesture sensitivity to the human body and liquids. We demonstrate the rich capabilities of Touche with five example setups from different application domains and conduct experimental studies that show gesture classification accuracies of 99% are achievable with our technology.

Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects

This paper introduces a low cost, fast and accessible technology to support the rapid prototyping of functional electronic devices Central to this approach of 'instant inkjet circuits' is the ability to print highly conductive traces and patterns onto flexible substrates such as paper and plastic films cheaply and quickly In addition to providing an alternative to breadboarding and conventional printed circuits, we demonstrate how this technique readily supports large area sensors and high frequency applications such as antennas Unlike existing methods for printing conductive patterns, conductivity emerges within a few seconds without the need for special equipment We demonstrate that this technique is feasible using commodity inkjet printers and commercially available ink, for an initial investment of around US$300 Having presented this exciting new technology, we explain the tools and techniques we have found useful for the first time Our main research contribution is to characterize the performance of instant inkjet circuits and illustrate a range of possibilities that are enabled by way of several example applications which we have built We believe that this technology will be of immediate appeal to researchers in the ubiquitous computing domain, since it supports the fabrication of a variety of functional electronic device prototypes

/pdf/instant-inkjet-circuits-lab-based-inkjet-printing-to-support-3d9184pzat.pdf

Instant inkjet circuits: lab-based inkjet printing to support rapid prototyping of UbiComp devices

Smartwatches and wearables are unique in that they reside on the body, presenting great potential for always-available input and interaction. Their position on the wrist makes them ideal for capturing bio-acoustic signals. We developed a custom smartwatch kernel that boosts the sampling rate of a smartwatch's existing accelerometer to 4 kHz. Using this new source of high-fidelity data, we uncovered a wide range of applications. For example, we can use bio-acoustic data to classify hand gestures such as flicks, claps, scratches, and taps, which combine with on-device motion tracking to create a wide range of expressive input modalities. Bio-acoustic sensing can also detect the vibrations of grasped mechanical or motor-powered objects, enabling passive object recognition that can augment everyday experiences with context-aware functionality. Finally, we can generate structured vibrations using a transducer, and show that data can be transmitted through the human body. Overall, our contributions unlock user interface techniques that previously relied on special-purpose and/or cumbersome instrumentation, making such interactions considerably more feasible for inclusion in future consumer devices.

ViBand: High-Fidelity Bio-Acoustic Sensing Using Commodity Smartwatch Accelerometers

At present, touchscreens can differentiate multiple points of contact, but not who is touching the device. In this work, we consider how the electrical properties of humans and their attire can be used to support user differentiation on touchscreens. We propose a novel sensing approach based on Swept Frequency Capacitive Sensing, which measures the impedance of a user to the environment (i.e., ground) across a range of AC frequencies. Different people have different bone densities and muscle mass, wear different footwear, and so on. This, in turn, yields different impedance profiles, which allows for touch events, including multitouch gestures, to be attributed to a particular user. This has many interesting implications for interactive design. We describe and evaluate our sensing approach, demonstrating that the technique has considerable promise. We also discuss limitations, how these might be overcome, and next steps.

Capacitive fingerprinting: exploring user differentiation by sensing electrical properties of the human body

Botanicus Interacticus is a technology for designing highly expressive interactive plants, both living and artificial. We are motivated by the rapid fusion of computing and our dwelling spaces, as well as the increasingly tactile and gestural nature of our interactions with digital devices. Today, however, this interaction happens either on the touch screens of tablet computers and smart phones, or in free air, captured by camera-based devices, such as the Kinect. What if, instead of this limited range of devices, a broad variety of objects in living, social and working spaces become aware and responsive to human presence, touch and gesture?

Botanicus Interacticus: interactive plants technology

In this research, we aim to realize a gustatory display that enhances our experience of enjoying food. However, generating a sense of taste is very difficult because the human gustatory system is quite complicated and is not yet fully understood. This is so because gustatory sensation is based on chemical signals whereas visual and auditory sensations are based on physical signals. In addition, the brain perceives flavor by combining the senses of gustation, smell, sight, warmth, memory, etc. The aim of our research is to apply the complexity of the gustatory system in order to realize a pseudo-gustatory display that presents flavors by means of visual feedback. This paper reports on the prototype system of such a display that enables us to experience various tastes without changing their chemical composition through the superimposition of virtual color. The fundamental thrust of our experiment is to evaluate the influence of cross-sensory effects by superimposing virtual color onto actual drinks and recording the responses of subjects who drink them. On the basis of experimental results, we concluded that visual feedback sufficiently affects our perception of flavor to justify the construction of pseudo-gustatory displays.

Evaluating cross-sensory perception of superimposing virtual color onto real drink: toward realization of pseudo-gustatory displays

Surface and object recognition is of significant importance in ubiquitous and wearable computing. While various techniques exist to infer context from material properties and appearance, they are typically neither designed for real-time applications nor for optically complex surfaces that may be specular, textureless, and even transparent. These materials are, however, becoming increasingly relevant in HCI for transparent displays, interactive surfaces, and ubiquitous computing. We present SpecTrans, a new sensing technology for surface classification of exotic materials, such as glass, transparent plastic, and metal. The proposed technique extracts optical features by employing laser and multi-directional, multi-spectral LED illumination that leverages the material's optical properties. The sensor hardware is small in size, and the proposed classification method requires significantly lower computational cost than conventional image-based methods, which use texture features or reflectance analysis, thereby providing real-time performance for ubiquitous computing. Our evaluation of the sensing technique for nine different transparent materials, including air, shows a promising recognition rate of 99.0%. We demonstrate a variety of possible applications using SpecTrans' capabilities.

Munehiko Sato

Papers

Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects

Capacitive fingerprinting: exploring user differentiation by sensing electrical properties of the human body

Botanicus Interacticus: interactive plants technology

Evaluating cross-sensory perception of superimposing virtual color onto real drink: toward realization of pseudo-gustatory displays

SpecTrans: Versatile Material Classification for Interaction with Textureless, Specular and Transparent Surfaces