Computer-readable media, computer systems, and computing devices for enabling a user to view content related to an item within media content is provided. In one embodiment, a tagging gesture that indicates a user interest in an item presented within media content displayed to the user is detected. Such a tagging gesture can define a region of displayed content having the item. In response to detecting the tagging gesture, an advertisement provided by a third party that is related to the item is presented.

Gesture-based tagging to view related content

Developing an accurate activity recognition system with sensor readings is a challenging task due to the fact that sensors are subjected to a variety of errors and the nature of human activities is dynamic and uncertainties. Recent studies have shown that machine learning approaches can effectively classify human activities. In this study, we analyze sensor readings from accelerometers and gyroscopes using long short-term memory (LSTM) recurrent neural networks to identify human activities and present a location-aware approach to improve the recognition accuracy. The location information is obtained from analyzing images captured by a wearable camera based on a pre-trained model of the deep convolutional neural networks. With the location information, some unlikely activities can be ruled out, leading to better recognition accuracy.

LSTM-RNNs combined with scene information for human activity recognition

One or more techniques and/or systems are provided for personal tracking of hygiene compliance of a user. For example, a mobile device may host a tracking component (e.g., a personal hygiene compliance app deployed on a cell phone or tablet). The tracking component may be configured to establish a communication connection, such as a Bluetooth communication connect, with a hygiene device (e.g., a soap dispenser). The tracking component may generate either a hygiene compliance metric or a non-hygiene compliance metric based upon whether a dispense event notification, indicating compliance by the user with a hygiene opportunity, is received from the hygiene device. In this way, the user may personally track hygiene compliance information based upon automated communication signals shared between the mobile device and the hygiene device.

Hygiene tracking compliance

An interactive monitoring system for arbitrarily complex hand-washing processes from simple hand¬ washing to full surgical scrub comprising user identification, multi-camera monitoring of hand-washing motions, various media for instructional or other feedback to the user, and a central computing system for storing hand-washing and instructional rules, instructional and other media, user-specific characteristics and rules and data on the status, quality and compliance of all hand-washing events monitored.

Virtual hand-washing coach

There are many applications using hand gesture as a nature control interface, such as humanmachine interaction and interactive entertainment. With the advancement of computer vision and machine learning, the development of vision-based hand gesture recognition systems has received more and more attention in recent years. However, much previous research was focused on software algorithms running on a PC-based platform, which makes it inapplicable to real-time applications. In this study, a hardware implemented gesture recognition system based on improved optical flow was proposed to accelerate the execution and achieve real-time operation. Experimental results showed that the proposed framework is feasible and can reach the speed of 30 frames per second.

A Novel FPGA-based Hand Gesture Recognition System

Most visual-based gesture control systems are bound to specific applications. They used predefined postures for users to control devices. Users need to learn and be familiar with those predefined postures to issue a command. This makes it difficult to transfer one gesture control interface into different applications. This study presented a generic framework for the design of a human machine interface based on visual-based hand gesture recognition techniques. Experimental results showed that the proposed approach is feasible for practical applications.

A generic framework for the design of visual-based gesture control interface

An intelligent hand washing monitoring system identifies if a user has actually washed every parts of his/her hands and informs the user with washed parts highlighted on a display unit. An image recognition technique is used to determine if the user uses a standard hand washing procedure to wash their hands and suggests to the user which part to wash with respect to the standard hand washing procedure in real time.

Yu-Hong Lin

Papers

A generic framework for the design of visual-based gesture control interface

Intelligent hand washing monitoring system

Hand gesture control system