Emotion and Adaptation

Please provide a short abstract of 100 to 250 words. The abstract should not contain any undefined abbreviations or unspecified references. Work planned but not completed should not appear in the abstract.

User Modeling and User-Adapted Interaction

Some aspects of this disclosure involve generation of crowd-based results based on measurements of affective response of users. In some embodiments described herein, sensors are used to take measurements of affective response of at least ten users who were at a certain location. The measurements may include various values indicative of physiological signals and/or behavioral cues of the at least ten users. Some examples of locations in this disclosure include vacation destinations, businesses, establishments that provide entertainment, certain geographical regions, and virtual environments. User interfaces are configured to receive data describing a location score computed based on the measurements of the at least ten users, which represents the affective response of the at least ten users to being at the certain location. The user interfaces may be used to report the location score (e.g., to a user who may be interested in visiting the certain location).

Crowd-Based Scores for Locations from Measurements of Affective Response

Three-dimensional (3D) data acquisition and real-time processing is a critical issue in an artificial vision system. The developing time-of-flight (TOF) camera as a real-time vision sensor for obtaining depth images has now received wide attention, due to its great potential in many areas, such as 3D perception, computer vision, robot navigation, human–machine interaction, augmented reality, and so on. This paper survey advances in TOF imaging technology mainly from the last decade. We focus only on recent progress of overcoming limitations such as systematic errors, object boundary ambiguity, multipath error, phase wrapping, and motion blur, and address the theoretical principles and future research trends as well.

Recent Advances in 3D Data Acquisition and Processing by Time-of-Flight Camera

We present a calibration method of a time-of-flight (ToF) sensor and a color camera pair to align the 3D measurements with the color image correctly. We have designed a 2.5D pattern board with irregularly placed holes to be accurately detected from low resolution depth images of a ToF camera as well as from high resolution color images. In order to improve the accuracy of the 3D measurements of a ToF camera, we propose to perform ray correction and range bias correction. We reset the transformation of the ToF sensor which transforms the radial distance into the scene depth in Cartesian coordinate through ray correction. Then we capture a planar scene from different depths to correct the distance error that is shown to be dependent not only on the distance but also on the pixel location. The range error profiles along the calibrated distance are classified according to their wiggling shapes and each cluster of profiles with similar shape are separately estimated using a B-spline function. The standard deviation of the remaining random noise is recorded as an uncertainty information of distance measurements. We show the performance of our calibration method quantitatively and qualitatively on various datasets, and validate the impact of our method by demonstrating an RGB-D shape refinement application.

Time-of-Flight Sensor Calibration for a Color and Depth Camera Pair

Detecting emotions of a crowd to control the situation is an area of emerging interest. The purpose of this paper is to present a novel idea to detect the emotions of the crowd. Emotions are defined as evolving quantities arising from the reaction to contextual situations in a set of dynamic pattern of events. These events depend on internal and external interaction states in an already mapped space. The emotions of multiple people constituting a crowd in any surveillance environment are estimated by their social and collective behaviors using sensor signals e.g., a camera, which captures and tracks their motion. The feature space is constructed based on local features to model the contextual situations and the different interactions corresponding to different emergent behaviors are modeled using bio-inspired dynamic model. The changes in emotions correspond to behavioral changes which are produced to regulate behaviors under different encountered situations. Proposed algorithm involves the probabilistic signal processing modelling techniques for analysis of different types of collective behaviors based on interactions among people and classification models to estimate emotions as positive or negative. The evaluations are performed on simulated data show the proposed algorithm effectively recognizes the emotions of the crowd under specific situations.

/pdf/crowd-emotion-detection-using-dynamic-probabilistic-models-45oe9nfsfp.pdf

Crowd emotion detection using dynamic probabilistic models

3D scene rendering requires depth maps and color information to produce high quality 3D results. Unfortunately, depth maps captured with the Time-of-flight (TOF) cameras have limited resolution and poor image quality, being severely influenced by the random and systematic noise, which makes them inapposite for generating high quality 3D images. In this paper, we have further analyzed a framework for upsampling the resolution of depth maps that jointly uses Gaussians of spatial and depth differences of low resolution depth map's pixels along with Gaussian of color intensity difference from high resolution 2D color image of the same scene. The variance of the Gaussian functions controls the amount of smoothing in uni-planner area and sharpness at boundaries. Using bigger variance smooths uni-planner area but blurs edges and vice versa. We have devised a method to adaptively calculate and use variance to get smoother surface and sharper edges of upsampled depth map with minimized noise.

Adaptive bilateral filtering for noise removal in depth upsampling

Hand gestures can be used for natural and intuitive human-computer interaction. Our new method combines existing techniques of skin color based ROI segmentation and Viola-Jones Haar-like feature based object detection, to optimize hand gesture recognition for mouse operation. A mouse operation has two parts, movement of cursor and clicking using the right or left mouse button. In this paper, color is used as a robust feature to first define a Region of Interest (ROI). Then within this ROI, hand postures are detected by using Haar-like features and AdaBoost learning algorithm. The AdaBoost learning algorithm significantly speeds up the performance and constructs an accurate cascaded classifier by combining a sequence of weak classifiers.

New hand gesture recognition method for mouse operations

Perceiving crowd emotions and understand the situation is vital to control the situations in surveillance applications. This paper introduces the evolution of methods for crowd emotion perception based on bio-inspired probabilistic models. The emotions have been perceived both in an offline and online manner from the crowd. We focus on the perception of emotion from crowd behavior and dynamics. The paper explains few probabilistic algorithms and compares these for detection of emotion of crowds and proposes a probabilistic modelling approach which is trained on data to perceive the emotions of the crowd in an area under surveillance. Emotions are defined as evolving dynamic patterns arising due to interaction of people in an environment with their relationships to the past interaction patterns. Camera sensors are used to track the motion of the individuals within a crowd scenario under observation. The data mining techniques are used to distinguish between different behaviors and events into positive and negative emotions. The results have been evaluated using simulated data from a proposed office environment.

/pdf/perception-of-emotions-from-crowd-dynamics-170lu5ntg0.pdf

Perception of emotions from crowd dynamics

Detection of emotions of a crowd is a new research area, which has never, to our knowledge, been accounted for research in previous literature. A bio-inspired model for representation of emotional patterns in crowds has been demonstrated. Emotions have been defined as evolving patterns as part of a dynamic pattern of events. This model has been developed to detect emotions of a crowd based on the knowledge from a learned context, psychology and experience of people in crowd management. The emotions of multiple people making a crowd in any surveillance environment are estimated by sensors signals such as a camera and are being tracked and their behavior is modeled using bio-inspired dynamic model. The behavior changes correspond to changes in emotions. The proposed algorithm involves the probabilistic signal processing modelling techniques for analysis of different types of behavior, interaction detection and estimation of emotions. The emotions are recognized by the expectation of temporal occurrences of causal events modeled by Gaussian mixture model. The model has been evaluated using the simulated behavioral model of a crowd.

Mirza Waqar Baig

Papers

Crowd emotion detection using dynamic probabilistic models

Adaptive bilateral filtering for noise removal in depth upsampling

New hand gesture recognition method for mouse operations

Perception of emotions from crowd dynamics

Bio-inspired probabilistic model for crowd emotion detection