Living in a computer era, the synergy between man and machine is a must, as the computers are integrated into our everyday life. The computers are surrounding us but their interfaces are far from being friendly. One possible approach to create a friendlier human-computer interface is to build an emotion-sensitive machine that should be able to recognize a human facial expression with a satisfactory classification rate and, eventually, to synthesize an artificial facial expression onto embodied conversational agents (ECAs), defined as friendly and intelligent user interfaces built to mimic human gestures, speech or facial expressions. Computer scientists working in computer interfaces (HCI) put up impressive efforts to create a fully automatic system capable to identifying and generating photo realistic human facial expressions through animation. This chapter aims at presenting current state-of-the-art techniques and approaches developed over time to deal with facial expression synthesis and animation. The topic’s importance will be further highlighted through modern applications including multimedia applications. The chapter ends up with discussions and open problems.

https://ict.usc.edu/pubs/Emotional%20Affordances%20within%20Interactive%20Experiences-%20A%20scientific%20approach%20to%20understanding%20artistic%20tools%20for%20stimulating%20emotions%20within%20Interactive%20Entertainment%20Experiences.pdf

Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives

Lighting design is an important element of game design, due to its role in directing attention, providing visibility, establishing important information concerning period and time of day, influenci ...

https://summit.sfu.ca/system/files/iritems1/13/SeifEl-Nasr_DynamicLighting.pdf

Dynamic Lighting for Tension in Games

L ighting design is an important topic of game development. There are many functions that lighting assumes in game environments, including directing attention, establishing good action visibility, evoking emotions, setting atmosphere, and providing depth. Current lighting design techniques rely on static manually designed lighting, where designers set up the positions, angles, and colors for each light in a level. Game environments are dynamic and unpredictable; physical and narrative scene content, including character locations, tension, and narrative goals, change unpredictably in real time due to user interaction. Thus, current static techniques often do not adequately adapt to serve desired aesthetic and communicative functions or perceptual effects. Recently, Doom 3 incorporated dynamic real-time lighting and demonstrated many advantages of using real-time dynamic lighting in games, including heightening the emotional engagement and enhancing the overall interactive experience. However, the technique is scripted and tightly coupled to game content. In this article, we present ELE (Expressive Lighting Engine), an intelligent lighting system that automatically sets and adjusts scene lighting in real time to achieve aesthetic and communicative functions, including evoking emotions, directing visual focus, and providing visibility and depth. ELE operates as a separate system that interacts with game/graphics engines through a standard interface. In this article, we will discuss ELE and its interface with Unreal Tournament 2003. We will also present results showing ELE in action. These results show: • Utility of real-time adaptive lighting in providing visual focus, setting atmosphere, evoking emotions, and establishing visibility during interaction in interactive environments • Acceleration in the development process due to the introduction of an automatic system for lighting that can be overridden by designers at a high level, thus eliminating the time-consuming process of setting individual light parameters for each level and scene. Journal-2005 2/23/05 10:25 AM Page 17

http://summit.sfu.ca/system/files/iritems1/596/SeifEl-Nasr_IntelligentLighting.pdf

Intelligent Lighting for Game Environments

Theatre directors, filmmakers, and animators have recognized and documented the importance of visual elements, including placements of characters, lighting effects, and camera shots, in conveying the narrative and engaging viewers. Even though, advances in computer graphics have facilitated the development of complex and intriguing synthetic worlds that can be visually manipulated in real-time, current interactive entertainment systems still rely on statically designed camera movements, lighting, and animations, which are labor intensive and result in context insensitive and visually unfulfilling interactive experiences. Very few research attempts have explored the utility of visually manipulating a scene to accommodate variations in the plot and dramatic situation imposed by interaction. Such studies focused mostly on camera movement and positioning with little implication on lighting and character movements. In this paper, I present an interactive narrative architecture that in addition to dynamically selecting narrative events that suit the situation, it dynamically reconfigures the visual design integrating aspects of several visual elements, including camera, lighting, and character. It uses rules, based on documented filmmaking, cinematography, and visual arts theories, to increase engagement and accommodate the evolving dramatic situation..

Interactive Narrative Architecture based on Filmmaking Theory

The interactive entertainment industry has become a multi-billion dollar industry with revenues overcoming those of the movie industry (ESA, 2009). Beyond the demand for high fidelity graphics or stylized imagery, participants in these environments have come to expect certain aesthetic and artistic qualities that engage them at a very deep emotional level. These qualities pertain to the visual aesthetic, dramatic structure, pacing, and sensory systems embedded within the experience. All these qualities are carefully crafted by the creator of the interactive experience to evoke affect. In this book chapter, we will attempt to discuss the design techniques developed by artists to craft such emotionally engaging experiences. In addition, we take a scientific approach whereby we discuss case studies of the use of these design techniques and experiments that attempt to validate their use in stimulating emotions.

Blain Brown

Papers

Motion Picture and Video Lighting