Artificially Intelligent Game Framework Based on Facial Expression Recognition

TLDR: In this article, an artificially intelligent game framework with smart features based on automatic facial expression recognition and adaptive game features was presented, where the gamer's emotions were recognized at run-time during gameplay using Deep Convolutional Neural Networks (CNN), and the game was adapted accordingly to the emotional condition.

Abstract: During gameplay, a player experiences emotional turmoil. In most of the cases, these emotions directly reflect the outcome of the game. Adapting game features based on players’ emotions necessitates a way to detect the current emotional state. Researchers in the area of “video game user research” has studied biometric data as a way to address the diverse characteristics of players, their individual preferences, gameplay expertise, and experiences. Identification of the player’s current state is fundamental for designing a game, which interacts with the player adaptively. In this paper, we present an artificially intelligent game framework with smart features based on automatic facial expression recognition and adaptive game features based on the gamer’s emotion. The gamer’s emotions are recognized at run-time during gameplay using Deep Convolutional Neural Networks (CNN), and the game is adapted accordingly to the emotional condition. Once identified, these features directly modify critical parameters of the underlying game engine to make the game more exciting and challenging.