Audiovisual Speech Synthesis

TLDR: This paper presents the main approaches used to synthesize talking faces, and provides greater detail on a handful of these approaches, and an attempt is made to distinguish between facial synthesis itself and the manner in which facial movements are rendered on a computer screen.

Abstract: This paper presents the main approaches used to synthesize talking faces, and provides greater detail on a handful of these approaches. An attempt is made to distinguish between facial synthesis itself (i.e. the manner in which facial movements are rendered on a computer screen), and the way these movements may be controlled and predicted using phonetic input. The two main synthesis techniques (model-based vs. image-based) are contrasted and presented by a brief description of the most illustrative existing systems. The challenging issues—evaluation, data acquisition and modeling—that may drive future models are also discussed and illustrated by our current work at ICP.