Design of a Silent Speech Interface using Facial Gesture Recognition and Electromyography

TLDR: The SSI discussed in this paper was built using lip region segmentation from facial images via Chan-Vese algorithm, simple feature extraction from these images, matching features to words using a Multi-Class Support Vector Machine Classifier and a text -to-speech module producing synthesized speech implemented using MATLAB.

Abstract: A Silent Speech Interface (SSI) aims to substitute natural speech production using various electronic techniques to aid the vocally challenged or impaired. Such interfaces have been experimentally developed using several technologies such as imaging of the lip, tongue and throat regions; mapping of articulatory information, direct mapping of electroencephalogram details, Brain-Computer Interfaces and lip movement tracking mechanisms. Several unimodal as well as multi-modal systems are in the experimental stages. The SSI discussed in this paper was built using lip region segmentation from facial images via Chan-Vese algorithm, simple feature extraction from these images, matching features to words using a Multi-Class Support Vector Machine Classifier and a text -to-speech module producing synthesized speech implemented using MATLAB, with a recognition accuracy of 97.5% for four words – ‘cat’, ‘dog’, ‘eat’ and ‘mum’. A multimodal system with facial electromyographic input for validation purposes has been explored.