This study investigated whether speech produced in spontaneous interactions when addressing a talker experiencing actual challenging conditions differs in acoustic-phonetic characteristics from speech produced (a) with communicative intent under more ideal conditions and (b) without communicative intent under imaginary challenging conditions (read, clear speech). It also investigated whether acoustic-phonetic modifications made to counteract the effects of a challenging listening condition are tailored to the condition under which communication occurs. Forty talkers were recorded in pairs while engaged in “spot the difference” picture tasks in good and challenging conditions. In the challenging conditions, one talker heard the other (1) via a three-channel noise vocoder (VOC); (2) with simultaneous babble noise (BABBLE). Read, clear speech showed more extreme changes in median F0, F0 range, and speaking rate than speech produced to counter the effects of a challenging listening condition. In the VOC condition, where F0 and intensity enhancements are unlikely to aid intelligibility, talkers did not change their F0 median and range; mean energy and vowel F1 increased less than in the BABBLE condition. This suggests that speech production is listener-focused, and that talkers modulate their speech according to their interlocutors’ needs, even when not directly experiencing the challenging listening condition.

/pdf/acoustic-phonetic-characteristics-of-speech-produced-with-2ccx09nify.pdf

Acoustic-phonetic characteristics of speech produced with communicative intent to counter adverse listening conditions.

People with cognitive impairments have problems organizing their daily life autonomously. A virtual agent as daily calendar assistant could provide valuable support, but this requires that these special user groups accept such a system and can interact with it successfully. In this paper we present studies to elucidate these questions for elderly users as well as cognitively impaired users. Results from interviews and focus groups show that acceptance can be increased by way of a participatory design method. Actual interaction studies with a prototype demonstrate the feasibility of spoken-language interaction and reveal strategies to mitigate understanding problems.

Virtual Agents as Daily Assistants for Elderly or Cognitively Impaired People

Individuals who rely on augmentative and alternative communication (AAC) devices to support their communication often have physical movement challenges that require alternative methods of access. Technology that supports access, particularly for those with the most severe movement deficits, have expanded substantially over the years. The purposes of this article are to review the state of the science of access technologies that interface with augmentative and alternative communication devices and to propose a future research and development agenda that will enhance access options for people with limited movement capability due to developmental and acquired conditions.

Access Interface Strategies

One requirement for researching and building spoken language systems is the availability of speech data that have been labeled and time-aligned at the phonetic level. Although manual phonetic alignment is considered more accurate than automatic methods, it is too time consuming to be commonly used for aligning large corpora. One reason for the greater accuracy of human labeling is that humans are better able to locate distinct events in the speech signal that correspond to specific phonetic characteristics. The development of the proposed method was motivated by the belief that if an automatic alignment method were to use such acoustic-phonetic information, its accuracy would become closer to that of human performance. Our hypothesis is that the integration of acoustic-phonetic information into a state-of-the-art automatic phonetic alignment system will significantly improve its accuracy and robustness. 
In developing an alignment system that uses acoustic-phonetic information, we use a measure of intensity discrimination in detecting voicing, glottalization, and burst-related impulses. We propose and implement a method of voicing determination that has average accuracy of 97.25% (which is an average 58% reduction in error over a baseline system), a fundamental-frequency extraction method with average absolute error of 3.12 Hz (representing a 45% reduction in error), and a method for detecting burst-related impulses with accuracy of 86.8% on the TIMIT corpus (which is a 45% reduction in error compared to reported results). In addition to these features, we propose a means of using acoustics-dependent transition information in the HMM framework. One aspect of successful implementation of this method is the use of distinctive phonetic features. 
To evaluate the proposed and baseline phonetic alignment systems, we measure agreement with manual alignments and robustness. On the TIMIT corpus, the proposed method has 92.57% agreement within 20 msec. The average agreement of the proposed method represents a 28% reduction in error over our state-of-the-art baseline system. In measuring robustness, the proposed method has 14% less standard deviation when evaluated on 12 versions of the TIMIT corpus.

Automatic time alignment of phonemes using acoustic-phonetic information

Significant progress has been made in the application of natural language processing (NLP) to augmentative and alternative communication (AAC), particularly in the areas of interface design and word prediction This article will survey the current state-of-the-science of NLP in AAC and discuss its future applications for the development of next generation of AAC technology

The Application of Natural Language Processing to Augmentative and Alternative Communication.

Speakers naturally adopt a special “clear” (CLR) speaking style in order to be better understood by listeners who are moderately impaired in their ability to understand speech due to a hearing impairment, the presence of background noise, or both. In contrast, speech intended for nonimpaired listeners in quiet environments is referred to as “conversational” (CNV). Studies have shown that the intelligibility of CLR speech is usually higher than that of CNV speech in adverse circumstances. It is not known which individual acoustic features or combinations of features cause the higher intelligibility of CLR speech. The objective of this study is to determine the contribution of some acoustic features to intelligibility for a single speaker. The proposed method creates “hybrid” (HYB) speech stimuli that selectively combine acoustic features of one sentence spoken in the CNV and CLR styles. The intelligibility of these stimuli is then measured in perceptual tests, using 96 phonetically balanced sentences. Resu...

Hybridizing conversational and clear speech to determine the degree of contribution of acoustic features to intelligibility

This study described preliminary work with the Supplemented Speech Recognition (SSR) system for speakers with dysarthria. SSR incorporated automatic speech recognition optimized for dysarthric speech, alphabet supplementation, and word prediction. Participants included seven individuals with a range of dysarthria severity. Keystroke savings using SSR averaged 68.2% for typical sentences and 67.5% for atypical phrases. This was significantly different to using word prediction alone. The SSR correctly identified an average of 80.7% of target stimulus words for typical sentences and 82.8% for atypical phrases. Statistical significance could not be claimed for the relations between sentence intelligibility and keystroke savings or sentence intelligibility and system performance. The results suggest that individuals with dysarthria using SSR could achieve comparable keystroke savings regardless of speech severity.

Evaluation of a speech recognition prototype for speakers with moderate and severe dysarthria: a preliminary report.

“Clear” (CLR) speech is a speaking style that speakers adopt to be understood correctly in a difficult communication environment. Studies have shown that CLR speech, as opposed to “conversational” (CNV) speech, has significantly higher intelligibility in various conditions. While many differences in acoustic features have been identified, it is not known which individual feature or combinations of features cause the higher intelligibility of CLR speech. The objectives of the current study are to examine whether it is possible to improve speech intelligibility by approximating CLR speech features and to determine which acoustic features contribute to intelligibility. Our approach creates speech samples that combine acoustic features of CNV and CLR speech, using a hybridization algorithm. Results with normalhearing listeners showed significant sentence-level intelligibility improvements of 11–23% over CNV speech when replacing certain acoustic features with those from CLR speech.

Hybridizing conversational and clear speech

We present a real-coded genetic algorithm that efficiently estimates parameters of a formant trajectory model. The genetic algorithm uses roulette-wheel selection and elitism to minimize the root mean square error between the observed formant trajectory and the model trajectory. Parameters, including vowel and consonant target values and coarticulation parameters, are estimated for a corpus of English clear and conversational CVC words. Results show consistent consonant formant targets, even when those consonants do not themselves have formant structure. We also present findings of a relationship between a coarticulation parameter and the consonant identity.

Using a genetic algorithm to estimate parameters of a coarticulation model

It is generally acknowledged that time-alignment of phonemes is of better quality when obtained from manual segmentation as compared to automatic segmentation. However, empirical evidence for this belief is sparse. This paper describes a controlled study of two recognizers created using manually-aligned and automatically-aligned training data. Both recognizers were trained on the digits task using telephone- channel continuous speech. The manual alignments were generated by expert labelers, while the automatic alignments were obtained from our best general-purpose forced alignment system. The two recognizers were trained and evaluated using the same set of data and parameters whenever possible. The results of the recognizer trained on manually- aligned data were 97.54% word accuracy and 90.18% sentence accuracy. The results for the recognizer trained on automatically-aligned data were 97.24% word accuracy and 88.80% sentence accuracy. This represents an 11% reduction in error at the word level and a 12% reduction in error at the sentence level. The sentence-level results are statistically significant using McNemar''s test, with p=0.002.

John Paul Hosom

Papers

Hybridizing conversational and clear speech to determine the degree of contribution of acoustic features to intelligibility

Evaluation of a speech recognition prototype for speakers with moderate and severe dysarthria: a preliminary report.

Hybridizing conversational and clear speech

Using a genetic algorithm to estimate parameters of a coarticulation model

A Comparison of Speech Recognizers Created Using Manually-Aligned andAutomatically-Aligned Training Data