https://apps.dtic.mil/sti/pdfs/ADA157550.pdf

The TRACE model of speech perception.

Infants learn language with remarkable speed, but how they do it remains a mystery. New data show that infants use computational strategies to detect the statistical and prosodic patterns in language input, and that this leads to the discovery of phonemes and words. Social interaction with another human being affects speech learning in a way that resembles communicative learning in songbirds. The brain's commitment to the statistical and prosodic patterns that are experienced early in life might help to explain the long-standing puzzle of why infants are better language learners than adults. Successful learning by infants, as well as constraints on that learning, are changing theories of language acquisition.

/pdf/early-language-acquisition-cracking-the-speech-code-bf9adht4u8.pdf

Early language acquisition: cracking the speech code

In this article the author proposes an episodic theory of spoken word representation, perception, and production. By most theories, idiosyncratic aspects of speech (voice details, ambient noise, etc.) are considered noise and are filtered in perception. However, episodic theories suggest that perceptual details are stored in memory and are integral to later perception. In this research the author tested an episodic model (MINERVA 2; D. L. Hintzman, 1986) against speech production data from a word-shadowing task. The model predicted the shadowing-response-time patterns, and it correctly predicted a tendency for shadowers to spontaneously imitate the acoustic patterns of words and nonwords. It also correctly predicted imitation strength as a function of "abstract" stimulus properties, such as word frequency. Taken together, the data and theory suggest that detailed episodes constitute the basic substrate of the mental lexicon. Early in the 20th century, Semon (1909/1923) described a memory theory that anticipated many aspects of contemporary theories (Schacter, Eich, & Tulving, 1978). In modem parlance, this was an episodic (or exemplar) theory, which assumes that every experience, such as perceiving a spoken word, leaves a unique memory trace. On presentation of a new word, all stored traces are activated, each according to its similarity to the stimulus. The most activated traces connect the new word to stored knowledge, the essence of recognition. The multiple-trace assumption allowed Semon's theory to explain the apparent permanence of specific memories; the challenge was also to create

/pdf/echoes-of-echoes-an-episodic-theory-of-lexical-access-4u3ubhcqbh.pdf

Echoes of echoes? An episodic theory of lexical access.

Infant sensitivity to distributional information can affect phonetic discrimination.

In many everyday situations, our senses are bombarded by many different unisensory signals at any given time. To gain the most veridical, and least variable, estimate of environmental stimuli/properties, we need to combine the individual noisy unisensory perceptual estimates that refer to the same object, while keeping those estimates belonging to different objects or events separate. How, though, does the brain “know” which stimuli to combine? Traditionally, researchers interested in the crossmodal binding problem have focused on the roles that spatial and temporal factors play in modulating multisensory integration. However, crossmodal correspondences between various unisensory features (such as between auditory pitch and visual size) may provide yet another important means of constraining the crossmodal binding problem. A large body of research now shows that people exhibit consistent crossmodal correspondences between many stimulus features in different sensory modalities. For example, people consistently match high-pitched sounds with small, bright objects that are located high up in space. The literature reviewed here supports the view that crossmodal correspondences need to be considered alongside semantic and spatiotemporal congruency, among the key constraints that help our brains solve the crossmodal binding problem.

/pdf/crossmodal-correspondences-a-tutorial-review-20vme8gbi3.pdf

Crossmodal correspondences: A tutorial review

Studies of the McGurk effect have shown that when discrepant phonetic information is delivered to the auditory and visual modalities, the information is combined into a new percept not originally presented to either modality. In typical experiments, the auditory and visual speech signals are generated by the same talker. The present experiment examined whether a discrepancy in the gender of the talker between the auditory and visual signals would influence the magnitude of the McGurk effect. A male talker’s voice was dubbed onto a videotape containing a female talker’s face, and vice versa. The gender-incongruent videotapes were compared with gender-congruent videotapes, in which a male talker’s voice was dubbed onto a male face and a female talker’s voice was dubbed onto a female face. Even though there was a clear incompatibility in talker characteristics between the auditory and visual signals on the incongruent videotapes, the resulting magnitude of the McGurk effectwas not significantly different for the incongruent as opposed to the congruent videotapes. The results indicate that the mechanism for integrating speech information from the auditory and the visual modalities is not disrupted by a gender incompatibility even when it is perceptually apparent. The findings are compatible with the theoretical notion that information about voice characteristics of the talker is extracted and used to normalize the speech signal at an early stage of phonetic processing, prior to the integration of the auditory and the visual information.

/pdf/integrating-speech-information-across-talkers-gender-and-1lkxp00uln.pdf

Integrating speech information across talkers, gender, and sensory modality: female faces and male voices in the McGurk effect.

This study investigated the perceptual adjustments that occur when listeners recognize highly compressed speech In Experiment 1, adjustment was examined as a function of the amount of exposure to compressed speech by use of 2 different speakers and compression rates The results demonstrated that adjustment takes place over a number of sentences, depending on the compression rate Lower compression rates required less experience before full adjustment occurred In Experiment 2, the impact of an abrupt change in talker characteristics was investigated; in Experiment 3, the impact of an abrupt change in compression rate was studied The results of these 2 experiments indicated that sudden changes in talker characteristics or compression rate had little impact on the adjustment process The findings are discussed with respect to the level of speech processing at which such adjustment might occur

Perceptual adjustment to highly compressed speech: effects of talker and rate changes.

When listeners process segmentally relevant properties of the speech signal they do so in a rate-dependent manner. This is seen as a shift in the perceptual category boundary; as rate declines and overall syllable duration increases, the category boundary moves toward a longer value of the acoustic property in question. Focusing on the /b/-/p/ distinction specified by voice onset time (VOT), we investigated whether the acoustic modifications that occur with an alteration in speaking rate accord with this pattern of perceptual boundary shift. Two main findings emerged. First, as speaking rate became slower and overall syllable duration became longer, the VOT value of the consonant, especially that of the voiceless /p/, also became longer. Second, and most important, the VOT value that optimally separated the /b/ and /p/ VOT distributions also changed with rate, increasing with increasing syllable duration. However, the magnitude of the boundary shift obtained for these production data was greater than that typically found in perceptual experiments. This suggests the existence of constraints on the extent to which the perceptual system can accommodate for alterations due to rate of speech.

Speaking Rate and Segments: A Look at the Relation between Speech Production and Speech Perception for the Voicing Contrast

It is well established that listeners process segmentally relevant properties of the speech signal in relation to the rate at which the speech was produced. We investigated whether the critical rate information for this effect is limited to the auditory modality or, alternatively, whether visual rate information provided by the talker’s face also plays a role. Audio-visual syllables were created by pairing tokens from a moderate-rate, auditory /bi/-/pi/ series with visual tokens of /bi/ or /pi/ produced at a faster or slower rate of speech; these visual tokens provided information about speaking rate, but could not themselves be identified correctly as /bi/ or /pi/. Each audiovisual pairing produced the phenomenal experience of a single, unified syllable, /bi/ or /pi/, spoken at a single rate of speech. The change in visual rate information across the syllables influenced the judged rate of the audio-visual syllables and, more importantly, affected their identification as /bi/ or /pi/. These results indicate that visual information about speaking rate is relevant to the perception of voicing and, more generally, suggest that the mechanisms underlying rate-dependent speech processing have a bimodal (or amodal) component.

/pdf/on-the-role-of-visual-rate-information-in-phonetic-19afxbbwma.pdf

On the role of visual rate information in phonetic perception

Visual information provided by a talker's mouth movements can influence the perception of certain speech features. Thus, the "McGurk effect" shows that when the syllable (bi) is presented audibly, in synchrony with the syllable (gi), as it is presented visually, a person perceives the talker as saying (di). Moreover, studies have shown that interactions occur between place and voicing features in phonetic perception, when information is presented audibly. In our first experiment, we asked whether feature interactions occur when place information is specificed by a combination of auditory and visual information. Members of an auditory continuum ranging from (ibi) to (ipi) were paired with a video display of a talker saying (igi). The auditory tokens were heard as ranging from (ibi) to (ipi), but the auditory-visual tokens were perceived as ranging from (idi) to (iti). The results demonstrated that the voicing boundary for the auditory-visual tokens was located at a significantly longer VOT value than the voicing boundary for the auditory continuum presented without the visual information. These results demonstrate that place-voice interactions are not limited to situations in which place information is specified audibly.(ABSTRACT TRUNCATED AT 250 WORDS)

/pdf/the-role-of-visual-information-in-the-processing-of-place-2debl2otwo.pdf

Kerry P. Green

Papers

Integrating speech information across talkers, gender, and sensory modality: female faces and male voices in the McGurk effect.

Perceptual adjustment to highly compressed speech: effects of talker and rate changes.

Speaking Rate and Segments: A Look at the Relation between Speech Production and Speech Perception for the Voicing Contrast

On the role of visual rate information in phonetic perception

The role of visual information in the processing of place and manner features in speech perception