Despite decades of research, the functional neuroanatomy of speech processing has been difficult to characterize. A major impediment to progress may have been the failure to consider task effects when mapping speech-related processing systems. We outline a dual-stream model of speech processing that remedies this situation. In this model, a ventral stream processes speech signals for comprehension, and a dorsal stream maps acoustic speech signals to frontal lobe articulatory networks. The model assumes that the ventral stream is largely bilaterally organized--although there are important computational differences between the left- and right-hemisphere systems--and that the dorsal stream is strongly left-hemisphere dominant.

The cortical organization of speech processing

Speech and language are considered uniquely human abilities: animals have communication systems, but they do not match human linguistic skills in terms of recursive structure and combinatorial power. Yet, in evolution, spoken language must have emerged from neural mechanisms at least partially available in animals. In this paper, we will demonstrate how our understanding of speech perception, one important facet of language, has profited from findings and theory in nonhuman primate studies. Chief among these are physiological and anatomical studies showing that primate auditory cortex, across species, shows patterns of hierarchical structure, topographic mapping and streams of functional processing. We will identify roles for different cortical areas in the perceptual processing of speech and review functional imaging work in humans that bears on our understanding of how the brain decodes and monitors speech. A new model connects structures in the temporal, frontal and parietal lobes linking speech perception and production.

/pdf/maps-and-streams-in-the-auditory-cortex-nonhuman-primates-4a7cmw3941.pdf

Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing.

https://www.cell.com/trends/cognitive-sciences/pdf/S1364-6613(00)01816-7.pdf

Structure and function of auditory cortex: music and speech

Neuronal oscillations are ubiquitous in the brain and may contribute to cognition in several ways: for example, by segregating information and organizing spike timing. Recent data show that delta, theta and gamma oscillations are specifically engaged by the multi-timescale, quasi-rhythmic properties of speech and can track its dynamics. We argue that they are foundational in speech and language processing, 'packaging' incoming information into units of the appropriate temporal granularity. Such stimulus-brain alignment arguably results from auditory and motor tuning throughout the evolution of speech and language and constitutes a natural model system allowing auditory research to make a unique contribution to the issue of how neural oscillatory activity affects human cognition.

/pdf/cortical-oscillations-and-speech-processing-emerging-o1mnh4nv0n.pdf

Cortical oscillations and speech processing: emerging computational principles and operations.

Language processing is a trait of human species. The knowledge about its neurobiological basis has been increased considerably over the past decades. Different brain regions in the left and right hemisphere have been identified to support particular language functions. Networks involving the temporal cortex and the inferior frontal cortex with a clear left lateralization were shown to support syntactic processes, whereas less lateralized temporo-frontal networks subserve semantic processes. These networks have been substantiated both by functional as well as by structural connectivity data. Electrophysiological measures indicate that within these networks syntactic processes of local structure building precede the assignment of grammatical and semantic relations in a sentence. Suprasegmental prosodic information overtly available in the acoustic language input is processed predominantly in a temporo-frontal network in the right hemisphere associated with a clear electrophysiological marker. Studies with patients suffering from lesions in the corpus callosum reveal that the posterior portion of this structure plays a crucial role in the interaction of syntactic and prosodic information during language processing.

https://www.ehu.eus/HEB/KEPA/Advanced_2012/2011_Friederici_The%20brain%20basis%20of%20language%20processing%20From%20structure%20to%20function.pdf

The Brain Basis of Language Processing: From Structure to Function

Nearly perfect speech recognition was observed under conditions of greatly reduced spectral information. Temporal envelopes of speech were extracted from broad frequency bands and were used to modulate noises of the same bandwidths. This manipulation preserved temporal envelope cues in each band but restricted the listener to severely degraded information on the distribution of spectral energy. The identification of consonants, vowels, and words in simple sentences improved markedly as the number of bands increased; high speech recognition performance was obtained with only three bands of modulated noise. Thus, the presentation of a dynamic temporal pattern in only a few broad spectral regions is sufficient for the recognition of speech.

/pdf/speech-recognition-with-primarily-temporal-cues-121lmvokkq.pdf

Speech recognition with primarily temporal cues.

As the most successful neural prosthesis, cochlear implants have provided partial hearing to more than 120000 persons worldwide; half of which being pediatric users who are able to develop nearly normal language. Biomedical engineers have played a central role in the design, integration and evaluation of the cochlear implant system, but the overall success is a result of collaborative work with physiologists, psychologists, physicians, educators, and entrepreneurs. This review presents broad yet in-depth academic and industrial perspectives on the underlying research and ongoing development of cochlear implants. The introduction accounts for major events and advances in cochlear implants, including dynamic interplays among engineers, scientists, physicians, and policy makers. The review takes a system approach to address critical issues in cochlear implant research and development. First, the cochlear implant system design and specifications are laid out. Second, the design goals, principles, and methods of the subsystem components are identified from the external speech processor and radio frequency transmission link to the internal receiver, stimulator and electrode arrays. Third, system integration and functional evaluation are presented with respect to safety, reliability, and challenges facing the present and future cochlear implant designers and users. Finally, issues beyond cochlear implants are discussed to address treatment options for the entire spectrum of hearing impairment as well as to use the cochlear implant as a model to design and evaluate other similar neural prostheses such as vestibular and retinal implants.

Cochlear Implants: System Design, Integration, and Evaluation

Amplitude modulation (AM) and frequency modulation (FM) are commonly used in communication, but their relative contributions to speech recognition have not been fully explored. To bridge this gap, we derived slowly varying AM and FM from speech sounds and conducted listening tests using stimuli with different modulations in normal-hearing and cochlear-implant subjects. We found that although AM from a limited number of spectral bands may be sufficient for speech recognition in quiet, FM significantly enhances speech recognition in noise, as well as speaker and tone recognition. Additional speech reception threshold measures revealed that FM is particularly critical for speech recognition with a competing voice and is independent of spectral resolution and similarity. These results suggest that AM and FM provide independent yet complementary contributions to support robust speech recognition under realistic listening situations. Encoding FM may improve auditory scene analysis, cochlear-implant, and audiocoding performance.

/pdf/speech-recognition-with-amplitude-and-frequency-modulations-5azjjy7jxw.pdf

Speech recognition with amplitude and frequency modulations

Speech recognition in noise and music perception is especially challenging for current cochlear implant users. The present study utilizes the residual acoustic hearing in the nonimplanted ear in five cochlear implant users to elucidate the role of temporal fine structure at low frequencies in auditory perception and to test the hypothesis that combined acoustic and electric hearing produces better performance than either mode alone. The first experiment measured speech recognition in the presence of competing noise. It was found that, although the residual low-frequency (<1000 Hz) acoustic hearing produced essentially no recognition for speech recognition in noise, it significantly enhanced performance when combined with the electric hearing. The second experiment measured melody recognition in the same group of subjects and found that, contrary to the speech recognition result, the low-frequency acoustic hearing produced significantly better performance than the electric hearing. It is hypothesized that listeners with combined acoustic and electric hearing might use the correlation between the salient pitch in low-frequency acoustic hearing and the weak pitch in the envelope to enhance segregation between signal and noise. The present study suggests the importance and urgency of accurately encoding the fine-structure cue in cochlear implants.

/pdf/speech-and-melody-recognition-in-binaurally-combined-52u2trd1wk.pdf

Speech and melody recognition in binaurally combined acoustic and electric hearing

More than 60,000 people worldwide use cochlear implants as a means to restore functional hearing. Although individual performance variability is still high, an average implant user can talk on the phone in a quiet environment. Cochlear-implant research has also matured as a field, as evidenced by the exponential growth in both the patient population and scientific publication. The present report examines current issues related to audiologic, clinical, engineering, anatomic, and physiologic aspects of cochlear implants, focusing on their psychophysical, speech, music, and cognitive performance. This report also forecasts clinical and research trends related to presurgical evaluation, fitting protocols, signal processing, and postsurgical rehabilitation in cochlear implants. Finally, a future landscape in amplification is presented that requires a unique, yet complementary, contribution from hearing aids, middle ear implants, and cochlear implants to achieve a total solution to the entire spectrum of hearing loss treatment and management.

/pdf/trends-in-cochlear-implants-4cbgmyerjg.pdf

Fan-Gang Zeng

Papers

Speech recognition with primarily temporal cues.

Cochlear Implants: System Design, Integration, and Evaluation

Speech recognition with amplitude and frequency modulations

Speech and melody recognition in binaurally combined acoustic and electric hearing

Trends in Cochlear Implants