Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268

Recent studies of eye movements in reading and other information processing tasks, such as music reading, typing, visual search, and scene perception, are reviewed. The major emphasis of the review is on reading as a specific example of cognitive processing. Basic topics discussed with respect to reading are (a) the characteristics of eye movements, (b) the perceptual span, (c) integration of information across saccades, (d) eye movement control, and (e) individual differences (including dyslexia). Similar topics are discussed with respect to the other tasks examined. The basic theme of the review is that eye movement data reflect moment-to-moment cognitive processes in the various tasks examined. Theoretical and practical considerations concerning the use of eye movement data are also discussed.

/pdf/eye-movements-in-reading-and-information-processing-20-years-3z5ay8hdwc.pdf

Eye movements in reading and information processing: 20 years of research.

1980 Preface * 1999 Preface * 1999 Acknowledgements * Introduction * 1 Circular Logic * 2 Phase Singularities (Screwy Results of Circular Logic) * 3 The Rules of the Ring * 4 Ring Populations * 5 Getting Off the Ring * 6 Attracting Cycles and Isochrons * 7 Measuring the Trajectories of a Circadian Clock * 8 Populations of Attractor Cycle Oscillators * 9 Excitable Kinetics and Excitable Media * 10 The Varieties of Phaseless Experience: In Which the Geometrical Orderliness of Rhythmic Organization Breaks Down in Diverse Ways * 11 The Firefly Machine 12 Energy Metabolism in Cells * 13 The Malonic Acid Reagent ('Sodium Geometrate') * 14 Electrical Rhythmicity and Excitability in Cell Membranes * 15 The Aggregation of Slime Mold Amoebae * 16 Numerical Organizing Centers * 17 Electrical Singular Filaments in the Heart Wall * 18 Pattern Formation in the Fungi * 19 Circadian Rhythms in General * 20 The Circadian Clocks of Insect Eclosion * 21 The Flower of Kalanchoe * 22 The Cell Mitotic Cycle * 23 The Female Cycle * References * Index of Names * Index of Subjects

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/596B270197FE27DE5FABB598B6210622/S0025557200150892a.pdf/div-class-title-span-class-italic-the-geometry-of-biological-time-span-by-a-t-winfree-pp-544-dm68-corrected-second-printing-1990-isbn-3-540-52528-9-springer-div.pdf

The geometry of biological time , by A. T. Winfree. Pp 544. DM68. Corrected Second Printing 1990. ISBN 3-540-52528-9 (Springer)

The generation of words in speech involves a number of processing stages. There is, first, a stage of conceptual preparation; this is followed by stages of lexical selection, phonological encoding, phonetic encoding and articulation. In addition, the speaker monitors the output and, if necessary, self-corrects. Major parts of the theory have been computer modelled. The paper concentrates on experimental reaction time evidence in support of the theory.

/pdf/a-theory-of-lexical-access-in-speech-production-56yj14t9eb.pdf

A theory of lexical access in speech production

人教版高中英语新课程教材中,语言运用（Using Language）是每个单元必不可少的部分,提供了围绕单元中心话题的听、说、读、写的综合性练习,是单元中心话题的延续和升华.如何设计Using Language部分的教学,使自己的教学模式既不落俗套,又能真正体现新课程标准所倡导的教学理念,正是广大一线英语教师一直努力探索的问题.

打磨Using Language,倡导新理念

Advances in speech driven animation techniques allow the creation of convincing animations for virtual characters solely from audio data. Many existing approaches focus on facial and lip motion and they often do not provide realistic animation of the inner mouth. This paper addresses the problem of speech-driven inner mouth animation. Obtaining performance capture data of the tongue and jaw from video alone is difficult because the inner mouth is only partially observable during speech. In this work, we introduce a large-scale speech and mocap dataset that focuses on capturing tongue, jaw, and lip motion. This dataset enables research using data-driven techniques to generate realistic inner mouth animation from speech. We then propose a deep-learning based method for accurate and generalizable speech to tongue and jaw animation, and evaluate several encoder-decoder network architectures and audio feature encoders. We find that recent self-supervised deep learning based audio feature encoders are robust, generalize well to unseen speakers and content, and work best for our task. To demonstrate the practical application of our approach, we show animations on high-quality parametric 3D face models driven by the landmarks generated from our speech-to-tongue animation method.

Speech Driven Tongue Animation

This paper will summarize our work at Queen's University and ATR Laboratories on cross-modal speech perception and production. Our approach has been to study these two sides of speech together and to use the multi-modal speech production data to parameterize and control audiovisual animation systems. Two approaches to production-based facial animation have been pursued — one statistical and the other physical. In both cases, realistic talking head animations are generated from continuous input of production data. The statistical animation method of AV synthesis extends our multi-linear techniques developed for the analysis of orofacial motion and speech acoustics to include the correlation between measured 3D positions on the face and deformation coefficients of the facial surface. In the physical approach, the dynamic form of the animation is determined by the biophysical characteristics of the animated object. The physical model consists of multiple structural layers: model skull and jaw surfaces, an orofacial muscle layer, and a three-layer polygon model of the soft tissue. In a series of studies using these animation approaches we have examined the conditions under which speech perception in noise is enhanced by simultaneous visual presentation. Our data show a distinction between visual prosody and segmental perception as well as demonstrating that our animated stimuli produce natural increases in speech intelligibility. One of the classic problems of speech research is how to relate speech perception to speech production. Al Liberman called this problem, the problem of parity. "In all communication, sender and receiver must be bound by a common understanding about what counts; what counts for the sender must count for the receiver, else communication does not occur. Moreover the processes of production and perception must somehow be linked; their representation must, at some point, be the same.”(Liberman, 1996). In recent years our laboratories at ATR and Queen's University have been pursuing this question by studying audiovisual (AV) speech perception and speech production. This work has two main aspects. First, we have documented the relations or “mappings” between different speech signal types so that one signal type can be estimated from another (e.g., electromyographic (EMG), kinematic, acoustic). Secondly, we have used these mappings as the basis for animation that can be used in perceptual research. Two AV animation systems have been developed. One is a statistical model that uses face motion to estimate the deformations of a speaker's face through time. Using the mappings specified by the speech production model, the face motion input can be estimated from muscle EMG activity, vocal tract motion, or the speech acoustics (Kuratate, Munhall, Rubin, Vatikiotis-Bateson, & Yehia, 1999). The second system is a physical model whose dynamic parameters determine the interaction of skeletal, muscular, and tissue structures. This system generates muscle forces and subsequent facial deformations using muscle EMG signals as input (Lucero & Munhall, 1999). Both systems produce realistic motions and are controlled by production-based parameters whose contribution to perception can be monitored. Our goal has been to understand which production parameters the perceptual system is sensitive to and to explore audiovisual perception using natural stimuli that are well specified. One of the major challenges facing AV speech research is the need to explicitly describe the visual stimuli used in AV perception experiments. Individual speakers differ in the extent and clarity of the phonetic ! " ISCA Archive

Studies of audiovisual speech perception using production-based animation.

In natural conversation, visual and auditory information about speech not only provide linguistic information but also provide information about the identity and the emotional state of the speaker. Thus, listeners must process a wide range of information in parallel to understand the full meaning in a message. In this series of studies, we examined how different types of visual information conveyed by a speaker’s face are processed by measuring the gaze patterns exhibited by subjects watching audiovisual recordings of spoken sentences. In three experiments, subjects were asked to judge the emotion and the identity of the speaker, and to report the words that they heard under different auditory conditions. As in previous studies, eye and mouth regions dominated the distribution of the gaze fixations. It was hypothesized that the eyes would attract more fixations for more social judgment tasks, rather than tasks which rely more on verbal comprehension. Our results support this hypothesis. In addition, the location of gaze on the face did not influence the accuracy of the perception of speech in noise.

The influence of task on gaze during audiovisual speech perception

When engaging in conversation, we efficiently go back and forth with our partner, organizing our contributions in reciprocal turn-taking behavior. Using multiple auditory and visual cues, we make online decisions about when it is the appropriate time to take our turn. In two experiments, we demonstrated, for the first time, that auditory and visual information serve complementary roles when making such turn-taking decisions. We presented clips of single utterances spoken by individuals engaged in conversations in audiovisual, auditory-only or visual-only modalities. These utterances occurred either right before a turn exchange (i.e., ‘Turn-Ends’) or right before the next sentence spoken by the same talker (i.e., ‘Turn-Continuations’). In Experiment 1, participants discriminated between Turn-Ends and Turn-Continuations in order to synchronize a button-press response to the moment the talker would stop speaking. We showed that participants were best at discriminating between Turn-Ends and Turn-Continuations in the audiovisual condition. However, in terms of response synchronization, participants were equally precise at timing their responses to a Turn-End in the audiovisual and auditory-only conditions, showing no advantage of visual information. In Experiment 2, we used a gating paradigm, where increasing segments of Turns-Ends and Turn-Continuations were presented, and participants predicted if a turn exchange would occur at the end of the sentence. We found an audiovisual advantage in detecting an upcoming turn early in the perception of a turn exchange. Together, these results suggest that visual information functions as an early signal indicating an upcoming turn exchange while auditory information is used to precisely time a response to the turn end.

/pdf/knowing-when-to-respond-the-role-of-visual-information-in-2aoj88sn3g.pdf

Knowing when to respond: the role of visual information in conversational turn exchanges.

Previous studies have demonstrated a wide range in individuals’ compensations in response to real-time alterations of the auditory feedback of both pitch and formant frequencies. One potential source of this variability may be individual differences in the relative weighting of auditory and somatosensory feedback. The present study examined this variability by comparing individuals’ compensations during two perturbation conditions: a pitch shift (+200 cents) and a formant shift (F1 +200 Hz, F2 250 Hz). While no significant correlation was found between the two perturbation conditions, a modest correlation between compensations in pitch and formant frequency was observed within the pitch perturbation condition.

Kevin G. Munhall

Papers

Speech Driven Tongue Animation

Studies of audiovisual speech perception using production-based animation.

The influence of task on gaze during audiovisual speech perception

Knowing when to respond: the role of visual information in conversational turn exchanges.

A Preliminary Study of Individual Responses to Real-Time Pitch and Formant Perturbations