Foreign language (FL) films with subtitles are becoming increasingly popular, and many European countries use subtitling as a cheaper alternative to dubbing. However, the extent to which people process subtitles under different subtitling conditions remains unclear. In this study, participants watched part of a film under standard (FL soundtrack and native language subtitles), reversed (native language soundtrack and FL subtitles), or intralingual (FL soundtrack and FL subtitles) subtitling conditions while their eye movements were recorded. The results revealed that participants read the subtitles irrespective of the subtitling condition. However, participants exhibited more regular reading of the subtitles when the film soundtrack was in an unknown FL. To investigate the incidental acquisition of FL vocabulary, participants also completed an unexpected auditory vocabulary test. Because the results showed no vocabulary acquisition, the need for more sensitive measures of vocabulary acquisition are discussed. Finally, the reading of the subtitles is discussed in relation to the saliency of subtitles and automatic reading behavior.

/pdf/processing-of-native-and-foreign-language-subtitles-in-films-4vkqwdgtjr.pdf

Processing of native and foreign language subtitles in films: an eye tracking study

Video captions, also known as same-language subtitles, benefit everyone who watches videos (children, adolescents, college students, and adults). More than 100 empirical studies document that captioning a video improves comprehension of, attention to, and memory for the video. Captions are particularly beneficial for persons watching videos in their non-native language, for children and adults learning to read, and for persons who are D/deaf or hard of hearing. However, despite U.S. laws, which require captioning in most workplace and educational contexts, many video audiences and video creators are naive about the legal mandate to caption, much less the empirical benefit of captions.

Video Captions Benefit Everyone

One of the most frequently recurring themes in captioning is whether captions should be edited or verbatim. The authors report on the results of an eye-tracking study of captioning for deaf and hard of hearing viewers reading different types of captions. By examining eye movement patterns when these viewers were watching clips with verbatim, standard, and edited captions, the authors tested whether the three different caption styles were read differently by the study participants ( N = 40): 9 deaf, 21 hard of hearing, and 10 hearing individuals. Interesting interaction effects for the proportion of dwell time and fixation count were observed. In terms of group differences, deaf participants differed from the other two groups only in the case of verbatim captions. The results are discussed with reference to classical reading studies, audiovisual translation, and a new concept of viewing speed.

Verbatim, Standard, or Edited?: Reading Patterns of Different Captioning Styles Among Deaf, Hard of Hearing, and Hearing Viewers

People watch subtitled audiovisual materials more than ever before. With the proliferation of subtitled content, we are also witnessing an increase in subtitle speeds. However, there is an ongoing controversy about what optimum subtitle speeds should be. This study looks into whether viewers can keep up with increasingly fast subtitles and whether the way people cope with subtitled content depends on their familiarity with subtitling and on their knowledge of the language of the film soundtrack. We tested 74 English, Polish and Spanish viewers watching films subtitled at different speeds (12, 16 and 20 characters per second). The films were either in Hungarian, a language unknown to the participants (Experiment 1), or in English (Experiment 2). We measured viewers' comprehension, self-reported cognitive load, scene and subtitle recognition, preferences and enjoyment. By analyzing people's eye gaze, we were able to discover that most viewers could read the subtitles as well as follow the images, coping well even with fast subtitle speeds. Slow subtitles triggered more re-reading, particularly in English clips, causing more frustration and less enjoyment. Faster subtitles with unreduced text were preferred in the case of English videos, and slower subtitles with text edited down in Hungarian videos. The results provide empirical grounds for revisiting current subtitling practices to enable more efficient processing of subtitled videos for viewers.

/pdf/viewers-can-keep-up-with-fast-subtitles-evidence-from-eye-q1912qhy33.pdf

Viewers can keep up with fast subtitles: Evidence from eye movements.

In this paper we examine the influence of text editing (edited vs. verbatim subtitles) and subtitle presentation rates (12 vs. 15 characters per second) on the comprehension and reading patterns of interlingual and intralingual subtitles among a group of 44 deaf, 33 hard of hearing and 60 hearing Polish adult subjects. The results of the eyetracking study show no benefit of editing down the text of subtitles, particularly in the case of intralingual subtitling and deaf viewers. Verbatim subtitles displayed with the higher presentation rate yielded slightly better comprehension results, were skipped less often, and resulted in more effective reading patterns. Deaf and hard of hearing participants had lower comprehension than hearing people; they also had a higher number of fixations per subtitle and were found to dwell on subtitles longer than the hearing.

/pdf/the-effects-of-text-editing-and-subtitle-presentation-rate-1dljzmjufe.pdf

The effects of text editing and subtitle presentation rate on the comprehension and reading patterns of interlingual and intralingual subtitles among deaf, hard of hearing and hearing viewers

Caption rate and text reduction are factors that appear to affect the comprehension of captions by people who are deaf or hard of hearing. These 2 factors are confounded in everyday captioning; rate (in words per minute) is slowed by text reduction. In this study, caption rate and text reduction were manipulated independently in 2 experiments to assess any differential effects and possible benefits for comprehension by deaf and hard-of-hearing adults. Volunteers for the study included adults with a range of reading levels, self-reported hearing status, and different communication and language preferences. Results indicate that caption rate (at 130, 180, 230 words per minute) and text reduction (at 84%, 92%, and 100% original text) have different effects for different adult users, depending on hearing status, age, and reading level. In particular, reading level emerges as a dominant factor: more proficient readers show better comprehension than poor readers and are better able to benefit from caption rate and, to some extent, text reduction modifications.

Parameters in television captioning for deaf and hard of hearing adults: effects of caption rate versus text reduction on comprehension

Despite the popularity of television captioning among deaf and hard-of-hearing Australians and its utility for other sections of the community (e.g., second language learners of English), there is little systematic research on relevant factors in captioning and patterns of caption use. This paper reports data from the Television Caption Users Survey (conducted October–December, 2000), the largest Australian survey to date of caption users' viewing habits and preferences. The results of this survey have not been previously reported beyond the submission of an industry report, but remain entirely pertinent to the current situation of caption usage in Australia and internationally. Results indicate that caption use is influenced strongly by functional hearing level and experience with captions, as well as by demographic factors. Possible underuse of captions by certain groups warrants further research both by survey and experimental approaches.

Alex Varley

Papers

Parameters in television captioning for deaf and hard of hearing adults: effects of caption rate versus text reduction on comprehension

Who uses television captions, when, and why?: analyses based on the Australian Television Caption Users Survey