scispace - formally typeset
Search or ask a question
Proceedings Article

Palabras. Crowdsourcing transcriptions of L2 speech

TL;DR: A web application for crowdsourcing transcriptions of Dutch words spoken by Spanish L2 learners and the design of the application and the influence of metadata and various forms of feedback is discussed.
Abstract: We developed a web application for crowdsourcing transcriptions of Dutch words spoken by Spanish L2 learners. In this paper we discuss the design of the application and the influence of metadata and various forms of feedback. Useful data were obtained from 159 participants, with an average of over 20 transcriptions per item, which seems a satisfactory result for this type of research. Informing participants about how many items they still had to complete, and not how many they had already completed, turned to be an incentive to do more items. Assigning participants a score for their performance made it more attractive for them to carry out the transcription task, but this seemed to influence their performance. We discuss possible advantages and disadvantages in connection with the aim of the research and consider possible lessons for designing future experiments.
Citations
More filters
Dissertation
01 Jan 2018
TL;DR: In this paper, the authors studied the pronunciation problems of adult Spanish learners of Dutch, and their possible sources, as well as to find out how well native Dutch listeners perceive Spanish-accented Dutch pronunciation, in terms of intelligibility.
Abstract: New Spanish migrants began to arrive in the Netherlands nearly ten years ago, following the economic crisis in 2008 and the steep rise in the Spanish unemployment rate. These Spanish migrants are highly skilled, mobile, highly educated, and speak English well. Most of them work in the high-tech and healthcare sectors. While they can get along communicating in English at first, they soon become aware of the importance of speaking Dutch, because it is required at work or because they want to improve their social interaction. Learning Dutch is hard for adult Spaniards, and when asked what the most difficult aspect of learning Dutch is, most of them would probably answer: “la pronunciacion”, ‘the pronunciation’. The main aim of this investigation is to study the pronunciation problems of adult Spanish learners of Dutch, and their possible sources, as well as to find out how well native Dutch listeners perceive Spanish-accented Dutch pronunciation, in terms of intelligibility. This investigation contributes to the development of specific learning tools for native speakers of Spanish who wish to improve their pronunciation accuracy in Dutch. The outcomes of this dissertation throw light on the specific pronunciation problems Spanish learners of Dutch have, as well as their sources. Such insights can help to propose pedagogical direction in phonological instruction in the Dutch L2 classroom, to develop dedicated CAPT (Computer Assisted Pronunciation Training) programs, and to create materials aimed at raising phonological awareness among Spanish learners.

10 citations


Cites methods from "Palabras. Crowdsourcing transcripti..."

  • ...The same web application was used in the crowdsource study (cf. Burgos et al., 2015; Sanders et al., 2016)....

    [...]

Proceedings ArticleDOI
06 Sep 2015
TL;DR: The paper presented at the 16th Annual Conference of the International Speech Communication Association, 6 september 2015, focused on the development of awareness and understanding of language impairment in the context of speech communication.
Abstract: 16th Annual Conference of the International Speech Communication Association, 6 september 2015

9 citations


Additional excerpts

  • ...shared common transcriptions (see [18])....

    [...]

01 Jan 2019
TL;DR: It is suggested that crowdsourcing could offer solutions for analyzing the large amounts of L2 speech that can be collected through ASR-based CALL applications and that are necessary for effectiveness studies.
Abstract: Despite long-standing interest and recent innovative developments in ASR-based pronunciation instruction and CALL, there is still scepticism about the added value of ASR technology. In this paper we first review recent trends in pronunciation research and important requirements for pronunciation instruction. We go on to consider the difficulties involved in developing ASR-based systems for pronunciation instruction and the possible causes for the paucity of effectiveness studies in ASR-based CALL. We suggest that crowdsourcing could offer solutions for analyzing the large amounts of L2 speech that can be collected through ASR-based CALL applications and that are necessary for effectiveness studies. We provide a brief overview of our own research on ASR-based CALL and of the lessons we learned. Finally, we discuss possible future avenues for research and development.

1 citations


Cites background or methods from "Palabras. Crowdsourcing transcripti..."

  • ...In our own research, for example, we have used crowdsourcing to obtain evaluations of intelligibility of L2 learner speech (Burgos et al., 2015, Sanders et al., 2016) and pathological speech (Ganzeboom et al., 2016)....

    [...]

  • ...Different types of feedback were provided, like percentage correct, words still to transcribe and the majority transcription (Sanders et al. 2016)....

    [...]

References
More filters
Proceedings Article
01 Jan 2011
TL;DR: This work adapts different models from classic motivation theory, work motivation theory and Open Source Software Development to crowdsourcing markets and finds that the extrinsic motivational categories have a strong effect on the time spent on the platform.
Abstract: The payment in paid crowdsourcing markets like Amazon Mechanical Turk is very low, and still collected demographic data shows that the participants are a very diverse group including highly skilled full time workers. Many existing studies on their motivation are rudimental and not grounded on established motivation theory. Therefore, we adapt different models from classic motivation theory, work motivation theory and Open Source Software Development to crowdsourcing markets. The model is tested with a survey of 431 workers on Mechanical Turk. We find that the extrinsic motivational categories (imme-diate payoffs, delayed payoffs, social motivation) have a strong effect on the time spent on the platform. For many workers, however, intrinsic motivation aspects are more important, especially the different facets of enjoyment based motivation like “task autonomy” and “skill variety”. Our contribution is a preliminary model based on established theory intended for the comparison of different crowdsourcing platforms.

546 citations


"Palabras. Crowdsourcing transcripti..." refers background in this paper

  • ...…amount of interesting data, and that 2) feedback and reward had a positive effect because they motivated the participants to continue as in (Kaufmann et al., 2011), but they did not always have a desirable effect on transcription behavior, which can be considered an important lesson for…...

    [...]

Book
15 Feb 2013
TL;DR: This introduction to crowdsourcing as a means of rapidly processing speech data offers speech researchers the hope that they can spend much less time dealing with the data gathering/annotation bottleneck, leaving them to focus on the scientific issues.
Abstract: Provides an insightful and practical introduction to crowdsourcing as a means of rapidly processing speech dataIntended for those who want to get started in the domain and learn how to set up a task, what interfaces are available, how to assess the work, etc. as well as for those who already have used crowdsourcing and want to create better tasks and obtain better assessments of the work of the crowd. It will include screenshots to show examples of good and poor interfaces; examples of case studies in speech processing tasks, going through the task creation process, reviewing options in the interface, in the choice of medium (MTurk or other) and explaining choices, etc.Provides an insightful and practical introduction to crowdsourcing as a means of rapidly processing speech data.Addresses important aspects of this new technique that should be mastered before attempting a crowdsourcing application.Offers speech researchers the hope that they can spend much less time dealing with the data gathering/annotation bottleneck, leaving them to focus on the scientific issues. Readers will directly benefit from the books successful examples of how crowd- sourcing was implemented for speech processing, discussions of interface and processing choices that worked and choices that didnt, and guidelines on how to play and record speech over the internet, how to design tasks, and how to assess workers.Essential reading for researchers and practitioners in speech research groups involved in speech processing

106 citations


"Palabras. Crowdsourcing transcripti..." refers background in this paper

  • ...It turned out that in an application in which participants do not get any monetary remuneration (Cooke et al., 2013), adding a score to the application made it more attractive to do the transcription task....

    [...]

  • ...The use of crowdsourcing to obtain annotations or scorings of intelligibility or accentedness of non-native speech is not new (Evanini et al., 2010; Cooke et al., 2013; Wang et al., 2013)....

    [...]

  • ...In their crowdsourcing experiment (Cooke et al., 2013) had observed that limited feedback could lead to low task engagement....

    [...]

  • ...Keywords: crowdsourcing, transcription, L2 speech...

    [...]

Proceedings Article
06 Jun 2010
TL;DR: The results show that the merged MTurk transcriptions are as accurate as an individual expert transcriber for the read-aloud responses, and are only slightly less accurate for the spontaneous responses.
Abstract: This study investigates the use of Amazon Mechanical Turk for the transcription of non-native speech. Multiple transcriptions were obtained from several distinct MTurk workers and were combined to produce merged transcriptions that had higher levels of agreement with a gold standard transcription than the individual transcriptions. Three different methods for merging transcriptions were compared across two types of responses (spontaneous and read-aloud). The results show that the merged MTurk transcriptions are as accurate as an individual expert transcriber for the read-aloud responses, and are only slightly less accurate for the spontaneous responses.

60 citations


"Palabras. Crowdsourcing transcripti..." refers background in this paper

  • ...The use of crowdsourcing to obtain annotations or scorings of intelligibility or accentedness of non-native speech is not new (Evanini et al., 2010; Cooke et al., 2013; Wang et al., 2013)....

    [...]

Journal ArticleDOI
TL;DR: The authors compare the traditional target approach with two dynamic approaches for investigating regional patterns of variation: the multiple time point approach, which measures formants at successive time points, and the regression approach which estimates formant dynamics over time by fitting polynomial regression equations to formant contours.
Abstract: It is common practice in sociophonetics to measure vowel formants at one (monophthongs) or two (diphthongs) time points. This paper compares this traditional target approach with two dynamic approaches for investigating regional patterns of variation: the multiple time point approach, which measures formants at successive time points, and the regression approach, which estimates formant dynamics over time by fitting polynomial regression equations to formant contours. The speech material consisted of monosyllabic words containing all full vowels of Dutch, except for /y/. These words were read out by 160 speakers of Standard Dutch, who were distributed over four regions in the Netherlands and four regions in Flanders, Belgium. The results show that dynamic approaches outperform the target approach in uncovering regional vowel differences, which suggests that sociophonetic vowel studies that apply the target approach run the risk of overlooking important sociolinguistic patterns.

33 citations


"Palabras. Crowdsourcing transcripti..." refers background in this paper

  • ...The 29 words contain the 15 vowels of Dutch followed by /s/ or /t/, as these consonants are known to alter the preceding vowel least (van der Harst, 2011; van der Harst et al., 2014)....

    [...]

Dissertation
01 Jan 2007

27 citations


"Palabras. Crowdsourcing transcripti..." refers background in this paper

  • ...The 29 words contain the 15 vowels of Dutch followed by /s/ or /t/, as these consonants are known to alter the preceding vowel least (van der Harst, 2011; van der Harst et al., 2014)....

    [...]