BIROn - Birkbeck Institutional Research Online
Saito, Kazuya and Trofimovich, Pavel and Isaacs, T. (2017) Using listener
judgments to investigate linguistic influences on L2 comprehensibility and
accentedness: a validation and generalization study. Applied Linguistics 38
(4), pp. 439-462. ISSN 0142-6001.
Downloaded from:
Usage Guidelines:
Please refer to usage guidelines at or alternatively
contact lib-eprints@bbk.ac.uk.
Title:
Using Listener Judgements to Investigate Linguistic Influences on L2 Comprehensibility and
Accentedness: A Validation and Generalization Study
Running Head:
COMPREHENSIBILITY VERSUS ACCENT
COMPREHENSIBILITY VERSUS ACCENT 1
Abstract
The current study investigated linguistic influences on comprehensibility (ease of
understanding) and accentedness (linguistic nativelikeness) in second language (L2) learners’
extemporaneous speech. Target materials included picture narratives from 40 native French
speakers of English from different proficiency levels. The narratives were subsequently rated
by 20 native speakers with or without linguistic and pedagogical experience for
comprehensibility, accentedness, and 11 linguistic variables spanning the domains of
phonology, lexis, grammar, and discourse structure. Results showed that comprehensibility
was associated with several linguistic variables (vowel/consonant errors, word stress, fluency,
lexis, grammar), whereas accentedness was chiefly linked to pronunciation (vowel/consonant
errors, word stress). Native speaking listeners thus appear to pay particular attention to
pronunciation, rather than lexis and grammar, to evaluate nativelikeness but tend to consider
various sources of linguistic information in L2 speech in judging comprehensibility. The use
of listener ratings (perceptual measures) in evaluating linguistic aspects of learner speech and
their implications for language assessment and pedagogy are discussed.
Key words. Comprehensibility, accent, second language speech, pronunciation, fluency,
vocabulary, grammar, discourse
COMPREHENSIBILITY VERSUS ACCENT 2
Using Listener Judgements to Investigate Linguistic Influences on L2 Comprehensibility and
Accentedness: A Validation and Generalization Study
Learning a second language (L2) has clearly become a necessity in the current global
society. Although many teachers and their students around the world view nativelike
linguistic abilities as the ultimate goal of L2 learning (e.g., Derwing, 2003), previous research
has convincingly shown that few adult learners can attain this goal, even if they start at an
early age, and that a perceptible foreign accent is a common feature of L2 speech (e.g., Flege,
Munro, & MacKay, 1995). Thus, it is important to set realistic instructional goals for learners,
prioritizing comprehensibility, which refers to listeners’ perception of how easy or difficult it
is for them to understand L2 speech, over linguistic nativelikeness, typically measured
through accentedness or listeners’ perception of how closely speakers can approximate
speech patterns of the target-language community (see Derwing & Munro, 2009). Indeed,
learners can communicate successfully in the vast majority of business and academic settings
without needing to sound nativelike (Derwing & Munro, 2009). Underlying this view is an
assumption that comprehensibility and accentedness are two interrelated yet separable
constructs and that not all linguistic errors linked to accent equally hinder comprehensibility.
While there is some evidence suggesting that comprehensibility is distinct from accentedness
(e.g., Jenkins, 2000; Kang, Rubin, & Pickering, 2010; Munro & Derwing, 1995), it is still
relatively unclear which linguistic aspects primarily underlie comprehensibility and which are
uniquely associated with accent. Furthermore, it has remained controversial whether and to
what degree native speakers who are accustomed to listening to accented L2 speech (e.g.,
experienced ESL teachers, graduate students in applied linguistics) can perceive various
linguistic dimensions of comprehensibility and accentedness, compared to those speakers
who do not have much experience with foreign accents (Isaacs & Thomson, 2013). Therefore,
the goal of this study is twofold: (a) to identify 11 linguistic variables that can be used by
linguistically experienced and inexperienced listeners to evaluate L2 speech, and (b) to
examine the contribution of these variables to L2 comprehensibility and accentedness.
Why Target Comprehensibility?
Before reviewing relevant background literature on the relationship between
comprehensibility and accentedness, it is first important to clarify why the current study
targeted comprehensibility rather than intelligibility as a measure of understanding. In a
review of L2 intelligibility research, Levis (2006) outlined a distinction between broad and
narrow views of intelligibility. In a narrow sense, intelligibility is conceptualized as a product
of understanding and is operationally defined as accuracy with which listeners
orthographically transcribe L2 speech (e.g., Munro & Derwing, 1995) or answer
comprehension questions related to its content (e.g., Hahn, 2004). In a broad sense, however,
intelligibility refers to listeners’ subjective perception of how much or how easily they
understand L2 speech. In this sense, according to Levis, intelligibility is “not usually
distinguished from closely related terms such as comprehensibility” (p. 252), in that both
constructs are measured through listeners’ scalar ratings, without reference to any objective
measure of understanding (e.g., Varonis & Gass, 1982). Indeed, outside research, most real-
world applications of intelligibility, such as high-stakes assessment instruments (e.g., TOEFL,
IELTS, CEFR), involve scalar ratings, which implies that the targeted construct is in fact
comprehensibility. Thus, comprehensibility is subsumed within Levis’s broad sense of
intelligibility and represents a common and easy-to-use metric of understanding in both
research and real-world contexts (e.g., Levis, 2006). In keeping with this tradition of using a
rated measure of understanding, the current study therefore targets intelligibility in its broad
sense, focusing on comprehensibility, with the overall goal of distinguishing those linguistic
COMPREHENSIBILITY VERSUS ACCENT 3
dimensions of L2 speech that feed into comprehensibility from those that are linked to
accentedness.
Comprehensibility versus Accentedness
From a theoretical perspective, a focus on comprehensibility (rather than
accentedness) is central to the idea that language learning is most efficient when learners are
exposed to meaningful language, especially through interaction. The Interaction Hypothesis,
for instance, posits that language learning primarily takes place in situations when
communication is compromised during L2 conversational interaction (Long, 1996). When
interlocutors encounter communication breakdowns, they make intuitive or conscious effort
to repair linguistic errors causing misunderstanding, engaging in negotiation for meaning.
Assuming that certain linguistic features in learner speech are more likely than others to
cause communication breakdowns and thus trigger negotiation for meaning (Mackey, Gass,
& McDonough, 2000), the learning value of L2 conversational interaction will be greatest for
those linguistic features that are tied to comprehensibility rather than those that only
contribute to the perception of accent (Derwing & Munro, 2009).
Although interaction itself is thought to make problematic features available to the
learner, conversationally-modified input and output appear to be facilitative of L2 learning
only when learners are developmentally ready, that is, when they have some degree of
metalinguistic awareness of the problematic features in question (Mackey & Philp, 1998).
Interaction also seems to play a facilitative role in L2 learning as a way of consolidating
partially-acquired knowledge rather than as a means of acquiring new knowledge (Shintani,
Li, & Ellis, 2013). Therefore, identifying and teaching the linguistic features that
predominantly impact comprehensibility should equip learners with the kinds of knowledge
that would be necessary for them to make the most of the L2 input and interaction.
Comprehensibility, with its focus on the ease or difficulty of interlocutors’ mutual
understanding, thus becomes crucially important in enabling researchers and teachers to both
isolate and target such linguistic dimensions.
Several studies have investigated which aspects of pronunciation, fluency, lexis, and
grammar impact listener understanding through subjective judgments of comprehensibility or
through more objective measures of intelligibility. With respect to pronunciation and fluency,
listener understanding is associated with various aspects of L2 speech, including individual
vowels and consonants with high functional load (Munro & Derwing, 2006), sentence stress
(Hahn, 2004), word stress (Field, 2005), speech rate (Derwing, Rossiter, Munro, & Thomson,
2004), as well as pitch range, stress, and pause or syllable length (Kang et al., 2010; Tajima,
Port, & Dalby, 1997; Winters & O’Brien, 2013). With regard to lexis and grammar, which
are a focus of a smaller volume of research, comprehensibility is associated with measures of
grammar accuracy (Munro & Derwing, 1995), and listener understanding is compromised
when listeners are exposed to ungrammatical sentences (Varonis & Gass, 1982) or poor word
choice (Fayer & Krasinski, 1987). In sum, listener understanding appears to be linked to a
variety of linguistic variables, spanning the domains of pronunciation, fluency, lexis, and
grammar. Yet there exists a substantial overlap between the constructs of comprehensibility
and accentedness in the domains of pronunciation and fluency, with such factors as segmental
accuracy, temporal measures, syllable duration, stress, and pitch range also contributing to
the perception of an L2 accent (Anderson-Hsieh, Johnson, & Koehler, 1992; Winters &
O’Brien, 2013).
While influences of individual properties of speech on comprehensibility and
accentedness are relatively well understood, it is still unclear how multiple linguistic
dimensions interact and whether they affect comprehensibility differently from accentedness.
One reason for this is that previous studies included only a handful of linguistic measures or a
small number of listeners (e.g., Zielinski, 2008), and few examined both comprehensibility