Developing rigor in qualitative research: problems and opportunities within sport and exercise psychology

TL;DR: In this paper, the authors discuss three commonly used ways to demonstrate rigor when conducting sport and exercise psychology research, and discuss the importance of rigor in sport psychology research.

Abstract: Qualitative research has grown within sport and exercise psychology and is now widely conducted The purpose of this review is to discuss three commonly used ways to demonstrate rigor when conducti

Summary

Introduction

• Durham Research Online Deposited in DRO: 03 March 2020 Version of attached le: Accepted Version Peer-review status of attached le: Peer-reviewed Citation for published item: Smith, B. and McGannon, K.R. (2018) 'Developing rigor in qualitative research : problems and opportunities within sport and exercise psychology.',.
• These are the method of member checking, the method of inter-rater reliability, 5 and the notion of universal criteria.
• Recommendations for developing rigor when 14 conducting and/or judging qualitative research within sport and exercise psychology are also 15 offered.
• Developments in their thinking can 8 mean that certain historically popular qualitative methods and methodologies might now need 9 rejecting, corrective action, or exigent deliberation.

Rigor in Qualitative Research: Problems and Possibilities 18

• What is 21 meant by rigor can vary amongst scholars; it can mean different things to different people.
• Methods are techniques which, 3 when properly applied, are said to provide rigor.

Member Checking 5

• One extensively used method of rigor within qualitative research is member checking.
• 22 Within sport and exercise psychology member checking is frequently used when 23 conducting qualitative research.
• If researchers 3 wish to hold onto ontological realism then the second direction is to drop epistemological 4 constructionism and confirm the existence of foundations and of a reality outside of ourselves 5 that can be known independently/objectively through the appropriate use of techniques.
• One practical problem researchers may encounter is the possibility that the participant 23 and researcher might provide interpretations of the findings that contradict each other.
• 24 However, member checking provides no means to decide between contradictory claims to 25 problematic as a verification method because a researcher is unable to know with certainty 2 that each participant has faithfully engaged with member checking.

Inter-rater Reliability 6

• According to Culver et al. (2012), in sport and exercise psychology 82.2% of 7 qualitative studies clearly reported reliability testing.
• The latter requires that two or more researchers come together to compare codes 14 and then reconcile through discussion whatever coding discrepancies they may have for the 15 same unit of text.
• The problems with inter-rater reliability 17 already highlighted are important reasons as to why reliability is rejected as a criteria for 18 evaluating the rigor of qualitative research.
• 25 this lack of concern (Wolcott, 1995), is justified since reliability doesn’t make sense when 2 collecting qualitative data.
• Importantly, dialogue with critical friends acknowledges that other 20 and/or additional plausible interpretations of the data can exist that are also defendable but 21 are not being utilized in a particular study or at that time.

Full text

Durham Research Online
Deposited in DRO:
03 March 2020
Version of attached le:
Accepted Version
Peer-review status of attached le:
Peer-reviewed
Citation for published item:
Smith, B. and McGannon, K.R. (2018) 'Developing rigor in qualitative research : problems and opportunities
within sport and exercise psychology.', International review of sport and exercise psychology., 11 (1). pp.
101-121.
Further information on publisher's website:
https://doi.org/10.1080/1750984X.2017.1317357
Publisher's copyright statement:
This is an Accepted Manuscript of an article published by Taylor Francis in International review of sport and exercise
psychology on 14 May 2017 available online:http://www.tandfonline.com/10.1080/1750984X.2017.1317357
Use policy
The full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for
personal research or study, educational, or not-for-prot purposes provided that:
•
a full bibliographic reference is made to the original source
•
a link is made to the metadata record in DRO
•
the full-text is not changed in any way
The full-text must not be sold in any format or medium without the formal permission of the copyright holders.
Please consult the full DRO policy for further details.
Durham University Library, Stockton Road, Durham DH1 3LY, United Kingdom
Tel : +44 (0)191 334 3042 | Fax : +44 (0)191 334 2971
https://dro.dur.ac.uk

Rigor in Qualitative Research 1
1
2
Developing Rigor in Qualitative Research: Problems and Opportunities within
3
Sport and Exercise Psychology
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

Rigor in Qualitative Research 2
Abstract
1
Qualitative research has grown within sport and exercise psychology and is now
2
widely conducted. The purpose of this review is to discuss three commonly used ways to
3
demonstrate rigor when conducting or judging qualitative research in sport and exercise
4
psychology. These are the method of member checking, the method of inter-rater reliability,
5
and the notion of universal criteria. Problems with each method are first highlighted. Member
6
checking and inter-rater reliability are shown to be ineffective for verification,
7
trustworthiness, or reliability purposes. Next, universal criteria within the context of Tracy’s
8
(2010) heavily drawn on paper within sport and exercise psychology is problematized.
9
Throughout the discussion of each method and universal criteria more suitable possibilities
10
for conducting rigorous qualitative research are offered. The paper concludes that to support
11
high quality qualitative research, scholars - including journal editors and reviewers - need to
12
change how rigor is developed and judged, rather than perpetuate the problems with how it
13
has been commonly evaluated in the past. Recommendations for developing rigor when
14
conducting and/or judging qualitative research within sport and exercise psychology are also
15
offered.
16
17
18
Key Words: Member Checking; Inter-rater Reliability; Universal Criteria; Research Quality
19
20
21
22
23
24
25

Rigor in Qualitative Research 3
Qualitative research has been utilized as a form of inquiry within sport and exercise
1
psychology for over three decades. During this time there has also been a rapid growth of
2
qualitative research in the field. For example, in their up-dated review of qualitative research
3
in three North American journals (i.e., Journal of Applied Sport Psychology, Journal of Sport
4
and Exercise Psychology, and The Sport Psychologist) Culver, Gilbert and Sparkes (2012)
5
highlighted that between 2000–2009 there was a 68% increase in the percentage of
6
qualitative studies published since the 1990’s (from 17.3% to 29%). A significant increase in
7
the number of different authors publishing qualitative research in these journals was also
8
noted.
9
In 2009 when Culver et al’s. (2012) review period ended the international journal
10
Qualitative Research in Sport, Exercise and Health began. Attracting hundreds of
11
submissions yearly, and with 5 issues per year, the journal has published empirical papers
12
within sport and exercise psychology, supported different paradigms and theories, and
13
encouraged innovative methods and methodologies. In recent years other international
14
journals (e.g., Psychology of Sport and Exercise and International Journal of Sport and
15
Exercise Psychology) have similarly published different kinds of qualitative research,
16
creating space for work grounded in different and/or multiple methods, methodologies and
17
ways of knowing. The growth of qualitative research within the broad field of sport and
18
exercise is further evident in the increasing number of heavily cited books devoted solely to
19
qualitative research (e.g., Jones, Brown, & Holloway, 2012; Pitney & Parker, 2009; Smith &
20
Sparkes, 2016a; Sparkes & Smith, 2013; Young & Atkinson, 2012). Moreover, a growing
21
number of conferences and workshops are attempting to address the demand for qualitative
22
research from students, researchers, practitioners, and policy makers. For instance, the bi-
23
annual International Conference for Qualitative Sport and Exercise (see twitter
24
@QRSE2018) and quarterly congress by The European Federation of Sport Psychology (see
25

Rigor in Qualitative Research 4
http://www.fepsac.com) have hosted workshops dedicated to qualitative research and
1
showcased hundreds of qualitative research papers from established scholars and newcomers
2
from around the world.
3
In light of the foregoing, it would appear that within sport and exercise psychology
4
qualitative research has flourished considerably in recent years. Yet despite flourishing, we
5
cannot be complacent. Like any vibrant field, important intellectual developments within
6
qualitative research have taken place. These developments include work on rigor. Keeping
7
abreast of intellectual developments is of course crucial. Developments in our thinking can
8
mean that certain historically popular qualitative methods and methodologies might now need
9
rejecting, corrective action, or exigent deliberation. Thus, in order for high-quality research to
10
be conducted researchers need to stay engaged with contemporary methodological thinking
11
by, for example, connecting with recently published work (e.g., Birt, Scott, Cavers,
12
Campbell, & Walter, 2016; Burke, 2016; Levitt, Motulsky, Wertz, Morrow, & Ponterrotto,
13
2016; Morse, 2016) on matters like rigor. When we ignore such thinking there is a risk of
14
producing outdated, flawed, stagnant and/or limited research.
15
As the qualitative research landscape continues to rapidly expand and flourish,
16
engaging with contemporary literature concerning the latest thoughts and developments can
17
be challenging. For instance, given our investments in certain methods or methodologies it
18
can be difficult to read work that substantially questions a method or methodology used often
19
in the past. With the large amount of research being regularly published, it may also be
20
difficult to keep abreast of developments in the qualitative methodological literature, even
21
when one is interested in embracing them. In that regard, reviews can be useful resources to
22
take stock of developments, offering ways forward in light of said developments. The
23
purpose of this review paper is to discuss one contemporary development around the theme
24
of ‘rigor in qualitative research’. Organized under that theme, three widely used ways of
25

Frequently asked questions

Q1. What are the contributions in this paper?

The purpose of this review is to discuss three commonly used ways to 3 demonstrate rigor when conducting or judging qualitative research in sport and exercise 4 psychology. The paper concludes that to support 11 high quality qualitative research, scholars including journal editors and reviewers need to 12 change how rigor is developed and judged, rather than perpetuate the problems with how it 13 has been commonly evaluated in the past.