Choosing a suitable sample size in qualitative research is an area of conceptual debate and practical uncertainty. That sample size principles, guidelines and tools have been developed to enable researchers to set, and justify the acceptability of, their sample size is an indication that the issue constitutes an important marker of the quality of qualitative research. Nevertheless, research shows that sample size sufficiency reporting is often poor, if not absent, across a range of disciplinary fields. A systematic analysis of single-interview-per-participant designs within three health-related journals from the disciplines of psychology, sociology and medicine, over a 15-year period, was conducted to examine whether and how sample sizes were justified and how sample size was characterised and discussed by authors. Data pertinent to sample size were extracted and analysed using qualitative and quantitative analytic techniques. Our findings demonstrate that provision of sample size justifications in qualitative health research is limited; is not contingent on the number of interviews; and relates to the journal of publication. Defence of sample size was most frequently supported across all three journals with reference to the principle of saturation and to pragmatic considerations. Qualitative sample sizes were predominantly – and often without justification – characterised as insufficient (i.e., ‘small’) and discussed in the context of study limitations. Sample size insufficiency was seen to threaten the validity and generalizability of studies’ results, with the latter being frequently conceived in nomothetic terms. We recommend, firstly, that qualitative health researchers be more transparent about evaluations of their sample size sufficiency, situating these within broader and more encompassing assessments of data adequacy. Secondly, we invite researchers critically to consider how saturation parameters found in prior methodological studies and sample size community norms might best inform, and apply to, their own project and encourage that data adequacy is best appraised with reference to features that are intrinsic to the study at hand. Finally, those reviewing papers have a vital role in supporting and encouraging transparent study-specific reporting.

/pdf/characterising-and-justifying-sample-size-sufficiency-in-2hn81gqvva.pdf

Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period

Author Ray Pawson presents a devastating critique of the dominant approach to systematic review namely the 'meta-analytic' approach as sponsored by the Cochrane and Campbell collaborations. In its place is commended an approach that he terms 'realist synthesis'. On this vision, the real purpose of systematic review is better to understand program theory, so that policies Author Ray Pawson presents a devastating critique of the dominant approach to systematic review namely the 'meta-analytic' approach as sponsored by the Cochrane and Campbell collaborations. In its place is commended an approach that he terms 'realist synthesis'. On this vision, the real purpose of systematic review is better to understand program theory, so that policies can be properly targeted and developed to counter an ever-changing landscape of social problems. The book will be essential reading for all those who loved (or loathed) the arguments developed in Realistic Evaluation (Sage, 1997). It offers a complete blueprint for research synthesis, supported by detailed illustrations and worked examples from across the policy waterfront.

Evidence-based Policy: A Realist Perspective

Few empirical studies exist to guide researchers in determining the number of focus groups necessary for a research study. The analyses described here provide foundational evidence to help research...

How Many Focus Groups Are Enough? Building an Evidence Base for Nonprobability Sample Sizes

Data saturation is the most commonly employed concept for estimating sample sizes in qualitative research. Over the past 20 years, scholars using both empirical research and mathematical/statistical models have made significant contributions to the question: How many qualitative interviews are enough? This body of work has advanced the evidence base for sample size estimation in qualitative inquiry during the design phase of a study, prior to data collection, but it does not provide qualitative researchers with a simple and reliable way to determine the adequacy of sample sizes during and/or after data collection. Using the principle of saturation as a foundation, we describe and validate a simple-to-apply method for assessing and reporting on saturation in the context of inductive thematic analyses. Following a review of the empirical research on data saturation and sample size estimation in qualitative research, we propose an alternative way to evaluate saturation that overcomes the shortcomings and challenges associated with existing methods identified in our review. Our approach includes three primary elements in its calculation and assessment: Base Size, Run Length, and New Information Threshold. We additionally propose a more flexible approach to reporting saturation. To validate our method, we use a bootstrapping technique on three existing thematically coded qualitative datasets generated from in-depth interviews. Results from this analysis indicate the method we propose to assess and report on saturation is feasible and congruent with findings from earlier studies.

/pdf/a-simple-method-to-assess-and-report-thematic-saturation-in-2g07rb6ots.pdf

A simple method to assess and report thematic saturation in qualitative research.

Thematic analysis (TA) is widely used in qualitative psychology. In using TA, researchers must choose between a diverse range of approaches that can differ considerably in their underlying (but often implicit) conceptualizations of qualitative research, meaningful knowledge production and key constructs such as themes, as well as analytic procedures. This diversity within the method of TA is typically poorly understood and rarely acknowledged, resulting in the frequent publication of research lacking in design coherence. Furthermore, because TA offers researchers something closer to a method (a trans-theoretical tool or technique) rather than a methodology (a theoretically-informed framework for research), one with considerable theoretical and design flexibility, researchers need to engage in careful conceptual and design thinking to produce TA research with methodological integrity. In this paper, we support researchers in their conceptual and design thinking for TA, and particularly for the reflexive approach we have developed, by guiding them through the conceptual underpinnings of different approaches to TA, and key design considerations. We outline our typology of three main “schools” of TA – coding reliability, codebook and reflexive – and consider how these differ in their conceptual underpinnings, with a particular focus on the distinct characteristics of our reflexive approach. We discuss key areas of design – research questions, data collection, participant/data item selection strategy and criteria, ethics, and quality standards and practices – and end with guidance on reporting standards for reflexive TA.

Conceptual and design thinking for thematic analysis

Thematic analysis is frequently used to analyse qualitative data in psychology, healthcare, social research and beyond. An important stage in planning a study is determining how large a sample size may be required, however current guidelines for thematic analysis are varied, ranging from around 2 to over 400 and it is unclear how to choose a value from the space in between. Some guidance can also not be applied prospectively. This paper introduces a tool to help users think about what would be a useful sample size for their particular context when investigating patterns across participants. The calculation depends on (a) the expected population theme prevalence of the least prevalent theme, derived either from prior knowledge or based on the prevalence of the rarest themes considered worth uncovering, e.g. 1 in 10, 1 in 100; (b) the number of desired instances of the theme; and (c) the power of the study. An adequately powered study will have a high likelihood of finding sufficient themes of the desired prevalence. This calculation can then be used alongside other considerations. We illustrate how to use the method to calculate sample size before starting a study and achieved power given a sample size, providing tables of answers and code for use in the free software, R. Sample sizes are comparable to those found in the literature, for example to have 80% power to detect two instances of a theme with 10% prevalence, 29 participants are required. Increasing power, increasing the number of instances or decreasing prevalence increases the sample size needed. We do not propose this as a ritualistic requirement for study design, but rather as a pragmatic supporting tool to help plan studies using thematic analysis.

Hww Potts

Papers

Supporting thinking on sample sizes for thematic analyses: A quantitative tool (vol 18, pg 669, 2015)