Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series
TL;DR: The rationale and conceptual basis for CERQual, the aims of the approach, how the approach was developed, and its main components are described, including the purpose and structure of this series and the growing role for qualitative evidence in decision-making are discussed.
Abstract: The GRADE-CERQual (‘Confidence in the Evidence from Reviews of Qualitative research’) approach provides guidance for assessing how much confidence to place in findings from systematic reviews of qualitative research (or qualitative evidence syntheses). The approach has been developed to support the use of findings from qualitative evidence syntheses in decision-making, including guideline development and policy formulation. Confidence in the evidence from qualitative evidence syntheses is an assessment of the extent to which a review finding is a reasonable representation of the phenomenon of interest. CERQual provides a systematic and transparent framework for assessing confidence in individual review findings, based on consideration of four components: (1) methodological limitations, (2) coherence, (3) adequacy of data, and (4) relevance. A fifth component, dissemination (or publication) bias, may also be important and is being explored. As with the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach for effectiveness evidence, CERQual suggests summarising evidence in succinct, transparent, and informative Summary of Qualitative Findings tables. These tables are designed to communicate the review findings and the CERQual assessment of confidence in each finding. This article is the first of a seven-part series providing guidance on how to apply the CERQual approach. In this paper, we describe the rationale and conceptual basis for CERQual, the aims of the approach, how the approach was developed, and its main components. We also outline the purpose and structure of this series and discuss the growing role for qualitative evidence in decision-making. Papers 3, 4, 5, 6, and 7 in this series discuss each CERQual component, including the rationale for including the component in the approach, how the component is conceptualised, and how it should be assessed. Paper 2 discusses how to make an overall assessment of confidence in a review finding and how to create a Summary of Qualitative Findings table. The series is intended primarily for those undertaking qualitative evidence syntheses or using their findings in decision-making processes but is also relevant to guideline development agencies, primary qualitative researchers, and implementation scientists and practitioners.
Citations
More filters
TL;DR: This review points to several barriers and facilitators that influenced healthcare workers' ability to adhere to IPC guidelines for respiratory infectious diseases.
Abstract: Background
This review is one of a series of rapid reviews that Cochrane contributors have prepared to inform the 2020 COVID‐19 pandemic.
When new respiratory infectious diseases become widespread, such as during the COVID‐19 pandemic, healthcare workers’ adherence to infection prevention and control (IPC) guidelines becomes even more important. Strategies in these guidelines include the use of personal protective equipment (PPE) such as masks, face shields, gloves and gowns; the separation of patients with respiratory infections from others; and stricter cleaning routines. These strategies can be difficult and time‐consuming to adhere to in practice. Authorities and healthcare facilities therefore need to consider how best to support healthcare workers to implement them.
Objectives
To identify barriers and facilitators to healthcare workers’ adherence to IPC guidelines for respiratory infectious diseases.
Search methods
We searched OVID MEDLINE on 26 March 2020. As we searched only one database due to time constraints, we also undertook a rigorous and comprehensive scoping exercise and search of the reference lists of key papers. We did not apply any date limit or language limits.
Selection criteria
We included qualitative and mixed‐methods studies (with a distinct qualitative component) that focused on the experiences and perceptions of healthcare workers towards factors that impact on their ability to adhere to IPC guidelines for respiratory infectious diseases. We included studies of any type of healthcare worker with responsibility for patient care. We included studies that focused on IPC guidelines (local, national or international) for respiratory infectious diseases in any healthcare setting. These selection criteria were framed by an understanding of the needs of health workers during the COVID‐19 pandemic.
Data collection and analysis
Four review authors independently assessed the titles, abstracts and full texts identified by our search. We used a prespecified sampling frame to sample from the eligible studies, aiming to capture a range of respiratory infectious disease types, geographical spread and data‐rich studies. We extracted data using a data extraction form designed for this synthesis. We assessed methodological limitations using an adapted version of the Critical Skills Appraisal Programme (CASP) tool. We used a ‘best fit framework approach’ to analyse and synthesise the evidence. This provided upfront analytical categories, with scope for further thematic analysis. We used the GRADE‐CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach to assess our confidence in each finding. We examined each review finding to identify factors that may influence intervention implementation and developed implications for practice.
Main results
We found 36 relevant studies and sampled 20 of these studies for our analysis. Ten of these studies were from Asia, four from Africa, four from Central and North America and two from Australia. The studies explored the views and experiences of nurses, doctors and other healthcare workers when dealing with severe acute respiratory syndrome (SARS), H1N1, MERS (Middle East respiratory syndrome), tuberculosis (TB), or seasonal influenza. Most of these healthcare workers worked in hospitals; others worked in primary and community care settings.
Our review points to several barriers and facilitators that influenced healthcare workers’ ability to adhere to IPC guidelines. The following factors are based on findings assessed as of moderate to high confidence.
Healthcare workers felt unsure as to how to adhere to local guidelines when they were long and ambiguous or did not reflect national or international guidelines. They could feel overwhelmed because local guidelines were constantly changing. They also described how IPC strategies led to increased workloads and fatigue, for instance because they had to use PPE and take on additional cleaning. Healthcare workers described how their responses to IPC guidelines were influenced by the level of support they felt that they received from their management team.
Clear communication about IPC guidelines was seen as vital. But healthcare workers pointed to a lack of training about the infection itself and about how to use PPE. They also thought it was a problem when training was not mandatory.
Sufficient space to isolate patients was also seen as vital. A lack of isolation rooms, anterooms and shower facilities was a problem. Other important practical measures described by healthcare workers included minimising overcrowding, fast‐tracking infected patients, restricting visitors, and providing easy access to handwashing facilities.
A lack of PPE, and equipment that was of poor quality, was a serious concern for healthcare workers and managers. They also pointed to the need to adjust the volume of supplies as infection outbreaks continued.
Healthcare workers believed that they followed IPC guidance more closely when they saw the value of it. Some healthcare workers felt motivated to follow the guidance because of fear of infecting themselves or their families, or because they felt responsible for their patients. Some healthcare workers found it difficult to use masks and other equipment when it made patients feel isolated, frightened or stigmatised. Healthcare workers also found masks and other equipment uncomfortable to use. The workplace culture could also influence whether healthcare workers followed IPC guidelines or not.
Across many of the findings, healthcare workers pointed to the importance of including all staff, including cleaning staff, porters, kitchen staff and other support staff when implementing IPC guidelines.
Authors' conclusions
Healthcare workers point to several factors that influence their ability and willingness to follow IPC guidelines when managing respiratory infectious diseases. These include factors tied to the guideline itself and how it is communicated, support from managers, workplace culture, training, physical space, access to and trust in personal protective equipment, and a desire to deliver good patient care. The review also highlights the importance of including all facility staff, including support staff, when implementing IPC guidelines.
413 citations
TL;DR: The aim of this review was to identify barriers and facilitators that may impact on the implementation of interventions aimed at supporting the resilience and mental health of frontline health and social care professionals during and after a disease outbreak, epidemic or pandemic.
Abstract: Background Evidence from disease epidemics shows that healthcare workers are at risk of developing short- and long-term mental health problems. The World Health Organization (WHO) has warned about the potential negative impact of the COVID-19 crisis on the mental well-being of health and social care professionals. Symptoms of mental health problems commonly include depression, anxiety, stress, and additional cognitive and social problems; these can impact on function in the workplace. The mental health and resilience (ability to cope with the negative effects of stress) of frontline health and social care professionals ('frontline workers' in this review) could be supported during disease epidemics by workplace interventions, interventions to support basic daily needs, psychological support interventions, pharmacological interventions, or a combination of any or all of these. Objectives Objective 1: to assess the effects of interventions aimed at supporting the resilience and mental health of frontline health and social care professionals during and after a disease outbreak, epidemic or pandemic. Objective 2: to identify barriers and facilitators that may impact on the implementation of interventions aimed at supporting the resilience and mental health of frontline health and social care professionals during and after a disease outbreak, epidemic or pandemic. Search methods On 28 May 2020 we searched the Cochrane Database of Systematic Reviews, CENTRAL, MEDLINE, Embase, Web of Science, PsycINFO, CINAHL, Global Index Medicus databases and WHO Institutional Repository for Information Sharing. We also searched ongoing trials registers and Google Scholar. We ran all searches from the year 2002 onwards, with no language restrictions. Selection criteria We included studies in which participants were health and social care professionals working at the front line during infectious disease outbreaks, categorised as epidemics or pandemics by WHO, from 2002 onwards. For objective 1 we included quantitative evidence from randomised trials, non-randomised trials, controlled before-after studies and interrupted time series studies, which investigated the effect of any intervention to support mental health or resilience, compared to no intervention, standard care, placebo or attention control intervention, or other active interventions. For objective 2 we included qualitative evidence from studies that described barriers and facilitators to the implementation of interventions. Outcomes critical to this review were general mental health and resilience. Additional outcomes included psychological symptoms of anxiety, depression or stress; burnout; other mental health disorders; workplace staffing; and adverse events arising from interventions. Data collection and analysis Pairs of review authors independently applied selection criteria to abstracts and full papers, with disagreements resolved through discussion. One review author systematically extracted data, cross-checked by a second review author. For objective 1, we assessed risk of bias of studies of effectiveness using the Cochrane 'Risk of bias' tool. For objective 2, we assessed methodological limitations using either the CASP (Critical Appraisal Skills Programme) qualitative study tool, for qualitative studies, or WEIRD (Ways of Evaluating Important and Relevant Data) tool, for descriptive studies. We planned meta-analyses of pairwise comparisons for outcomes if direct evidence were available. Two review authors extracted evidence relating to barriers and facilitators to implementation, organised these around the domains of the Consolidated Framework of Implementation Research, and used the GRADE-CERQual approach to assess confidence in each finding. We planned to produce an overarching synthesis, bringing quantitative and qualitative findings together. Main results We included 16 studies that reported implementation of an intervention aimed at supporting the resilience or mental health of frontline workers during disease outbreaks (severe acute respiratory syndrome (SARS): 2; Ebola: 9; Middle East respiratory syndrome (MERS): 1; COVID-19: 4). Interventions studied included workplace interventions, such as training, structure and communication (6 studies); psychological support interventions, such as counselling and psychology services (8 studies); and multifaceted interventions (2 studies). Objective 1: a mixed-methods study that incorporated a cluster-randomised trial, investigating the effect of a work-based intervention, provided very low-certainty evidence about the effect of training frontline healthcare workers to deliver psychological first aid on a measure of burnout. Objective 2: we included all 16 studies in our qualitative evidence synthesis; we classified seven as qualitative and nine as descriptive studies. We identified 17 key findings from multiple barriers and facilitators reported in studies. We did not have high confidence in any of the findings; we had moderate confidence in six findings and low to very low confidence in 11 findings. We are moderately confident that the following two factors were barriers to intervention implementation: frontline workers, or the organisations in which they worked, not being fully aware of what they needed to support their mental well-being; and a lack of equipment, staff time or skills needed for an intervention. We are moderately confident that the following three factors were facilitators of intervention implementation: interventions that could be adapted for local needs; having effective communication, both formally and socially; and having positive, safe and supportive learning environments for frontline workers. We are moderately confident that the knowledge or beliefs, or both, that people have about an intervention can act as either barriers or facilitators to implementation of the intervention. Authors' conclusions There is a lack of both quantitative and qualitative evidence from studies carried out during or after disease epidemics and pandemics that can inform the selection of interventions that are beneficial to the resilience and mental health of frontline workers. Alternative sources of evidence (e.g. from other healthcare crises, and general evidence about interventions that support mental well-being) could therefore be used to inform decision making. When selecting interventions aimed at supporting frontline workers' mental health, organisational, social, personal, and psychological factors may all be important. Research to determine the effectiveness of interventions is a high priority. The COVID-19 pandemic provides unique opportunities for robust evaluation of interventions. Future studies must be developed with appropriately rigorous planning, including development, peer review and transparent reporting of research protocols, following guidance and standards for best practice, and with appropriate length of follow-up. Factors that may act as barriers and facilitators to implementation of interventions should be considered during the planning of future research and when selecting interventions to deliver within local settings.
310 citations
TL;DR: The end point of a CERQual assessment is described and should be read in conjunction with the other papers in the series that provide information on assessing individual CerQual components.
Abstract: The GRADE-CERQual (Confidence in Evidence from Reviews of Qualitative research) approach has been developed by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group. The approach has been developed to support the use of findings from qualitative evidence syntheses in decision making, including guideline development and policy formulation. CERQual includes four components for assessing how much confidence to place in findings from reviews of qualitative research (also referred to as qualitative evidence syntheses): (1) methodological limitations, (2) coherence, (3) adequacy of data and (4) relevance. This paper is part of a series providing guidance on how to apply CERQual and focuses on making an overall assessment of confidence in a review finding and creating a CERQual Evidence Profile and a CERQual Summary of Qualitative Findings table. We developed this guidance by examining the methods used by other GRADE approaches, gathering feedback from relevant research communities and developing consensus through project group meetings. We then piloted the guidance on several qualitative evidence syntheses before agreeing on the approach. Confidence in the evidence is an assessment of the extent to which a review finding is a reasonable representation of the phenomenon of interest. Creating a summary of each review finding and deciding whether or not CERQual should be used are important steps prior to assessing confidence. Confidence should be assessed for each review finding individually, based on the judgements made for each of the four CERQual components. Four levels are used to describe the overall assessment of confidence: high, moderate, low or very low. The overall CERQual assessment for each review finding should be explained in a CERQual Evidence Profile and Summary of Qualitative Findings table. Structuring and summarising review findings, assessing confidence in those findings using CERQual and creating a CERQual Evidence Profile and Summary of Qualitative Findings table should be essential components of undertaking qualitative evidence syntheses. This paper describes the end point of a CERQual assessment and should be read in conjunction with the other papers in the series that provide information on assessing individual CERQual components.
271 citations
Cites background or methods from "Applying GRADE-CERQual to qualitati..."
...The importance of assessing confidence in qualitative evidence is discussed in the first paper in this series [1]....
[...]
...The methods used to develop CERQual are described in more detail in the first paper in this series [1]....
[...]
...The GRADE-CERQual approach (hereafter referred to as CERQual) is applied to individual review findings from a qualitative evidence synthesis [1]....
[...]
...Four levels are used to describe the overall assessment of confidence in a review finding—high, moderate, low or very low—and these are defined in Table 3 in the first paper in this series [1]....
[...]
TL;DR: This Review presents a high-level synthesis of global gender data, summarise progress towards gender equality in science, medicine, and global health, review the evidence for why gender Equality in these fields matters in terms of health and social outcomes, and reflect on strategies to promote change.
265 citations
TL;DR: The qualitative and quantitative evidence demonstrate the extensive harms associated with criminalisation of sex work, including laws and enforcement targeting the sale and purchase of sex, and activities relating to sex work organisation.
Abstract: Background Sex workers are at disproportionate risk of violence and sexual and emotional ill health, harms that have been linked to the criminalisation of sex work. We synthesised evidence on the extent to which sex work laws and policing practices affect sex workers’ safety, health, and access to services, and the pathways through which these effects occur. Methods and findings We searched bibliographic databases between 1 January 1990 and 9 May 2018 for qualitative and quantitative research involving sex workers of all genders and terms relating to legislation, police, and health. We operationalised categories of lawful and unlawful police repression of sex workers or their clients, including criminal and administrative penalties. We included quantitative studies that measured associations between policing and outcomes of violence, health, and access to services, and qualitative studies that explored related pathways. We conducted a meta-analysis to estimate the average effect of experiencing sexual/physical violence, HIV or sexually transmitted infections (STIs), and condomless sex, among individuals exposed to repressive policing compared to those unexposed. Qualitative studies were synthesised iteratively, inductively, and thematically. We reviewed 40 quantitative and 94 qualitative studies. Repressive policing of sex workers was associated with increased risk of sexual/physical violence from clients or other parties (odds ratio [OR] 2.99, 95% CI 1.96–4.57), HIV/STI (OR 1.87, 95% CI 1.60–2.19), and condomless sex (OR 1.42, 95% CI 1.03–1.94). The qualitative synthesis identified diverse forms of police violence and abuses of power, including arbitrary arrest, bribery and extortion, physical and sexual violence, failure to provide access to justice, and forced HIV testing. It showed that in contexts of criminalisation, the threat and enactment of police harassment and arrest of sex workers or their clients displaced sex workers into isolated work locations, disrupting peer support networks and service access, and limiting risk reduction opportunities. It discouraged sex workers from carrying condoms and exacerbated existing inequalities experienced by transgender, migrant, and drug-using sex workers. Evidence from decriminalised settings suggests that sex workers in these settings have greater negotiating power with clients and better access to justice. Quantitative findings were limited by high heterogeneity in the meta-analysis for some outcomes and insufficient data to conduct meta-analyses for others, as well as variable sample size and study quality. Few studies reported whether arrest was related to sex work or another offence, limiting our ability to assess the associations between sex work criminalisation and outcomes relative to other penalties or abuses of police power, and all studies were observational, prohibiting any causal inference. Few studies included trans- and cisgender male sex workers, and little evidence related to emotional health and access to healthcare beyond HIV/STI testing. Conclusions Together, the qualitative and quantitative evidence demonstrate the extensive harms associated with criminalisation of sex work, including laws and enforcement targeting the sale and purchase of sex, and activities relating to sex work organisation. There is an urgent need to reform sex-work-related laws and institutional practices so as to reduce harms and barriers to the realisation of health.
231 citations
References
More filters
TL;DR: The advantages of the GRADE system are explored, which is increasingly being adopted by organisations worldwide and which is often praised for its high level of consistency.
Abstract: Guidelines are inconsistent in how they rate the quality of evidence and the strength of recommendations. This article explores the advantages of the GRADE system, which is increasingly being adopted by organisations worldwide
13,324 citations
"Applying GRADE-CERQual to qualitati..." refers background in this paper
...However, we would argue that the accessibility and transparency of this approach outweigh its limitations [40]....
[...]
TL;DR: The GRADE process begins with asking an explicit question, including specification of all important outcomes, and provides explicit criteria for rating the quality of evidence that include study design, risk of bias, imprecision, inconsistency, indirectness, and magnitude of effect.
6,093 citations
TL;DR: Methods for qualitative synthesis vary across a range of dimensions and broadly fall into 'realist' or 'idealist' epistemologies, which partly accounts for these differences.
Abstract: In recent years, a growing number of methods for synthesising qualitative research have emerged, particularly in relation to health-related research. There is a need for both researchers and commissioners to be able to distinguish between these methods and to select which method is the most appropriate to their situation. A number of methodological and conceptual links between these methods were identified and explored, while contrasting epistemological positions explained differences in approaches to issues such as quality assessment and extent of iteration. Methods broadly fall into 'realist' or 'idealist' epistemologies, which partly accounts for these differences. Methods for qualitative synthesis vary across a range of dimensions. Commissioners of qualitative syntheses might wish to consider the kind of product they want and select their method – or type of method – accordingly.
1,486 citations
"Applying GRADE-CERQual to qualitati..." refers background in this paper
...An ever-expanding array of qualitative synthesis methods are available [41, 42]....
[...]
Book•
01 Jan 1992TL;DR: In this article, the authors discuss the relationship between qualitative and quantitative research in case study research, and discuss the importance of case studies in case-study research and the logic of theory-testing in case studies.
Abstract: Part One: Ethnography, Theory and Reality 1. What's wrong with ethnography? The myth of theoretical description 2. Some questions about theory in ethnography and history 3. Ethnoraphy and realism 4. By what criteria should ethnographic research be judged? Part Two: Ethnography, Relevance and Practice 5. The relevance of ethnography 6. Critical theory as a model for ethnography 7. Parts that even ethnography cannot reach: Some reflections on the relationship between research and policy 8. On practitioner ethnography Part Three: Qualitative versus Quantitative Method 9. Deconstructing the quantitative-qualitative divide 10. Keeping the converstion open: the relationship between quantitative and qualitative 11. The logic of theory-testing in case study research 12. So, what are case studies?
1,187 citations
"Applying GRADE-CERQual to qualitati..." refers methods in this paper
...However, in our approach to qualitative evidence synthesis and the development of CERQual, we have adopted the ‘subtle realist’ position [22] which maintains that the existence of phenomena does not depend on our subjective perceptions of them....
[...]
TL;DR: The major problem yet to be resolved is developing usable and communicable systematic approaches to conducting metasynthesis projects that maintain the integrity of individual studies.
Abstract: There has been an accumulation of qualitative studies in recent years, but little cumulation of the understandings gained from them. Qualitative research appears endangered both by efforts to synthesize studies and by the failure to do so. Techniques used have included reciprocal translations of key metaphors and concepts and qualitative and quantitative comparative analyses to produce narrative and theoretical integrations. The major problem yet to be resolved is developing usable and communicable systematic approaches to conducting metasynthesis projects that maintain the integrity of individual studies.
1,010 citations