Patient and public involvement in the design of clinical trials: An overview of systematic reviews
Summary (7 min read)
- The requirements for the planning of Patient (or Personal) and Public Involvement (PPI) in research has increased 1 to encourage research that is ‘with’ or ‘by’ members of the public and patients rather than ‘on,’ ‘to’, ‘about’ or ‘for’ them.
- Many researchers lack PPI 4 training and experience.
- In addition impaired communications between patients, clinicians and researchers are well documented and may obstruct meaningful involvement 5,6 .
- It summarizes what has been found and reported about PPI in clinical trials; identifies the context, methods or processes that facilitate PPI 8 ; collates the perceptions of the influence of PPI on the research process, outcomes.
- And dissemination of results; and promotes the uptake of effective strategies to improve PPI in research and reduce resource costs and that might result from ineffectual PPI.
Why it is important to do this overview
- Research in this field varies in quality, scope, size, and focus, making a systematic overview a practical option 8,9 .
- This enables comparison and critical appraisal of choices made in review selection and can collate, analyze and interpret study results across the separate reviews.
- What can the authors learn from existing systematic reviews about involving the public and patients in the design of clinical trials in terms of: 1. How patients and the public are involved in the design of clinical trials 2.
- What is known about good and bad practice for PPI in the design of clinical trials 3.
- Identify existing systematic reviews that examine PPI in trials.
- Critically appraise these reviews to assess their methodological quality.
- Extract data from these reviews and use these data to describe how, and to what extent, the public and patients have been involved in trials (other than as participants).
- Seek examples of what worked and what did not to identify good practice.
- Identify research gaps in PPI and trials design.
Research for consideration
- Systematic reviews and overviews published in any language that reviewed existing public or patient involvement in clinical trials (other than as participants) were eligible.
- The involvement could include but was not limited to, prioritization of the research question, involvement in the design or conduct of the trial, analysis, presentation of results, or dissemination of findings.
- Reviews of PPI in clinical trials were eligible if they searched a minimum two databases, appraised the included studies, provided summary findings and included a synthesis of the data and the information retrieved10.
Data Sources and Search Strategy
- The following databases were searched from 1995 until December 2015: Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations and Ovid MEDLINE(R), EMBASE, CINAHL, PsycINFO, Science Citation Index, Cochrane Library, Database of Abstracts of Reviews of Effects, PROSPERO, Global Health Library, Health Technology Assessment, The Joanna Briggs Institute EBP Database, McMaster Knowledge Translation and WHOLIS.
- The PRESS checklist 11 was used to ensure inclusion of essential elements in the search strategy.
- Reference lists and search terms of reviews captured by the initial searches were searched for additional reviews and topic experts were contacted.
- Volunteers were invited to co-create the plain language summary, review the paper for readability and work collaboratively to build an infographic to represent the overview.
Screening and Selection of Reviews
- All citations were screened in RAYYAN 12 , a free online tool that allows the use of unlimited volunteers, tracking and blind review.
- To improve screening accuracy, retrieved citations were screened by one author (AP) and then rescreened by her 4 weeks later.
- Full papers were retrieved for articles that appeared eligible or potentially eligible on the basis of their title and abstract, and for a 1% random sample of those judged to be ineligible to check for correct exclusion 13 .
- Reviewers were not blinded to author, institution, or journal.
Full Paper Retrieval
- Full papers were downloaded to a shared folder and de-duplicated in Mendeley 14 , where overview authors could write and share notes, add questions and additional data.
- Papers were categorized as include, exclude or unsure.
- Papers classified as unsure were discussed and agreement at all stages was reached by consensus of three authors.
- Two review authors independently extracted key data for included reviews, using a data extraction form in EPPI reviewer 15 that was piloted on a small sample of reviews.
- Data on public involvement in clinical trials design and preparation was extracted, covering exploration of roles, policy, impact, reporting, interventions, and theoretical frameworks.
- The CASP16 checklists for systematic reviews 17 were used as a preliminary screening tool when assessing systematic reviews for eligibility.
- The first three questions are general and can be used to include or exclude the review.
- The following domains were included when assessing quality: aims, methodology, search quality, recruitment, data collection, data analysis, reflexivity, ethical considerations, findings, and research contributions.
Risk of Bias
- This is based on four components: limitations of methodology, relevance to the research question, coherence and the adequacy of the data presented.
- CerQual enables ratings of “high”, “medium”, “low” and “very low” (although this final rating was not needed because such reviews were not eligible).
- The starting point of ‘high confidence’ reflects that each review finding is a reasonable representation of the question of interest and is downgraded if there are factors that would weaken this assumption 19 .
- After assessing all four components, authors agreed on overall confidence in each review finding and the relevance to their research.
- All measures were pre-specified prior to analysis.
- A strength, weakness, opportunity, and threat (SWOT) framework was used to analyze the findings and organize the data into themes and code them for analysis.
- This made it possible to identify and agree on methods and areas of involvement with positive or negative effect on trial design and to identify research gaps.
- The SWOT approach is used in healthcare research 21,22 to help teams to analyze data individually and then reach consensus on how to present their findings.
Descriptions of Information Presentation Forms
- The excluded reviews summary contains citations and reason for exclusion and is located in results under the heading full-text screening.
- All included reviews contained qualitative elements which meant that the authors need to report their results in a narrative format to describe areas of good and bad practice for PPI in clinical trials and the perceived value of PPI.
- Figure 1 uses a Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram to outline the process of study selection 23 .
- Three additional records were found by searching the reference lists of included reviews, one more was included from the EPPI Reviewer database and one review was identified by an expert in the field.
- Table 1 shows the citation number, first author, year of publication, type of review method, research question focus, number of included studies and funding support type for each included review.
- The total number of studies could not be reported because one review 30 did not report the number of studies reviewed.
- The use and definition of gray literature was variable.
- All academic funding reported was combined with other funding.
Quality Appraisal and Methodological Assessment of Included Reviews
- Conflicts of interest and risk of bias were reported for the included studies32,41,46,49 in four of the included reviews.
- In two reviews this was referred to but not reported by individual study31,49.
- As expected, although some reviews might have been good enough to answer their research question, they were a substandard source of evidence for their research question.
- Likewise, studies seeking impact across research fields 28 contained valuable background information but this was peripheral to the overview aim.
Extent of PPI involvement
- PPI was more frequent in the form of researchers asking members of the public and patients for feedback on the trial design or citizen to citizen interaction such as moderating forums and recruiting participants, rather than in active participation for hands-on research tasks such as study design, ethical review, policy, recruiting, analysis and dissemination.
- PPI impact was reported in 14 reviews using many formats.
- Four reviews 28–30,40 written by authors working together on the question of impact investigated the reporting of impact and have proposed reporting guidelines 61 , however, these reviews were hampered by inconsistent reporting within individual studies.
- Figure 2 shows how PPI was reported across the reviews and the methods of public involvement for various tasks.
- It shows surveys and focus groups were dominant methods of involvement, yet all 27 reviews reported the use of multiple tasks and methods.
Reviewers’ use of PPI in their review
- The authors recorded how review authors reported public involvement in their own reviews to supplement the inconsistent reporting of the numbers of studies or participants involved in tasks.
- This information builds a unique value statement about whether talking about PPI encourages practice.
- Fourteen reviews did not report any PPI in the review and activities were frequently passive.
- The public was updated by review authors and then were invited to advise or comment on the review, rather than engaging the public directly with the data.
- PPI extended to collaborative screening of the literature in three reviews, and analysis and study design roles were largely advisory as recorded in the multiple/other category with seven reviews.
Our PPI for this Systematic Overview
- One volunteer from the Cochrane Task Exchange and three volunteers from Empower-2-Go assisted with screening, data extraction, analysis, synthesis, prioritizing what to report, and editing.
- One lay volunteer and co-author underwent treatment with chemotherapy and radiation for lung cancer and other volunteers completed her tasks.
- Volunteers will help with dissemination planning, conduct and implementation of the overview and are working with us to prepare teaching materials.
- They co-created the plain language summary, suggested improvements for the tables and figures, reviewed the paper for readability and will work collaboratively to build an infographic to represent the overview.
- Shared internal benefits of PPI include knowledge of conditions, interventions and expanding of perspectives.
- Negative stereotypes and power imbalances were lessened through working together and were replaced by mutual respect.
- Researchers were encouraged by volunteers’ resiliency, innovation and tenacity and report newly acquired motivation and inspiration to work towards solutions.
- Patients cite greater confidence, research literacy, hope, trust and a sense of community.
- They felt participating gave their lives purpose, meaning, and identity.
- Consultation with volunteers contributed to salient, pragmatic study designs and raised issues that researchers would not otherwise have anticipated.
- Volunteers improved recruiting, interviews, influenced policy setting, and accessed funding for research.
- In addition, there was community influence where PPI was considered a factor in de-stigmatizing mental health, age issues, disease stereotypes, and cultural challenges.
- The external benefits of PPI were reported from early stages in the design of a clinical trial, including in protocol consultation, setting user-focused research objectives and finalizing research questions; developing questionnaires, interview schedules, and consent processes; planning data analysis, user testing, and implementation and dissemination.
- Progress was noted for research awareness, literacy, transparency, and training materials.
Weaknesses of Public Involvement
- Weaknesses were coded by tensions and barriers.
- Shared tensions and barriers were followed by those specific to volunteers, researchers, and organizations.
Tensions in Public Involvement
- Shared tensions revolved around unclear roles, absent or ill-fitting reporting guidelines, tokenism, exclusion, framework limitations, resource allocation and administrative boundaries.
- Jargon was blamed for exclusion and confusion.
- Tensions were balanced by an overarching desire to carve out a mutually agreed path.
- They worried about inappropriate conclusions from composite outcomes but lacked opportunity to share these concerns and noted they would benefit from research methods training.
- Instances were reported where group dynamics changed and overly aggressive patients and those without respect for rules of confidentiality or data protection harmed the research.
Barriers to Public Involvement
- Shared barriers included those imposed by cultures, values, and power hierarchies.
- Patients reported that involving them too late in a trial meant that the design was already funded and fixed and that the priorities and outcomes were not reversible, leaving them with only user experience to contribute.
- They were vulnerable to negative attitudes or dismissive behavior and felt overloaded when drawn into internal strife.
- Researchers struggled to identify a lead for public involvement, a lack of relevant recruiting networks, difficulties with information about structured, practical methods of involvement, and insufficient time to plan for PPI.
- Organizational or gatekeeper barriers ranged from concerns about data being hijacked by opinion rather being centered by evidence.
- A focus on triangulation of teams and clear direction by senior research team members can reduce assumptions.
- Research methods and PPI training increased parity between researchers and volunteers.
- Using flexible responsive approaches to tasks increased the efficiency and quality of involvement.
- Researchers suggested combining interviews and focus groups to reduce scheduling conflicts and manage costs.
- Reviews suggest engaging volunteers in poststudy reflection.
- These suggestions were developed as a result of findings for good and bad practice in PPI and all the included reviews contributed to this theme.
- The table was developed with feedback from volunteers and review authors.
- The tables are used in informal researcher/volunteer training Amy Price developed for The BMJ 68,69 and for Tabula Rasa an asynchronous online medical support learning network 70 .
- The tables were adapted for use in an interactive workshop at The Cochrane Canada Symposium 2017 71 and as an element of the course structure at the FORCE11 Scholarly Communication Institute (FSCI) 2017 72.
Threats to public involvement
- Themes identified as threats are poor reporting, data contamination, ethical breaches, and bad practice.
- All included reviews contributed to this theme.
- Threats centered around poor reporting and inadequate quality appraisal of studies and the absence of pre-study published protocols.
- Conflicts of interest revolved around patientprovider relationships, industry and undue influence of advocacy organizations.
- Planning, training, and information deficits hindered volunteers’ ability to contribute.
- Unpublished methods were lost opportunities for learning.
- Potential harms of PPI need to be balanced against potential benefits with the caveat that patients and carers might be vulnerable populations.
- Supplemental files are valuable for reviewing and learning from the research as they can contain a level of detail that may not be available in the main paper however if they are inaccessible or not linked to the paper their use is limited at best.
- Research students without support are inappropriate for troubleshooting and managing volunteers.
- Public involvement was reported as beneficial for volunteers, researchers, and systems in a variety of settings, including different stages of trial design, cultures and disease states.
- As patients become research collaborators, provide recruiting testimonials, conduct interviews with participants and exert cultural change through social media declaring all conflicts of interest would be best practice 75 .
- The overlap across reviews in impact appraisal, research prioritization and choice of outcomes may contribute to an overstatement of equality between researchers and citizens.
- Kirkham and colleagues have defined a methodological approach for assessing the uptake of core outcome sets from findings of randomized controlled trials of rheumatoid arthritis listed on using ClinicalTrials.gov.
- This method may also prove useful for tracking the uptake of PPI and research prioritization 86 .
Limitations of this research
- The absence of dedicated funding for this systematic overview limited double screening to a sample of citations.
- Unspecified MESH terms at the time of the search may have compromised search sensitivity and specificity.
- The deficits in standardized language, research methods and reporting of PPI provided challenges for identifying search terms, assessing quality and risk of bias and this impacted their interpretation of data and scope of comparisons for the overview.
- Differences between protocol and review References in the protocol to quantitative methods, effect sizes, meta-analysis, GRADE 88 , and AMSTAR 89 were not relevant to the final systematic overview because all included reviews were reported qualitatively.
- The authors changed the emphasis to “value reported” rather than “impact reported” because the term impact was based on differing cultural assumptions across disciplines.
- PPI was wide-ranging and innovative in the reviews the authors identified.
- Involving lay volunteers for problem-solving provided insights, enhanced research design and served to identify weaknesses and barriers.
- Contingency plans were useful for adapting to disease progression and competing priorities.
- The use of PPI in dissemination planning, design, implementation, and distribution could increase public involvement, contribute to health literacy and expand knowledge for patient values and preferences.
- Research is evaluated externally by peer review.
Did you find this useful? Give us your feedback
Cites background from "Patient and public involvement in t..."
...In PPI, the public become active partners in one or multiple aspects of the research, including generating the research question, grant writing, study design, study conduct, analysis, evaluation, cowriting publications and their dissemination.(2) Involving patients and the public in research can improve the quality, consistency, content, experience and value of health research to end users....
...However, until requirements for PPI reporting are standardised and enforced by journals, funding institutions and sponsors, there is a danger that much research methodology is likely to remain unreported and will contribute to research waste.(2) 10...
...Reporting of PPI is hampered by inconsistent terminology and little consensus on reporting methods.(2) Journals and peer reviewers do not usually request PPI information within manuscripts, so an absence of information about PPI in research papers could be attributable to either a lack of reporting or a lack of PPI activity....
Cites background from "Patient and public involvement in t..."
...Given the importance of patient involvement in research (28), we asked clinical trials groups and the European Federation of Critical Care Nurses associations for contact details of support groups in their countries, but received no suggestions highlighting a need for more work to encourage patient and carer engagement in research....
"Patient and public involvement in t..." refers methods in this paper
...Figure 1 uses a Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) flow diagram to outline the process of study selection.(23) Our search of 11 databases yielded a total of 9433 records....
"Patient and public involvement in t..." refers methods in this paper
...Consistent reporting of the design, conduct, analysis, and interpretation of the PPI in clinical trials could facilitate reproducibility and reduce correctable error.(2,3) However, many researchers lack PPI(4) training and experience....
Related Papers (5)
Frequently Asked Questions (2)
Q1. What contributions have the authors mentioned in the paper "Patient and public involvement in the design of clinical trials: an overview of systematic reviews" ?
Background Funders encourage lay-volunteer inclusion in research but this is not without controversy or resistance, given concerns of role confusion, exploratory methods and limited evidence about what value this brings to research.
Q2. What have the authors stated for future works in "Patient and public involvement in the design of clinical trials: an overview of systematic reviews" ?
Threats to research integrity might be averted through reporting personal conflicts of interest and appraisal of bias in mixed methods or non-quantitative research. The addition of patient reviewers by journals may contribute to health literacy and provide insights for future participatory research practice.