In this commentary, we discuss the science of stakeholder engagement in research. We propose a classification system with definitions to determine where projects lie on the stakeholder engagement continuum. We discuss the key elements of implementation and evaluation of stakeholder engagement in research posing key questions to consider when doing this work. We commend and critique the work of Hamilton et al. in their multilevel stakeholder engagement in a VA implementation trial of evidence-based quality improvement in women's health primary care. We also discuss the need for more work in this area to enhance the science of stakeholder engagement in research.

The science of stakeholder engagement in research: classification, implementation, and evaluation.

Background. Community-based participatory research (CBPR) is increasingly used by community and academic partners to examine health inequities and promote health equity in communities. Despite increasing numbers of CBPR partnerships, there is a lack of consensus in the field regarding what defines partnership success and how to measure factors contributing to success in long-standing CBPR partnerships. Aims. To identify indicators and measures of success in long-standing CBPR partnerships as part of a larger study whose aim is to develop and validate an instrument measuring success across CBPR partnerships. Methods. The Joanna Briggs Institute framework and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guided searches of three databases (PubMed, CINAHL, Scopus) for articles published between 2007 and 2017 and evaluating success in CBPR partnerships existing longer than 4 years. Results. Twenty-six articles met search criteria. We identified 3 key domains and 7 subdomains with 28 underlying indicators of success. Six partnerships developed or used instruments to measure their success; only one included reliability or validity data. Discussion. CBPR partnerships reported numerous intersecting partner, partnership, and outcome indicators important for success. These results, along with data from key informant interviews with community and academic partners and advisement from a national panel of CBPR experts, will inform development of items for an instrument measuring CBPR partnership success. Conclusion. The development of a validated instrument measuring indicators of success will allow long-standing CBPR partnerships to evaluate their work toward achieving health equity and provide a tool for newly forming CBPR partnerships aiming to achieve long-term success.

Success in Long-Standing Community-Based Participatory Research (CBPR) Partnerships: A Scoping Literature Review:

Objectives 
To develop and examine the content and face validity of the Patient Engagement In Research Scale (PEIRS) for assessing the quality of patient engagement in research projects from a patient partner perspective.


Methods 
Our team of researchers and patient partners conducted a mixed qualitative and quantitative study in three phases. Participants were English-speaking adult patients (including informal caregivers, family members, and friends) with varying experiences as partners in research projects in Canada. 1) Questionnaire items were generated following thematic analysis of in-depth interviews and published literature. 2) A three-round e-Delphi survey process via email correspondence was undertaken to refine and select the items for a provisional PEIRS. 3) Two rounds of cognitive interviewing elicited participants’ understanding and opinions of each item and the structure of the PEIRS.


Results 
One hundred and twenty items were generated from 18 interviews and organized across eight themes of meaningful engagement of patients in health research to form an initial questionnaire. The e-Delphi survey and cognitive interviewing each included 12 participants with a range of self-reported diseases, health-related conditions, and use of healthcare services. The e-Delphi survey yielded a 43-item provisional PEIRS. The PEIRS was then reduced to 37 items organized across seven themes after 1) refinement of problems in its instructions and items, and 2) the combining of two themes into one.


Conclusions 
We developed a 37-item self-reported questionnaire that has demonstrated preliminary content and face validity for assessing the quality of patient engagement in research.

/pdf/development-and-pre-testing-of-the-patient-engagement-in-59nj44do15.pdf

Development and pre-testing of the Patient Engagement In Research Scale (PEIRS) to assess the quality of engagement from a patient perspective.

Stakeholder engagement in research has received increasing attention in recent years1, 2 The term “stakeholder engagement” refers to the process of meaningful involvement of those who are engaged in making decisions about programs3 Engaging members of the target population is often key to improving the relevance of the issues studied, the procedures used for study, and the interpretation of outcomes of research studies, health promotion activities, and disease prevention initiatives4, 5, 6 The utility of stakeholder engagement has been well established in the literature,7, 8, 9 but there are few examples of measurement and evaluation of the degree to which stakeholders are engaged in these activities and the impact of engagement on positive outcomes These types of evaluations have been limited in scope, and largely focused on qualitative approaches10, 11, 12, 13, 14 Qualitative methods cannot be easily compared across programs or institutions15 Necessary reliability and validity information describing self‐reported levels of stakeholder engagement are also lacking, and is essential to identifying the impact of engagement on the scientific process and scientific discovery

Systematic Review of Quantitative Measures of Stakeholder Engagement

Policy Points Community-engaged research (CEnR) engenders meaningful academic-community partnerships to improve research quality and health outcomes. CEnR has increasingly been adopted by health care systems, funders, and communities looking for solutions to intractable problems. It has been difficult to systematically measure CEnR's impact, as most evaluations focus on project-specific outcomes. Similarly, partners have struggled with identifying appropriate measures to assess outcomes of interest. To make a case for CEnR's value, we must demonstrate the impacts of CEnR over time. We compiled recent measures and developed an interactive data visualization to facilitate more consistent measurement of CEnR's theoretical domains. Context Community-engaged research (CEnR) aims to engender meaningful academic-community partnerships to increase research quality and impact, improve individual and community health, and build capacity for uptake of evidence-based practices. Given the urgency to solve society's pressing public health problems and increasing competition for funding, it is important to demonstrate CEnR's value. Most evaluations focus on project-specific outcomes, making it difficult to demonstrate CEnR's broader impact. Moreover, it is challenging for partnerships to identify assessments of interest beyond process measures. We conducted a mapping review to help partnerships find and select measures to evaluate CEnR projects and to characterize areas where further development of measures is needed. Methods We searched electronic bibliographic databases using relevant search terms from 2009 to 2018 and scanned CEnR projects to identify unpublished measures. Through review and reduction, we found 69 measures of CEnR's context, process, or outcomes that are potentially generalizable beyond a specific health condition or population. We abstracted data from descriptions of each measure to catalog purpose, aim (context, process, or outcome), and specific domains being measured. Findings We identified 28 measures of the conditions under which CEnR is conducted and factors to support effective academic-community collaboration (context); 43 measures evaluating constructs such as group dynamics and trust (process); and 43 measures of impacts such as benefits and challenges of CEnR participation and system and capacity changes (outcomes). Conclusions We found substantial variation in how academic-community partnerships conceptualize and define even similar domains. Achieving more consistency in how partnerships evaluate key constructs could reduce measurement confusion apparent in the literature. A hybrid approach whereby partnerships discuss common metrics and develop locally important measures can address CEnR's multiple goals. Our accessible data visualization serves as a convenient resource to support partnerships' evaluation goals and may help to build the evidence base for CEnR through the use of common measures across studies.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/1468-0009.12458

Measuring Community‐Engaged Research Contexts, Processes, and Outcomes: A Mapping Review

Although the importance of community engagement in research has been previously established, there are few evidence-based approaches for measuring the level of community engagement in research projects. A quantitative community engagement measure was developed, aligned with 11 engagement principles (EPs) previously established in the literature. The measure has 96 Likert response items; 3-5 quality items and 3-5 quantity items measure each EP. Cronbach's alpha is used to examine the internal consistency of items that measure a single EP. Every EP item group had a Cronbach's alpha > .85, which indicates strong internal consistency for all question groups across both scales (quality and quantity). This information determines the level of community engagement, which can be correlated with other research outcomes.

/pdf/evaluating-community-engagement-in-research-quantitative-4zfp5fhgb5.pdf

Pravleen Bajwa

Papers

Evaluating community engagement in research: quantitative measure development