Many interventions found to be effective in health services research studies fail to translate into meaningful patient care outcomes across multiple contexts. Health services researchers recognize the need to evaluate not only summative outcomes but also formative outcomes to assess the extent to which implementation is effective in a specific setting, prolongs sustainability, and promotes dissemination into other settings. Many implementation theories have been published to help promote effective implementation. However, they overlap considerably in the constructs included in individual theories, and a comparison of theories reveals that each is missing important constructs included in other theories. In addition, terminology and definitions are not consistent across theories. We describe the Consolidated Framework For Implementation Research (CFIR) that offers an overarching typology to promote implementation theory development and verification about what works where and why across multiple contexts. We used a snowball sampling approach to identify published theories that were evaluated to identify constructs based on strength of conceptual or empirical support for influence on implementation, consistency in definitions, alignment with our own findings, and potential for measurement. We combined constructs across published theories that had different labels but were redundant or overlapping in definition, and we parsed apart constructs that conflated underlying concepts. The CFIR is composed of five major domains: intervention characteristics, outer setting, inner setting, characteristics of the individuals involved, and the process of implementation. Eight constructs were identified related to the intervention (e.g., evidence strength and quality), four constructs were identified related to outer setting (e.g., patient needs and resources), 12 constructs were identified related to inner setting (e.g., culture, leadership engagement), five constructs were identified related to individual characteristics, and eight constructs were identified related to process (e.g., plan, evaluate, and reflect). We present explicit definitions for each construct. The CFIR provides a pragmatic structure for approaching complex, interacting, multi-level, and transient states of constructs in the real world by embracing, consolidating, and unifying key constructs from published implementation theories. It can be used to guide formative evaluations and build the implementation knowledge base across multiple studies and settings.

/pdf/fostering-implementation-of-health-services-research-4xghpqk7rh.pdf

Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science

Statistics for Spatial Data.

Background Audit and feedback continues to be widely used as a strategy to improve professional practice. It appears logical that healthcare professionals would be prompted to modify their practice if given feedback that their clinical practice was inconsistent with that of their peers or accepted guidelines. Yet, audit and feedback has not been found to be consistently effective. Objectives To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes. Search strategy We searched the Cochrane Effective Practice and Organisation of Care Group's register up to January 2001. This was supplemented with searches of MEDLINE and reference lists, which did not yield additional relevant studies. Selection criteria Randomised trials of audit and feedback (defined as any summary of clinical performance over a specified period of time) that reported objectively measured professional practice in a healthcare setting or healthcare outcomes. Data collection and analysis Two reviewers independently extracted data and assessed study quality. Quantitative (meta-regression), visual and qualitative analyses were undertaken. Main results We included 85 studies, 48 of which have been added to the previous version of this review. There were 52 comparisons of dichotomous outcomes from 47 trials with over 3500 health professionals that compared audit and feedback to no intervention. The adjusted RDs of non-compliance with desired practice varied from 0.09 (a 9% absolute increase in non-compliance) to 0.71 (a 71% decrease in non-compliance) (median = 0.07, inter-quartile range = 0.02 to 0.11). The one factor that appeared to predict the effectiveness of audit and feedback across studies was baseline non-compliance with recommended practice. Reviewer's conclusions Audit and feedback can be effective in improving professional practice. When it is effective, the effects are generally small to moderate. The absolute effects of audit and feedback are more likely to be larger when baseline adherence to recommended practice is low.

Audit and feedback: effects on professional practice and healthcare outcomes

Executive Summary Introduction Methodology Used to Prepare the Guideline Epidemiology Incidence Etiology Major Epidemiologic Points Pathogenesis Major Points for Pathogenesis Modifiable Risk Factors Intubation and Mechanical Ventilation Aspiration, Body Position, and Enteral Feeding Modulation of Colonization: Oral Antiseptics and Antibiotics Stress Bleeding Prophylaxis, Transfusion, and Glucose Control Major Points and Recommendations for Modifiable Risk Factors Diagnostic Testing Major Points and Recommendations for Diagnosis Diagnostic Strategies and Approaches Clinical Strategy Bacteriologic Strategy Recommended Diagnostic Strategy Major Points and Recommendations for Comparing Diagnostic Strategies Antibiotic Treatment of Hospital-acquired Pneumonia General Approach Initial Empiric Antibiotic Therapy Appropriate Antibiotic Selection and Adequate Dosing Local Instillation and Aerosolized Antibiotics Combination versus Monotherapy Duration of Therapy Major Points and Recommendations for Optimal Antibiotic Therapy Specific Antibiotic Regimens Antibiotic Heterogeneity and Antibiotic Cycling Response to Therapy Modification of Empiric Antibiotic Regimens Defining the Normal Pattern of Resolution Reasons for Deterioration or Nonresolution Evaluation of the Nonresponding Patient Major Points and Recommendations for Assessing Response to Therapy Suggested Performance Indicators

Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia the official statement of the American Thoracic Society and the Infectious Disease Society of America (特集 救急診療ガイドライン) -- (海外のガイドライン)

Objective. —To review the literature relating to the effectiveness of education strategies designed to change physician performance and health care outcomes. Data Sources. —We searched MEDLINE, ERIC, NTIS, the Research and Development Resource Base in Continuing Medical Education, and other relevant data sources from 1975 to 1994, using continuing medical education (CME) and related terms as keywords. We manually searched journals and the bibliographies of other review articles and called on the opinions of recognized experts. Study Selection. —We reviewed studies that met the following criteria: randomized controlled trials of education strategies or interventions that objectively assessed physician performance and/or health care outcomes. These intervention strategies included (alone and in combination) educational materials, formal CME activities, outreach visits such as academic detailing, opinion leaders, patient-mediated strategies, audit with feedback, and reminders. Studies were selected only if more than 50% of the subjects were either practicing physicians or medical residents. Data Extraction. —We extracted the specialty of the physicians targeted by the interventions and the clinical domain and setting of the trial. We also determined the details of the educational intervention, the extent to which needs or barriers to change had been ascertained prior to the intervention, and the main outcome measure(s). Data Synthesis. —We found 99 trials, containing 160 interventions, that met our criteria. Almost two thirds of the interventions (101 of 160) displayed an improvement in at least one major outcome measure: 70% demonstrated a change in physician performance, and 48% of interventions aimed at health care outcomes produced a positive change. Effective change strategies included reminders, patient-mediated interventions, outreach visits, opinion leaders, and multifaceted activities. Audit with feedback and educational materials were less effective, and formal CME conferences or activities, without enabling or practice-reinforcing strategies, had relatively little impact. Conclusion. —Widely used CME delivery methods such as conferences have little direct impact on improving professional practice. More effective methods such as systematic practice-based interventions and outreach visits are seldom used by CME providers. ( JAMA . 1995;274:700-705)

https://jamanetwork.com/HttpHandlers/ArticlePdfHandler.ashx?journal=JAMA&articleId=389532&pdfFileName=jama_274_9_018.pdf

Changing Physician Performance: A Systematic Review of the Effect of Continuing Medical Education Strategies

Background The Institute of Medicine's 2001 report Crossing the Quality Chasm argued for fundamental redesign of the U.S. health care system. Six years later, many health care organizations have embraced the report's goals, but few have succeeded in making the substantial transformations needed to achieve those aims. Purposes This article offers a model for moving organizations from short-term, isolated performance improvements to sustained, reliable, organization-wide, and evidence-based improvements in patient care. Methodology Longitudinal comparative case studies were conducted in 12 health care systems using a mixed-methods evaluation design based on semistructured interviews and document review. Participating health care systems included seven systems funded through the Robert Wood Johnson Foundation's Pursuing Perfection Program and five systems with long-standing commitments to improvement and high-quality care. Findings Five interactive elements appear critical to successful transformation of patient care: (1) Impetus to transform; (2) Leadership commitment to quality; (3) Improvement initiatives that actively engage staff in meaningful problem solving; (4) Alignment to achieve consistency of organization goals with resource allocation and actions at all levels of the organization; and (5) Integration to bridge traditional intra-organizational boundaries among individual components. These elements drive change by affecting the components of the complex health care organization in which they operate: (1) Mission, vision, and strategies that set its direction and priorities; (2) Culture that reflects its informal values and norms; (3) Operational functions and processes that embody the work done in patient care; and (4) Infrastructure such as information technology and human resources that support the delivery of patient care. Transformation occurs over time with iterative changes being sustained and spread across the organization. Practice implications The conceptual model holds promise for guiding health care organizations in their efforts to pursue the Institute of Medicine aims of fundamental system redesign to achieve dramatically improved patient care.

Transformational change in health care systems: an organizational model.

A major national health policy objective is to improve the efficiency of hospital utilization. To evaluate programmatic interventions with this objective, such as the Professional Standards Review Organization program, measures of appropriate use are a fundamental need. This report represents the results of two developmental trials of a new technique, labeled the Appropriateness Evaluation Protocol (AEP), for assessing potentially unnecessary hospital days of care. Twenty-seven objective criteria items related to medical services, nursing/life support services and patient condition factors were incorporated in the protocol. If any one of the criteria was met, the day was deemed "appropriate," and if none was met, the day was deemed "inappropriate" at an acute hospital level of care. A reviewer could override the objective criteria in either direction in reaching a final judgment. Three reviewers, two nurses and one physician each reviewed 200 charts at a teaching hospital. After correcting for a small number of abstracting errors, overall agreement rates between pairs of reviewers ranged from 92 to 94 per cent, levels significant p less than 0.0001. Of all cases judged inappropriate by at least one of the reviewers, specific agreement rates for the reviewer pairs on which days were inappropriate ranged from 73 to 79 per cent. These overall agreement rates and specific agreement rates on days of care judged as inappropriate are higher than those of any previously reported assessment methods. A parallel study of the appropriateness of admissions in these same cases, using purely subjective reviewer judgments, found overall agreement rates averaging 90 per cent, but rates of specific agreement on inappropriate admissions were less than 40 per cent between pairs of reviewers. Along with comparisons to other, more subjective, assessment techniques, this finding suggests that objective criteria are a vital element in developing methodologically sound techniques for assessing appropriate hospital use.

The appropriateness evaluation protocol: a technique for assessing unnecessary days of hospital care.

Policy Points:
Composite measures of health care provider performance aggregate individual performance measures into an overall score, thus providing a useful summary of performance.
Numerous federal, state, and private organizations are adopting composite measures for provider profiling and pay-for-performance programs.
This article makes an important contribution to the literature by highlighting the advantages and disadvantages of different approaches to creating composite measures and also by summarizing key issues related to the use of the various methods.
Composite measures are a useful complement to individual measures when profiling and creating incentives for improvement, but because of the sensitivity of results to the methods used to create composite measures, careful analysis is necessary before they are implemented.



Context
Since the Institute of Medicine's 2001 report Crossing the Quality Chasm, there has been a rapid proliferation of quality measures used in quality-monitoring, provider-profiling, and pay-for-performance (P4P) programs. Although individual performance measures are useful for identifying specific processes and outcomes for improvement and tracking progress, they do not easily provide an accessible overview of performance. Composite measures aggregate individual performance measures into a summary score. By reducing the amount of data that must be processed, they facilitate (1) benchmarking of an organization's performance, encouraging quality improvement initiatives to match performance against high-performing organizations, and (2) profiling and P4P programs based on an organization's overall performance.

Methods
We describe different approaches to creating composite measures, discuss their advantages and disadvantages, and provide examples of their use.

Findings
The major issues in creating composite measures are (1) whether to aggregate measures at the patient level through all-or-none approaches or the facility level, using one of the several possible weighting schemes; (2) when combining measures on different scales, how to rescale measures (using z scores, range percentages, ranks, or 5-star categorizations); and (3) whether to use shrinkage estimators, which increase precision by smoothing rates from smaller facilities but also decrease transparency.

Conclusions
Because provider rankings and rewards under P4P programs may be sensitive to both context and the data, careful analysis is warranted before deciding to implement a particular method. A better understanding of both when and where to use composite measures and the incentives created by composite measures are likely to be important areas of research as the use of composite measures grows.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/1468-0009.12165

Composite Measures of Health Care Provider Performance: A Description of Approaches

Many objectives of the study examined here were to develop a questionnaire useful in eliciting the perceptions of patients regarding their hospital care, to compare the methodologic and substantive differences between a mail survey and a telephone survey of patients done approximately a week post discharge, and to develop methods allowing the questionnaire to be administered by an organization other than the hospital. Two independent surveys were conducted at the same hospital--a telephone survey with a telephone follow-up, and a mail survey with a telephone follow-up. The study demonstrated that an organization external to the hospital can economically conduct a patient satisfaction survey of a representative patient sample while ensuring confidentiality and producing potentially useful results. The mail survey was preferred over the telephone survey due to lower cost, lower chance of biased responses, and complete assurance of confidentiality. The aspects of hospital care most influencing patient satisfaction related to nursing services.

Obtaining information on patient satisfaction with hospital care: mail versus telephone.

Survey response rate is regarded as a key data-quality indicator, yet response rate is not necessarily predictive of nonresponse bias. Our study objective was to use a high-response-rate survey to assess nonresponse bias across successive waves. This survey of healthcare leaders utilized a web-based, self-report format with an initial invitation and four nonrespondent follow-ups. Across five waves, comparisons were made for demographic and facility characteristics, proportion of items completed, and distribution of three question types: factual reports of customized categorical responses; single-item evaluations using five-point Likert scales; and multi-item scales, across four- or five-point Likert scales. The overall response rate was 95 percent (118/124); waves did not differ by demographic and facility characteristics or missing data. Across waves, there were no significant differences between responses to two factual report questions or the single- or multi-item scale measures of attitudes. According to a “what-if” analysis of cumulative results by wave, the same conclusions would have been reached if data collection had been halted at earlier points in time. Precision and statistical power increased as number of respondents accumulated by wave. The high response rate facilitated studying the impact of nonresponse bias by wave. Although high response rates are desirable because of precision and power, as survey fatigue increases, absolute thresholds representing “adequate” response rates may be less realistic. Results from “low” response-rate surveys should be considered on their merits, as they may accurately represent attitudes of the population. Therefore, low response rates should not be cited as reasons to dismiss results as uninformative.

Joseph D. Restuccia

Papers

Transformational change in health care systems: an organizational model.

The appropriateness evaluation protocol: a technique for assessing unnecessary days of hospital care.

Composite Measures of Health Care Provider Performance: A Description of Approaches

Obtaining information on patient satisfaction with hospital care: mail versus telephone.

Response Rates, Nonresponse Bias, and Data QualityResults from a National Survey of Senior Healthcare Leaders