Showing papers in "Implementation Science in 2011"
TL;DR: Interventions and policies to change behaviour can be usefully characterised by means of a BCW comprising: a 'behaviour system' at the hub, encircled by intervention functions and then by policy categories, and a new framework aimed at overcoming their limitations is developed.
Abstract: Improving the design and implementation of evidence-based practice depends on successful behaviour change interventions. This requires an appropriate method for characterising interventions and linking them to an analysis of the targeted behaviour. There exists a plethora of frameworks of behaviour change interventions, but it is not clear how well they serve this purpose. This paper evaluates these frameworks, and develops and evaluates a new framework aimed at overcoming their limitations. A systematic search of electronic databases and consultation with behaviour change experts were used to identify frameworks of behaviour change interventions. These were evaluated according to three criteria: comprehensiveness, coherence, and a clear link to an overarching model of behaviour. A new framework was developed to meet these criteria. The reliability with which it could be applied was examined in two domains of behaviour change: tobacco control and obesity. Nineteen frameworks were identified covering nine intervention functions and seven policy categories that could enable those interventions. None of the frameworks reviewed covered the full range of intervention functions or policies, and only a minority met the criteria of coherence or linkage to a model of behaviour. At the centre of a proposed new framework is a 'behaviour system' involving three essential conditions: capability, opportunity, and motivation (what we term the 'COM-B system'). This forms the hub of a 'behaviour change wheel' (BCW) around which are positioned the nine intervention functions aimed at addressing deficits in one or more of these conditions; around this are placed seven categories of policy that could enable those interventions to occur. The BCW was used reliably to characterise interventions within the English Department of Health's 2010 tobacco control strategy and the National Institute of Health and Clinical Excellence's guidance on reducing obesity. Interventions and policies to change behaviour can be usefully characterised by means of a BCW comprising: a 'behaviour system' at the hub, encircled by intervention functions and then by policy categories. Research is needed to establish how far the BCW can lead to more efficient design of effective interventions.
6,692 citations
TL;DR: This review reinforced conclusions in the previous review with respect to positive relationships between general research utilization and: beliefs and attitudes, and current role and found methodological problems inherent in many of the studies that necessitate their findings be replicated in further research.
Abstract: Interventions that have a better than random chance of increasing nurses' use of research are important to the delivery of quality patient care. However, few reports exist of successful research utilization in nursing interventions. Systematic identification and evaluation of individual characteristics associated with and predicting research utilization may inform the development of research utilization interventions. To update the evidence published in a previous systematic review on individual characteristics influencing research utilization by nurses. As part of a larger systematic review on research utilization instruments, 12 online bibliographic databases were searched. Hand searching of specialized journals and an ancestry search was also conducted. Randomized controlled trials, clinical trials, and observational study designs examining the association between individual characteristics and nurses' use of research were eligible for inclusion. Studies were limited to those published in the English, Danish, Swedish, and Norwegian languages. A vote counting approach to data synthesis was taken. A total of 42,770 titles were identified, of which 501 were retrieved. Of these 501 articles, 45 satisfied our inclusion criteria. Articles assessed research utilization in general (n = 39) or kinds of research utilization (n = 6) using self-report survey measures. Individual nurse characteristics were classified according to six categories: beliefs and attitudes, involvement in research activities, information seeking, education, professional characteristics, and socio-demographic/socio-economic characteristics. A seventh category, critical thinking, emerged in studies examining kinds of research utilization. Positive relationships, at statistically significant levels, for general research utilization were found in four categories: beliefs and attitudes, information seeking, education, and professional characteristics. The only characteristic assessed in a sufficient number of studies and with consistent findings for the kinds of research utilization was attitude towards research; this characteristic had a positive association with instrumental and overall research utilization. This review reinforced conclusions in the previous review with respect to positive relationships between general research utilization and: beliefs and attitudes, and current role. Furthermore, attending conferences/in-services, having a graduate degree in nursing, working in a specialty area, and job satisfaction were also identified as individual characteristics important to research utilization. While these findings hold promise as potential targets of future research utilization interventions, there were methodological problems inherent in many of the studies that necessitate their findings be replicated in further research using more robust study designs and multivariate assessment methods.
728 citations
TL;DR: Numerous opportunities were identified by which guidelines could be modified to support various types of decision making by different users, and new governance structures may be required to accommodate development of guidelines with these features.
Abstract: Background: Guidelines continue to be underutilized, and a variety of strategies to improve their use have been suboptimal. Modifying guideline features represents an alternative, but untested way to promote their use. The purpose of this study was to identify and define features that facilitate guideline use, and examine whether and how they are included in current guidelines. Methods: A guideline implementability framework was developed by reviewing the implementation science literature. We then examined whether guidelines included these, or additional implementability elements. Data were extracted from publicly available high quality guidelines reflecting primary and institutional care, reviewed independently by two individuals, who through discussion resolved conflicts, then by the research team. Results: The final implementability framework included 22 elements organized in the domains of adaptability, usability, validity, applicability, communicability, accommodation, implementation, and evaluation. Data were extracted from 20 guidelines on the management of diabetes, hypertension, leg ulcer, and heart failure. Most contained a large volume of graded, narrative evidence, and tables featuring complementary clinical information. Few contained additional features that could improve guideline use. These included alternate versions for different users and purposes, summaries of evidence and recommendations, information to facilitate interaction with and involvement of patients, details of resource implications, and instructions on how to locally promote and monitor guideline use. There were no consistent trends by guideline topic. Conclusions: Numerous opportunities were identified by which guidelines could be modified to support various types of decision making by different users. New governance structures may be required to accommodate development of guidelines with these features. Further research is needed to validate the proposed framework of guideline implementability, develop methods for preparing this information, and evaluate how inclusion of this information influences guideline use.
396 citations
TL;DR: Researchers using PARIHS for targeted evidence-based practice (EBP) implementations with a strong task-orientation are invited to use this Guide as a companion and to apply the revised framework prospectively and comprehensively.
Abstract: Based on a critical synthesis of literature on use of the Promoting Action on Research Implementation in Health Services (PARIHS) framework, revisions and a companion Guide were developed by a group of researchers independent of the original PARIHS team. The purpose of the Guide is to enhance and optimize efforts of researchers using PARIHS in implementation trials and evaluations. Authors used a planned, structured process to organize and synthesize critiques, discussions, and potential recommendations for refinements of the PARIHS framework arising from a systematic review. Using a templated form, each author independently recorded key components for each reviewed paper; that is, study definitions, perceived strengths/limitations of PARIHS, other observations regarding key issues and recommendations regarding needed refinements. After reaching consensus on these key components, the authors summarized the information and developed the Guide. A number of revisions, perceived as consistent with the PARIHS framework's general nature and intent, are proposed. The related Guide is composed of a set of reference tools, provided in Additional files. Its core content is built upon the basic elements of PARIHS and current implementation science. We invite researchers using PARIHS for targeted evidence-based practice (EBP) implementations with a strong task-orientation to use this Guide as a companion and to apply the revised framework prospectively and comprehensively. Researchers also are encouraged to evaluate its use relative to perceived strengths and issues. Such evaluations and critical reflections regarding PARIHS and our Guide could thereby promote the framework's continued evolution.
347 citations
TL;DR: The development of a reliable method of specifying intervention components aimed at changing behaviour will strengthen the scientific basis for developing, evaluating, and reporting complex interventions and improve the precision of evidence syntheses of effectiveness, thus enhancing secondary, as well as primary, research.
Abstract: Background: The importance of behaviour change in improving health is illustrated by the increasing investment by funding bodies in the development and evaluation of complex interventions to change population, patient, and practitioner behaviours. The development of effective interventions is hampered by the absence of a nomenclature to specify and report their content. This limits the possibility of replicating effective interventions, synthesising evidence, and understanding the causal mechanisms underlying behaviour change. In contrast, biomedical interventions are precisely specified (e.g., the pharmacological ‘ingredients’ of prescribed drugs, their dose and frequency of administration). For most complex interventions, the precise ‘ingredients’ are unknown; descriptions (e. g., ‘behavioural counseling’) can mean different things to different researchers or implementers. The lack of a method for specifying complex interventions undermines the precision of evidence syntheses of effectiveness, posing a problem for secondary, as well as primary, research. We aim to develop a reliable method of specifying intervention components (’techniques’) aimed at changing behaviour. Methods/Design: The research will be conducted in three phases. The first phase will develop the nomenclature. We will refine a preliminary list of techniques and definitions. Using a formal consensus method, experts will then define the key attributes of each technique and how it relates to, and differs from, others. They will evaluate the techniques and their definitions until they achieve an agreed-upon list of clearly defined, nonredundant techniques. The second phase will test the nomenclature. Trained experts (primary researchers and systematic reviewers), equipped with a coding manual and guidance, will use the nomenclature to code published descriptions of complex interventions. Reliability between experts, over time, and across types of users will be assessed. We will assess whether using the nomenclature to write intervention descriptions enhances the clarity and replicability of interventions. The third phase will develop a web-based users’ resource of clearly specified and nonredundant techniques, which will aid the scientific understanding of, and development of, effective complex interventions. Dissemination throughout the project will be through stakeholder meetings, targeted multidisciplinary workshops, conference presentation, journal publication, and publication in an interactive webbased platform (a Wiki). Discussion: The development of a reliable method of specifying intervention components aimed at changing behaviour will strengthen the scientific basis for developing, evaluating, and reporting complex interventions. It will improve the precision of evidence syntheses of effectiveness, thus enhancing secondary, as well as primary, research.
261 citations
TL;DR: NPT can be used to explain observed variations in implementation processes, and may be useful in drawing planners' attention to potential problems with a view to addressing them during implementation planning.
Abstract: The use of information and communication technologies in healthcare is seen as essential for high quality and cost-effective healthcare. However, implementation of e-health initiatives has often been problematic, with many failing to demonstrate predicted benefits. This study aimed to explore and understand the experiences of implementers -- the senior managers and other staff charged with implementing e-health initiatives and their assessment of factors which promote or inhibit the successful implementation, embedding, and integration of e-health initiatives. We used a case study methodology, using semi-structured interviews with implementers for data collection. Case studies were selected to provide a range of healthcare contexts (primary, secondary, community care), e-health initiatives, and degrees of normalization. The initiatives studied were Picture Archiving and Communication System (PACS) in secondary care, a Community Nurse Information System (CNIS) in community care, and Choose and Book (CB these differences were well explained by collective action components of NPT. New technology was most likely to 'normalize' where implementers perceived that it had a positive impact on interactions between professionals and patients and between different professional groups, and fit well with the organisational goals and skill sets of existing staff. However, where implementers perceived problems in one or more of these areas, they also perceived a lower level of normalization. Implementers had rich understandings of barriers and facilitators to successful implementation of e-health initiatives, and their views should continue to be sought in future research. NPT can be used to explain observed variations in implementation processes, and may be useful in drawing planners' attention to potential problems with a view to addressing them during implementation planning.
253 citations
TL;DR: The evidence of CCDSS effectiveness is limited, especially with respect to the small number and size of studies measuring patient outcomes, so policy makers, healthcare administrators, and practitioners should be aware of this limited evidence.
Abstract: The use of computerized clinical decision support systems (CCDSSs) may improve chronic disease management, which requires recurrent visits to multiple health professionals, ongoing disease and treatment monitoring, and patient behavior modification. The objective of this review was to determine if CCDSSs improve the processes of chronic care (such as diagnosis, treatment, and monitoring of disease) and associated patient outcomes (such as effects on biomarkers and clinical exacerbations). We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews database, Inspec, and reference lists for potentially eligible articles published up to January 2010. We included randomized controlled trials that compared the use of CCDSSs to usual practice or non-CCDSS controls. Trials were eligible if at least one component of the CCDSS was designed to support chronic disease management. We considered studies 'positive' if they showed a statistically significant improvement in at least 50% of relevant outcomes. Of 55 included trials, 87% (n = 48) measured system impact on the process of care and 52% (n = 25) of those demonstrated statistically significant improvements. Sixty-five percent (36/55) of trials measured impact on, typically, non-major (surrogate) patient outcomes, and 31% (n = 11) of those demonstrated benefits. Factors of interest to decision makers, such as cost, user satisfaction, system interface and feature sets, unique design and deployment characteristics, and effects on user workflow were rarely investigated or reported. A small majority (just over half) of CCDSSs improved care processes in chronic disease management and some improved patient health. Policy makers, healthcare administrators, and practitioners should be aware that the evidence of CCDSS effectiveness is limited, especially with respect to the small number and size of studies measuring patient outcomes.
220 citations
TL;DR: The meaning of implementation climate is clarified, several measurement issues are discussed, and guidelines for empirical study are proposed.
Abstract: Background: Climate has a long history in organizational studies, but few theoretical models integrate the complex effects of climate during innovation implementation. In 1996, a theoretical model was proposed that organizations could develop a positive climate for implementation by making use of various policies and practices that promote organizational members’ means, motives, and opportunities for innovation use. The model proposes that implementation climate–or the extent to which organizational members perceive that innovation use is expected, supported, and rewarded–is positively associated with implementation effectiveness. The implementation climate construct holds significant promise for advancing scientific knowledge about the organizational determinants of innovation implementation. However, the construct has not received sufficient scholarly attention, despite numerous citations in the scientific literature. In this article, we clarify the meaning of implementation climate, discuss several measurement issues, and propose guidelines for empirical study. Discussion: Implementation climate differs from constructs such as organizational climate, culture, or context in two important respects: first, it has a strategic focus (implementation), and second, it is innovation-specific. Measuring implementation climate is challenging because the construct operates at the organizational level, but requires the collection of multi-dimensional perceptual data from many expected innovation users within an organization. In order to avoid problems with construct validity, assessments of within-group agreement of implementation climate measures must be carefully considered. Implementation climate implies a high degree of within-group agreement in climate perceptions. However, researchers might find it useful to distinguish implementation climate level (the average of implementation climate perceptions) from implementation climate strength (the variability of implementation climate perceptions). It is important to recognize that the implementation climate construct applies most readily to innovations that require collective, coordinated behavior change by many organizational members both for successful implementation and for realization of anticipated benefits. For innovations that do not possess these attributes, individual-level theories of behavior change could be more useful in explaining implementation effectiveness. Summary: This construct has considerable value in implementation science, however, further debate and development is necessary to refine and distinguish the construct for empirical use.
185 citations
TL;DR: Current available evidence does not identify any effective, generalisable strategies to change organisational culture and healthcare organisations considering implementing interventions aimed at changing culture should seriously consider conducting an evaluation to strengthen the evidence about this topic.
Abstract: Organisational culture is an anthropological metaphor used to inform research and consultancy and to explain organisational environments. In recent years, increasing emphasis has been placed on the need to change organisational culture in order to improve healthcare performance. However, the precise function of organisational culture in healthcare policy often remains underspecified and the desirability and feasibility of strategies to be adopted have been called into question. The objective of this review was to determine the effectiveness of strategies to change organisational culture in order to improve healthcare performance. We searched the following electronic databases: The Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, CINAHL, Sociological Abstracts, Web of Knowledge, PsycINFO, Business and Management, EThOS, Index to Theses, Intute, HMIC, SIGLE, and Scopus until October 2009. The Database of Abstracts of Reviews of Effectiveness (DARE) was searched for related reviews. We also searched the reference lists of all papers and relevant reviews identified, and we contacted experts in the field for advice on further potential studies. We considered randomised controlled trials (RCTs) or well designed quasi-experimental studies (controlled clinical trials (CCTs), controlled before and after studies (CBAs), and interrupted time series (ITS) analyses). Studies could be set in any type of healthcare organisation in which strategies to change organisational culture in order to improve healthcare performance were applied. Our main outcomes were objective measures of professional performance and patient outcome. The search strategy yielded 4,239 records. After the full text assessment, two CBA studies were included in the review. They both assessed the impact of interventions aimed at changing organisational culture, but one evaluated the impact on work-related and personal outcomes while the other measured clinical outcomes. Both were at high risk of bias. Both reported positive results. Current available evidence does not identify any effective, generalisable strategies to change organisational culture. Healthcare organisations considering implementing interventions aimed at changing culture should seriously consider conducting an evaluation (using a robust design, e.g., ITS) to strengthen the evidence about this topic.
175 citations
TL;DR: The Stages of Implementation Completion (SIC) is an observation-based measure used to track the time to achievement of key implementation milestones in an EBP being implemented in 51 counties in 53 sites (two counties have two sites) in two states in the United States.
Abstract: Background
An increasingly large body of research is focused on designing and testing strategies to improve knowledge about how to embed evidence-based programs (EBP) into community settings. Development of strategies for overcoming barriers and increasing the effectiveness and pace of implementation is a high priority. Yet, there are few research tools that measure the implementation process itself. The Stages of Implementation Completion (SIC) is an observation-based measure that is used to track the time to achievement of key implementation milestones in an EBP being implemented in 51 counties in 53 sites (two counties have two sites) in two states in the United States.
169 citations
TL;DR: Evidence supports the effectiveness of CCDSSs for screening and treatment of dyslipidaemia in primary care with less consistent evidence for CCDSs used in screening for cancer and mental health-related conditions, vaccinations, and other preventive care.
Abstract: Computerized clinical decision support systems (CCDSSs) are claimed to improve processes and outcomes of primary preventive care (PPC), but their effects, safety, and acceptance must be confirmed. We updated our previous systematic reviews of CCDSSs and integrated a knowledge translation approach in the process. The objective was to review randomized controlled trials (RCTs) assessing the effects of CCDSSs for PPC on process of care, patient outcomes, harms, and costs. We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews Database, Inspec, and other databases, as well as reference lists through January 2010. We contacted authors to confirm data or provide additional information. We included RCTs that assessed the effect of a CCDSS for PPC on process of care and patient outcomes compared to care provided without a CCDSS. A study was considered to have a positive effect (i.e., CCDSS showed improvement) if at least 50% of the relevant study outcomes were statistically significantly positive. We added 17 new RCTs to our 2005 review for a total of 41 studies. RCT quality improved over time. CCDSSs improved process of care in 25 of 40 (63%) RCTs. Cumulative scientifically strong evidence supports the effectiveness of CCDSSs for screening and management of dyslipidaemia in primary care. There is mixed evidence for effectiveness in screening for cancer and mental health conditions, multiple preventive care activities, vaccination, and other preventive care interventions. Fourteen (34%) trials assessed patient outcomes, and four (29%) reported improvements with the CCDSS. Most trials were not powered to evaluate patient-important outcomes. CCDSS costs and adverse events were reported in only six (15%) and two (5%) trials, respectively. Information on study duration was often missing, limiting our ability to assess sustainability of CCDSS effects. Evidence supports the effectiveness of CCDSSs for screening and treatment of dyslipidaemia in primary care with less consistent evidence for CCDSSs used in screening for cancer and mental health-related conditions, vaccinations, and other preventive care. CCDSS effects on patient outcomes, safety, costs of care, and provider satisfaction remain poorly supported.
TL;DR: The TICD project will provide much needed evidence on the advantages and disadvantages of different methods of identifying important determinants of practice and selecting implementation strategies that take account of those and provide five rigorous evaluations of tailored implementation interventions for five different chronic conditions.
Abstract: Background: The assumption underlying tailoring is that implementation interventions are most helpful if these effectively address the most important determinants of practice for improvement in the targeted setting. The aim of the Tailored Implementation For Chronic Diseases (TICD) project is to develop valid and efficient methods of tailoring implementation interventions to determinants of practice for knowledge implementation in chronic illness care. Methods: The TICD project has organized the planned empirical research in three work packages that follow the three main steps of tailoring: identification of determinants of healthcare practice, matching implementation interventions to identified determinants of practice, and applying and assessing the tailored implementation interventions. These three key steps of tailored implementation will be applied to targeted chronic conditions in five different healthcare systems: cardiovascular disease in the Netherlands, obesity in England, depression in Norway, chronic obstructive pulmonary disease in Poland, and multimorbidity in Germany. The design and interpretation of empirical research will be informed by systematic reviews of previous research on tailoring implementation interventions. Discussion: The TICD project will provide much needed evidence on the advantages and disadvantages of different methods of identifying important determinants of practice and selecting implementation strategies that take account of those. It will also provide five rigorous evaluations of tailored implementation interventions for five different chronic conditions.
TL;DR: A case study using the Grading of Recommendations Applicability, Development and Evaluation approach to develop a clinical practice guideline for the diagnosis of Cow Milk Allergy with the World Allergy Organization provides useful guidance for guideline developers and clinicians about what they ought to demand from clinical practice guidelines to facilitate implementation and strengthen confidence in recommendations about diagnostic tests.
Abstract: Accurate diagnosis is a fundamental aspect of appropriate healthcare. However, clinicians need guidance when implementing diagnostic tests given the number of tests available and resource constraints in healthcare. Practitioners of health often feel compelled to implement recommendations in guidelines, including recommendations about the use of diagnostic tests. However, the understanding about diagnostic tests by guideline panels and the methodology for developing recommendations is far from completely explored. Therefore, we evaluated the factors that guideline developers and users need to consider for the development of implementable recommendations about diagnostic tests. Using a critical analysis of the process, we present the results of a case study using the Grading of Recommendations Applicability, Development and Evaluation (GRADE) approach to develop a clinical practice guideline for the diagnosis of Cow Milk Allergy with the World Allergy Organization. To ensure that guideline panels can develop informed recommendations about diagnostic tests, it appears that more emphasis needs to be placed on group processes, including question formulation, defining patient-important outcomes for diagnostic tests, and summarizing evidence. Explicit consideration of concepts of diagnosis from evidence-based medicine, such as pre-test probability and treatment threshold, is required to facilitate the work of a guideline panel and to formulate implementable recommendations. This case study provides useful guidance for guideline developers and clinicians about what they ought to demand from clinical practice guidelines to facilitate implementation and strengthen confidence in recommendations about diagnostic tests. Applying a structured framework like the GRADE approach with its requirement for transparency in the description of the evidence and factors that influence recommendations facilitates laying out the process and decision factors that are required for the development, interpretation, and implementation of recommendations about diagnostic tests.
TL;DR: Within a programme of research and implementation, social networks are viewed as being centrally involved in the mobilisation and deployment of resources in the management of a chronic condition, forming the basis of a novel approach to understanding, designing, and implementing new forms of self-management support.
Abstract: Background: Increasing the effective targeting and promotion of self-care support for long-term conditions requires more of a focus on patient contexts and networks. The aim of this paper is to describe how within a programme of research and implementation, social networks are viewed as being centrally involved in the mobilisation and deployment of resources in the management of a chronic condition. This forms the basis of a novel approach to understanding, designing, and implementing new forms of self-management support. Methods: Drawing on evidence syntheses about social networks and capital and the role of information in selfmanagement, we build on four conceptual approaches to inform the design of our research on the implementation of self-care support for people with long-term conditions. Our approach takes into consideration the form and content of social networks, notions of chronic illness work, normalisation process theory (NPT), and the whole systems informing self-management engagement (WISE) approach to self-care support. Discussion: The translation and implementation of a self-care agenda in contemporary health and social context needs to acknowledge and incorporate the resources and networks operating in patients’ domestic and social environments and everyday lives. The latter compliments the focus on healthcare settings for developing and delivering self-care support by viewing communities and networks, as well as people suffering from long-term conditions, as a key means of support for managing long-term conditions. By focusing on patient work and socialnetwork provision, our aim is to open up a second frontier in implementation research, to translate knowledge into better chronic illness management, and to shift the emphasis towards support that takes place outside formal health services.
TL;DR: According to this review, nurses' reported use of research is moderate-high and has remained relatively consistent over time until the early 2000's, which may paint an overly optimistic picture of the extent to which nurses use research in their practice given the methodological problems inherent in the majority of studies.
Abstract: Background: In the past forty years, many gains have been made in our understanding of the concept of research utilization. While numerous studies exist on professional nurses’ use of research in practice, no attempt has been made to systematically evaluate and synthesize this body of literature with respect to the extent to which nurses use research in their clinical practice. The objective of this study was to systematically identify and analyze the available evidence related to the extent to which nurses use research findings in practice. Methods: This study was a systematic review of published and grey literature. The search strategy included 13 online bibliographic databases: Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, MEDLINE, CINAHL, EMBASE, HAPI, Web of Science, SCOPUS, OCLC Papers First, OCLC WorldCat, ABI Inform, Sociological Abstracts, and Dissertation Abstracts. The inclusion criteria consisted of primary research reports that assess professional nurses’ use of research in practice, written in the English or Scandinavian languages. Extent of research use was determined by assigning research use scores reported in each article to one of four quartiles: low, moderate-low, moderate-high, or high. Results: Following removal of duplicate citations, a total of 12,418 titles were identified through database searches, of which 133 articles were retrieved. Of the articles retrieved, 55 satisfied the inclusion criteria. The 55 final reports included cross-sectional/survey (n = 51) and quasi-experimental (n = 4) designs. A sensitivity analysis, comparing findings from all reports with those rated moderate (moderate-weak and moderate-strong) and strong quality, did not show significant differences. In a majority of the articles identified (n = 38, 69%), nurses reported moderate-high research use. Conclusions: According to this review, nurses’ reported use of research is moderate-high and has remained relatively consistent over time until the early 2000’s. This finding, however, may paint an overly optimistic picture of the extent to which nurses use research in their practice given the methodological problems inherent in the majority of studies. There is a clear need for the development of standard measures of research use and robust well-designed studies examining nurses’ use of research and its impact on patient outcomes. The relatively unchanged self-reports of moderate-high research use by nurses is troubling given that over 40 years have elapsed since the first studies in this review were conducted and the increasing emphasis in the past 15 years on evidence-based practice. More troubling is the absence of studies in which attempts are made to assess the effects of varying levels of research use on patient outcomes.
TL;DR: Lack of clear patient benefit and lack of data on harms and costs preclude a recommendation to adopt CCDSSs for drug therapy management.
Abstract: Computerized clinical decision support systems (CCDSSs) for drug therapy management are designed to promote safe and effective medication use. Evidence documenting the effectiveness of CCDSSs for improving drug therapy is necessary for informed adoption decisions. The objective of this review was to systematically review randomized controlled trials assessing the effects of CCDSSs for drug therapy management on process of care and patient outcomes. We also sought to identify system and study characteristics that predicted benefit. We conducted a decision-maker-researcher partnership systematic review. We updated our earlier reviews (1998, 2005) by searching MEDLINE, EMBASE, EBM Reviews, Inspec, and other databases, and consulting reference lists through January 2010. Authors of 82% of included studies confirmed or supplemented extracted data. We included only randomized controlled trials that evaluated the effect on process of care or patient outcomes of a CCDSS for drug therapy management compared to care provided without a CCDSS. A study was considered to have a positive effect (i.e., CCDSS showed improvement) if at least 50% of the relevant study outcomes were statistically significantly positive. Sixty-five studies met our inclusion criteria, including 41 new studies since our previous review. Methodological quality was generally high and unchanged with time. CCDSSs improved process of care performance in 37 of the 59 studies assessing this type of outcome (64%, 57% of all studies). Twenty-nine trials assessed patient outcomes, of which six trials (21%, 9% of all trials) reported improvements. CCDSSs inconsistently improved process of care measures and seldomly improved patient outcomes. Lack of clear patient benefit and lack of data on harms and costs preclude a recommendation to adopt CCDSSs for drug therapy management.
TL;DR: Some CCDSSs can modify practitioner test-ordering behavior, and to better inform development and implementation efforts, studies should describe in more detail potentially important factors such as system design, user interface, local context, implementation strategy, and evaluate impact on user satisfaction and workflow, costs, and unintended consequences.
Abstract: Underuse and overuse of diagnostic tests have important implications for health outcomes and costs. Decision support technology purports to optimize the use of diagnostic tests in clinical practice. The objective of this review was to assess whether computerized clinical decision support systems (CCDSSs) are effective at improving ordering of tests for diagnosis, monitoring of disease, or monitoring of treatment. The outcome of interest was effect on the diagnostic test-ordering behavior of practitioners. We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews database, Inspec, and reference lists for eligible articles published up to January 2010. We included randomized controlled trials comparing the use of CCDSSs to usual practice or non-CCDSS controls in clinical care settings. Trials were eligible if at least one component of the CCDSS gave suggestions for ordering or performing a diagnostic procedure. We considered studies 'positive' if they showed a statistically significant improvement in at least 50% of test ordering outcomes. Thirty-five studies were identified, with significantly higher methodological quality in those published after the year 2000 (p = 0.002). Thirty-three trials reported evaluable data on diagnostic test ordering, and 55% (18/33) of CCDSSs improved testing behavior overall, including 83% (5/6) for diagnosis, 63% (5/8) for treatment monitoring, 35% (6/17) for disease monitoring, and 100% (3/3) for other purposes. Four of the systems explicitly attempted to reduce test ordering rates and all succeeded. Factors of particular interest to decision makers include costs, user satisfaction, and impact on workflow but were rarely investigated or reported. Some CCDSSs can modify practitioner test-ordering behavior. To better inform development and implementation efforts, studies should describe in more detail potentially important factors such as system design, user interface, local context, implementation strategy, and evaluate impact on user satisfaction and workflow, costs, and unintended consequences.
TL;DR: An externally funded evaluation that focuses on implementation mechanisms and processes within three CLAHRCs to capture the processes and impacts of collaborative approaches for implementing research and should contribute to the evidence base about an increasingly popular, but poorly understood approach to knowledge translation.
Abstract: Background: The English National Health Service has made a major investment in nine partnerships between higher education institutions and local health services called Collaborations for Leadership in Applied Health Research and Care (CLAHRC). They have been funded to increase capacity and capability to produce and implement research through sustained interactions between academics and health services. CLAHRCs provide a natural ‘test bed’ for exploring questions about research implementation within a partnership model of delivery. This protocol describes an externally funded evaluation that focuses on implementation mechanisms and processes within three CLAHRCs. It seeks to uncover what works, for whom, how, and in what circumstances. Design and methods: This study is a longitudinal three-phase, multi-method realistic evaluation, which deliberately aims to explore the boundaries around knowledge use in context. The evaluation funder wishes to see it conducted for the process of learning, not for judging performance. The study is underpinned by a conceptual framework that combines the Promoting Action on Research Implementation in Health Services and Knowledge to Action frameworks to reflect the complexities of implementation. Three participating CLARHCS will provide indepth comparative case studies of research implementation using multiple data collection methods including interviews, observation, documents, and publicly available data to test and refine hypotheses over four rounds of data collection. We will test the wider applicability of emerging findings with a wider community using an interpretative forum. Discussion: The idea that collaboration between academics and services might lead to more applicable health research that is actually used in practice is theoretically and intuitively appealing; however the evidence for it is limited. Our evaluation is designed to capture the processes and impacts of collaborative approaches for implementing research, and therefore should contribute to the evidence base about an increasingly popular (e.g., Mode two, integrated knowledge transfer, interactive research), but poorly understood approach to knowledge translation. Additionally we hope to develop approaches for evaluating implementation processes and impacts particularly with respect to integrated stakeholder involvement.
TL;DR: This paper conducted a systematic review to evaluate interventions designed to increase the rate of breast, cervical, and colorectal cancer (CRC) screening, including client reminders, client incentives, mass media, small media, group education, one-on-one education, reduction in structural barriers, reducing in out-of-pocket costs, provider assessment and feedback interventions, and provider incentives.
Abstract: Appropriate screening may reduce the mortality and morbidity of colorectal, breast, and cervical cancers. However, effective implementation strategies are warranted if the full benefits of screening are to be realized. As part of a larger agenda to create an implementation guideline, we conducted a systematic review to evaluate interventions designed to increase the rate of breast, cervical, and colorectal cancer (CRC) screening. The interventions considered were: client reminders, client incentives, mass media, small media, group education, one-on-one education, reduction in structural barriers, reduction in out-of-pocket costs, provider assessment and feedback interventions, and provider incentives. Our primary outcome, screening completion, was calculated as the overall median post-intervention absolute percentage point (PP) change in completed screening tests. Our first step was to conduct an iterative scoping review in the research area. This yielded three relevant high-quality systematic reviews. Serving as our evidentiary foundation, we conducted a formal update. Randomized controlled trials and cluster randomized controlled trials, published between 2004 and 2010, were searched in MEDLINE, EMBASE and PSYCHinfo. The update yielded 66 studies new eligible studies with 74 comparisons. The new studies ranged considerably in quality. Client reminders, small media, and provider audit and feedback appear to be effective interventions to increase the uptake of screening for three cancers. One-on-one education and reduction of structural barriers also appears effective, but their roles with CRC and cervical screening, respectively, are less established. More study is required to assess client incentives, mass media, group education, reduction of out-of-pocket costs, and provider incentive interventions. The new evidence generally aligns with the evidence and conclusions from the original systematic reviews. This review served as the evidentiary foundation for an implementation guideline. Poor reporting, lack of precision and consistency in defining operational elements, and insufficient consideration of context and differences among populations are areas for additional research.
TL;DR: This study further enhances the value of realist evaluations by incorporating a social network analysis component to quantify the structural context associated with CoPs.
Abstract: Healthcare organisations seeking to manage knowledge and improve organisational performance are increasingly investing in communities of practice (CoPs). Such investments are being made in the absence of empirical evidence demonstrating the impact of CoPs in improving the delivery of healthcare. A realist evaluation is proposed to address this knowledge gap. Underpinned by the principle that outcomes are determined by the context in which an intervention is implemented, a realist evaluation is well suited to understand the role of CoPs in improving healthcare practice. By applying a realist approach, this study will explore the following questions: What outcomes do CoPs achieve in healthcare? Do these outcomes translate into improved practice in healthcare? What are the contexts and mechanisms by which CoPs improve healthcare? The realist evaluation will be conducted by developing, testing, and refining theories on how, why, and when CoPs improve healthcare practice. When collecting data, context will be defined as the setting in which the CoP operates; mechanisms will be the factors and resources that the community offers to influence a change in behaviour or action; and outcomes will be defined as a change in behaviour or work practice that occurs as a result of accessing resources provided by the CoP. Realist evaluation is being used increasingly to study social interventions where context plays an important role in determining outcomes. This study further enhances the value of realist evaluations by incorporating a social network analysis component to quantify the structural context associated with CoPs. By identifying key mechanisms and contexts that optimise the effectiveness of CoPs, this study will contribute to creating a framework that will guide future establishment and evaluation of CoPs in healthcare.
TL;DR: Tools to achieve conceptual clarity on the PR field were developed, as a beneficial precursor to the theoretically-driven synthesis using realist methods, to address challenges to PR assessment.
Abstract: Background: Participatory Research (PR) entails the co-governance of research by academic researchers and endusers. End-users are those who are affected by issues under study (e.g., community groups or populations affected by illness), or those positioned to act on the knowledge generated by research (e.g., clinicians, community leaders, health managers, patients, and policy makers). Systematic reviews assessing the generalizable benefits of PR must address: the diversity of research topics, methods, and intervention designs that involve a PR approach; varying degrees of end-user involvement in research co-governance, both within and between projects; and the complexity of outcomes arising from long-term partnerships. Methods: We addressed the above mentioned challenges by adapting realist review methodology to PR assessment, specifically by developing inductively-driven identification, selection, appraisal, and synthesis procedures. This approach allowed us to address the non-uniformity and complexity of the PR literature. Each stage of the review involved two independent reviewers and followed a reproducible, systematic coding and retention procedure. Retained studies were completed participatory health interventions, demonstrated high levels of participation by non-academic stakeholders (i.e., excluding studies in which end-users were not involved in cogoverning throughout the stages of research) and contained detailed descriptions of the participatory process and context. Retained sets are being mapped and analyzed using realist review methods. Results: The librarian-guided search string yielded 7,167 citations. A total of 594 citations were retained after the identification process. Eighty-three papers remained after selection. Principle Investigators (PIs) were contacted to solicit all companion papers. Twenty-three sets of papers (23 PR studies), comprising 276 publications, passed appraisal and are being synthesized using realist review methods. Discussion: The systematic and stage-based procedure addressed challenges to PR assessment and generated our robust understanding of complex and heterogeneous PR practices. To date, realist reviews have focussed on evaluations of relatively uniform interventions. In contrast our PR search yielded a wide diversity of partnerships and research topics. We therefore developed tools to achieve conceptual clarity on the PR field, as a beneficial precursor to our theoretically-driven synthesis using realist methods. Findings from the ongoing review will be provided in forthcoming publications.
TL;DR: This work has identified core competencies in this field and has developed a series of educational courses and materials for three training streams to meet the capacity demand for KT researchers and practitioners in Canada.
Abstract: Globally, healthcare systems are attempting to optimize quality of care. This challenge has resulted in the development of implementation science or knowledge translation (KT) and the resulting need to build capacity in both the science and practice of KT. We are attempting to meet these challenges through the creation of a national training initiative in KT. We have identified core competencies in this field and have developed a series of educational courses and materials for three training streams. We report the outline for this approach and the progress to date. We have prepared a strategy to develop, implement, and evaluate a national training initiative to build capacity in the science and practice of KT. Ultimately through this initiative, we hope to meet the capacity demand for KT researchers and practitioners in Canada that will lead to improved care and a strengthened healthcare system.
TL;DR: A decision-maker-researcher partnership systematic review found that the majority of CCDSSs demonstrated improvements in process of care, but patient outcomes were less likely to be evaluated and far more likely to show positive results.
Abstract: Background: Acute medical care often demands timely, accurate decisions in complex situations. Computerized clinical decision support systems (CCDSSs) have many features that could help. However, as for any medical intervention, claims that CCDSSs improve care processes and patient outcomes need to be rigorously assessed. The objective of this review was to systematically review the effects of CCDSSs on process of care and patient outcomes for acute medical care. Methods: We conducted a decision-maker-researcher partnership systematic review. MEDLINE, EMBASE, EvidenceBased Medicine Reviews databases (Cochrane Database of Systematic Reviews, DARE, ACP Journal Club, and others), and the Inspec bibliographic database were searched to January 2010, in all languages, for randomized controlled trials (RCTs) of CCDSSs in all clinical areas. We included RCTs that evaluated the effect on process of care or patient outcomes of a CCDSS used for acute medical care compared with care provided without a CCDSS. A study was considered to have a positive effect (i.e., CCDSS showed improvement) if at least 50% of the relevant study outcomes were statistically significantly positive. Results: Thirty-six studies met our inclusion criteria for acute medical care. The CCDSS improved process of care in 63% (22/35) of studies, including 64% (9/14) of medication dosing assistants, 82% (9/11) of management assistants using alerts/reminders, 38% (3/8) of management assistants using guidelines/algorithms, and 67% (2/3) of diagnostic assistants. Twenty studies evaluated patient outcomes, of which three (15%) reported improvements, all of which were medication dosing assistants. Conclusion: The majority of CCDSSs demonstrated improvements in process of care, but patient outcomes were less likely to be evaluated and far less likely to show positive results.
TL;DR: The CoP approach can complement traditional 'stage-of-change' theories used in the field of implementation research and provide a basis for designing theory-informed interventions and evaluations and suggest ways of crossing multiple boundaries to enable knowledge transfer and organisational learning.
Abstract: The paper combines the analytical and instrumental perspectives on communities of practice (CoPs) to reflect on potential challenges that may arise in the process of interprofessional and inter-organisational joint working within the Collaborations for Leaderships in Applied Health Research and Care (CLAHRCs)--partnerships between the universities and National Health Service (NHS) Trusts aimed at conducting applied health research and translating its findings into day-to-day clinical practice. The paper discusses seminal theoretical literature on CoPs as well as previous empirical research on the role of these communities in healthcare collaboration, which is organised around the following three themes: knowledge sharing within and across CoPs, CoP formation and manageability, and identity building in CoPs. It argues that the multiprofessional and multi-agency nature of the CLAHRCs operating in the traditionally demarcated organisational landscape of the NHS may present formidable obstacles to knowledge sharing between various professional groupings, formation of a shared 'collaborative' identity, and the development of new communities within the CLAHRCs. To cross multiple boundaries between various professional and organisational communities and hence enable the flow of knowledge, the CLAHRCs will have to create an effective system of 'bridges' involving knowledge brokers, boundary objects, and cross-disciplinary interactions as well as address a number of issues related to professional and organisational identification. The CoP approach can complement traditional 'stage-of-change' theories used in the field of implementation research and provide a basis for designing theory-informed interventions and evaluations. It can help to illuminate multiple boundaries that exist between professional and organisational groups within the CLAHRCs and suggest ways of crossing those boundaries to enable knowledge transfer and organisational learning. Achieving the aims of the CLAHRCs and producing a sustainable change in the ways applied health research is conducted and implemented may be influenced by how effectively these organisations can navigate through the multiple CoPs involved and promote the development of new multiprofessional and multi-organisational communities united by shared practice and a shared sense of belonging--an assumption that needs to be explored by further empirical research.
TL;DR: Successful implementation of EBPs requires consideration and utilization of existing social networks of high-status systems leaders that often cut across service organizations and their geographic jurisdictions.
Abstract: Background: The present study examines the structure and operation of social networks of information and advice and their role in making decisions as to whether to adopt new evidence-based practices (EBPs) among agency directors and other program professionals in 12 California counties participating in a large randomized controlled trial. Methods: Interviews were conducted with 38 directors, assistant directors, and program managers of county probation, mental health, and child welfare departments. Grounded-theory analytic methods were used to identify themes related to EBP adoption and network influences. A web-based survey collected additional quantitative information on members of information and advice networks of study participants. A mixed-methods approach to data analysis was used to create a sociometric data set (n = 176) for examination of associations between advice seeking and network structure. Results: Systems leaders develop and maintain networks of information and advice based on roles, responsibility, geography, and friendship ties. Networks expose leaders to information about EBPs and opportunities to adopt EBPs; they also influence decisions to adopt EBPs. Individuals in counties at the same stage of implementation accounted for 83% of all network ties. Networks in counties that decided not to implement a specific EBP had no extra-county ties. Implementation of EBPs at the two-year follow-up was associated with the size of county, urban versus rural counties, and in-degree centrality. Collaboration was viewed as critical to implementing EBPs, especially in small, rural counties where agencies have limited resources on their own. Conclusions: Successful implementation of EBPs requires consideration and utilization of existing social networks of high-status systems leaders that often cut across service organizations and their geographic jurisdictions.
TL;DR: This paper outlines the GM CLAHRC approach to designing and evaluating a large-scale, evidence- and theory-informed, context-sensitive implementation programme and presents an argument for adopting an integrative, co-production approach to planning and evaluating the implementation of research into practice.
Abstract: In response to policy recommendations, nine National Institute for Health Research (NIHR) Collaborations for Leadership in Applied Health Research and Care (CLAHRCs) were established in England in 2008, aiming to create closer working between the health service and higher education and narrow the gap between research and its implementation in practice. The Greater Manchester (GM) CLAHRC is a partnership between the University of Manchester and twenty National Health Service (NHS) trusts, with a five-year mission to improve healthcare and reduce health inequalities for people with cardiovascular conditions. This paper outlines the GM CLAHRC approach to designing and evaluating a large-scale, evidence- and theory-informed, context-sensitive implementation programme. The paper makes a case for embedding evaluation within the design of the implementation strategy. Empirical, theoretical, and experiential evidence relating to implementation science and methods has been synthesised to formulate eight core principles of the GM CLAHRC implementation strategy, recognising the multi-faceted nature of evidence, the complexity of the implementation process, and the corresponding need to apply approaches that are situationally relevant, responsive, flexible, and collaborative. In turn, these core principles inform the selection of four interrelated building blocks upon which the GM CLAHRC approach to implementation is founded. These determine the organizational processes, structures, and roles utilised by specific GM CLAHRC implementation projects, as well as the approach to researching implementation, and comprise: the Promoting Action on Research Implementation in Health Services (PARIHS) framework; a modified version of the Model for Improvement; multiprofessional teams with designated roles to lead, facilitate, and support the implementation process; and embedded evaluation and learning. Designing and evaluating a large-scale implementation strategy that can cope with and respond to the local complexities of implementing research evidence into practice is itself complex and challenging. We present an argument for adopting an integrative, co-production approach to planning and evaluating the implementation of research into practice, drawing on an eclectic range of evidence sources.
TL;DR: This review reveals significant underdevelopment in the measurement of research utilization, and substantial methodological advances with respect to construct clarity, use ofResearch utilization and related theory,Use of measurement theory, and psychometric assessment are required.
Abstract: In healthcare, a gap exists between what is known from research and what is practiced. Understanding this gap depends upon our ability to robustly measure research utilization. The objectives of this systematic review were: to identify self-report measures of research utilization used in healthcare, and to assess the psychometric properties (acceptability, reliability, and validity) of these measures. We conducted a systematic review of literature reporting use or development of self-report research utilization measures. Our search included: multiple databases, ancestry searches, and a hand search. Acceptability was assessed by examining time to complete the measure and missing data rates. Our approach to reliability and validity assessment followed that outlined in the Standards for Educational and Psychological Testing. Of 42,770 titles screened, 97 original studies (108 articles) were included in this review. The 97 studies reported on the use or development of 60 unique self-report research utilization measures. Seven of the measures were assessed in more than one study. Study samples consisted of healthcare providers (92 studies) and healthcare decision makers (5 studies). No studies reported data on acceptability of the measures. Reliability was reported in 32 (33%) of the studies, representing 13 of the 60 measures. Internal consistency (Cronbach's Alpha) reliability was reported in 31 studies; values exceeded 0.70 in 29 studies. Test-retest reliability was reported in 3 studies with Pearson's r coefficients > 0.80. No validity information was reported for 12 of the 60 measures. The remaining 48 measures were classified into a three-level validity hierarchy according to the number of validity sources reported in 50% or more of the studies using the measure. Level one measures (n = 6) reported evidence from any three (out of four possible) Standards validity sources (which, in the case of single item measures, was all applicable validity sources). Level two measures (n = 16) had evidence from any two validity sources, and level three measures (n = 26) from only one validity source. This review reveals significant underdevelopment in the measurement of research utilization. Substantial methodological advances with respect to construct clarity, use of research utilization and related theory, use of measurement theory, and psychometric assessment are required. Also needed are improved reporting practices and the adoption of a more contemporary view of validity (i.e., the Standards) in future research utilization measurement studies.
TL;DR: Assessment of potential differences between administrators/policymakers and those involved in direct practice regarding factors believed to be barriers or facilitating factors to evidence-based practice (EBP) implementation in a large public mental health service system in the United States found perceptions of importance of factors affecting EBP implementation varied between the two groups.
Abstract: Background: The goal of this study was to assess potential differences between administrators/policymakers and those involved in direct practice regarding factors believed to be barriers or facilitating factors to evidence-based practice (EBP) implementation in a large public mental health service system in the United States. Methods: Participants included mental health system county officials, agency directors, program managers, clinical staff, administrative staff, and consumers. As part of concept mapping procedures, brainstorming groups were conducted with each target group to identify specific factors believed to be barriers or facilitating factors to EBP implementation in a large public mental health system. Statements were sorted by similarity and rated by each participant in regard to their perceived importance and changeability. Multidimensional scaling, cluster analysis, descriptive statistics and t-tests were used to analyze the data. Results: A total of 105 statements were distilled into 14 clusters using concept-mapping procedures. Perceptions of importance of factors affecting EBP implementation varied between the two groups, with those involved in direct practice assigning significantly higher ratings to the importance of Clinical Perceptions and the impact of EBP implementation on clinical practice. Consistent with previous studies, financial concerns (costs, funding) were rated among the most important and least likely to change by both groups. Conclusions: EBP implementation is a complex process, and different stakeholders may hold different opinions regarding the relative importance of the impact of EBP implementation. Implementation efforts must include input from stakeholders at multiple levels to bring divergent and convergent perspectives to light. Background The implementation of evidence-based practices (EBPs) into real-world children’s mental health service settings is an important step in improving the quality of services and outcomes for youth and families [1,2]. This holds especially true for clients in the public sector who often have difficulty accessing services and have few alternatives if treatments are not effective. Public mental health services are embedded in local health and human service systems; therefore, input from multiple levels of stakeholders must be considered for effective major change efforts such as implementation of EBP [3,4]. In public mental healthcare, stakeholders include not only the individuals most directly involved–the consumers, clinicians, and administrative staff–but also program managers, agency directors, and local, state, and federal policymakers who may structure organizations and financing in ways more or less conducive to EBPs. Considerable resources are being used to increase the implementation of EBPs into community care; however, actual implementation requires consideration of multiple stakeholder groups and the different ways they may be impacted. Our conceptual model of EBP implementation in public sector services identifies four phases of
TL;DR: CCDSSs have potential for improving process of care for therapeutic drug monitoring and dosing, specifically insulin and vitamin K antagonist dosing and should be evaluated by independent researchers using cluster randomization and primarily assess patient outcomes related to drug efficacy and safety.
Abstract: Some drugs have a narrow therapeutic range and require monitoring and dose adjustments to optimize their efficacy and safety. Computerized clinical decision support systems (CCDSSs) may improve the net benefit of these drugs. The objective of this review was to determine if CCDSSs improve processes of care or patient outcomes for therapeutic drug monitoring and dosing. We conducted a decision-maker-researcher partnership systematic review. Studies from our previous review were included, and new studies were sought until January 2010 in MEDLINE, EMBASE, Evidence-Based Medicine Reviews, and Inspec databases. Randomized controlled trials assessing the effect of a CCDSS on process of care or patient outcomes were selected by pairs of independent reviewers. A study was considered to have a positive effect (i.e., CCDSS showed improvement) if at least 50% of the relevant study outcomes were statistically significantly positive. Thirty-three randomized controlled trials were identified, assessing the effect of a CCDSS on management of vitamin K antagonists (14), insulin (6), theophylline/aminophylline (4), aminoglycosides (3), digoxin (2), lidocaine (1), or as part of a multifaceted approach (3). Cluster randomization was rarely used (18%) and CCDSSs were usually stand-alone systems (76%) primarily used by physicians (85%). Overall, 18 of 30 studies (60%) showed an improvement in the process of care and 4 of 19 (21%) an improvement in patient outcomes. All evaluable studies assessing insulin dosing for glycaemic control showed an improvement. In meta-analysis, CCDSSs for vitamin K antagonist dosing significantly improved time in therapeutic range. CCDSSs have potential for improving process of care for therapeutic drug monitoring and dosing, specifically insulin and vitamin K antagonist dosing. However, studies were small and generally of modest quality, and effects on patient outcomes were uncertain, with no convincing benefit in the largest studies. At present, no firm recommendation for specific systems can be given. More potent CCDSSs need to be developed and should be evaluated by independent researchers using cluster randomization and primarily assess patient outcomes related to drug efficacy and safety.
TL;DR: The limited empirical data renders the strength of evidence weak for the effectiveness and the types of interventions that encourage health policymakers and managers to use systematic reviews in decision making.
Abstract: Background: Systematic reviews have the potential to inform decisions made by health policymakers and managers, yet little is known about the impact of interventions to increase the use of systematic reviews by these groups in decision making. Methods: We systematically reviewed the evidence on the impact of interventions for seeking, appraising, and applying evidence from systematic reviews in decision making by health policymakers or managers. Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Health Technology Assessment Database, and LISA were searched from the earliest date available until April 2010. Two independent reviewers selected studies for inclusion if the intervention intended to increase seeking, appraising, or applying evidence from systematic reviews by a health policymaker or manager. Minimum inclusion criteria were a description of the study population and availability of extractable data. Results: 11,297 titles and abstracts were reviewed, leading to retrieval of 37 full-text articles for assessment; four of these articles met all inclusion criteria. Three articles described one study where five systematic reviews were mailed to public health officials and followed up with surveys at three months and two years. The articles reported from 23% to 63% of respondents declaring they had used systematic reviews in policymaking decisions. One randomised trial indicated that tailored messages combined with access to a registry of systematic reviews had a significant effect on policies made in the area of healthy body weight promotion in health departments. Conclusions: The limited empirical data renders the strength of evidence weak for the effectiveness and the types of interventions that encourage health policymakers and managers to use systematic reviews in decision making. Background Policymakers and managers working within health systems make decisions in efforts to improve health for individuals. The impact of the choices made by policymakers is experienced in the health status and daily lives of people in the form of laws and regulations, guidelines, public education campaigns, among others [1]. The choices made by healthcare managers affect environments where common goals and strategies must be found between clinical and administrative environments [2]. Overall, decisions by policymakers and managers are made around burdensome health problems, within complex health systems, and ideally involve effective solutions and strategies to support their implementation.