Undertaking a review of the literature is an important part of any research project. The researcher both maps and assesses the relevant intellectual territory in order to specify a research question which will further develop the knowledge base. However, traditional 'narrative' reviews frequently lack thoroughness, and in many cases are not undertaken as genuine pieces of investigatory science. Consequently they can lack a means for making sense of what the collection of studies is saying. These reviews can be biased by the researcher and often lack rigour. Furthermore, the use of reviews of the available evidence to provide insights and guidance for intervention into operational needs of practitioners and policymakers has largely been of secondary importance. For practitioners, making sense of a mass of often-contradictory evidence has become progressively harder. The quality of evidence underpinning decision-making and action has been questioned, for inadequate or incomplete evidence seriously impedes policy formulation and implementation. In exploring ways in which evidence-informed management reviews might be achieved, the authors evaluate the process of systematic review used in the medical sciences. Over the last fifteen years, medical science has attempted to improve the review process by synthesizing research in a systematic, transparent, and reproducible manner with the twin aims of enhancing the knowledge base and informing policymaking and practice. This paper evaluates the extent to which the process of systematic review can be applied to the management field in order to produce a reliable knowledge stock and enhanced practice by developing context-sensitive research. The paper highlights the challenges in developing an appropriate methodology.

Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review

Undertaking a review of the literature is an important part of any research project. The researcher both maps and assesses the relevant intellectual territory in order to specify a research question which will further develop the knowledge hase. However, traditional 'narrative' reviews frequently lack thoroughness, and in many cases are not undertaken as genuine pieces of investigatory science. Consequently they can lack a means for making sense of what the collection of studies is saying. These reviews can he hiased by the researcher and often lack rigour. Furthermore, the use of reviews of the available evidence to provide insights and guidance for intervention into operational needs of practitioners and policymakers has largely been of secondary importance. For practitioners, making sense of a mass of often-contrad ictory evidence has hecome progressively harder. The quality of evidence underpinning decision-making and action has heen questioned, for inadequate or incomplete evidence seriously impedes policy formulation and implementation. In exploring ways in which evidence-informed management reviews might be achieved, the authors evaluate the process of systematic review used in the medical sciences. Over the last fifteen years, medical science has attempted to improve the review process hy synthesizing research in a systematic, transparent, and reproducihie manner with the twin aims of enhancing the knowledge hase and informing policymaking and practice. This paper evaluates the extent to which the process of systematic review can be applied to the management field in order to produce a reliable knowledge stock and enhanced practice by developing context-sensitive research. The paper highlights the challenges in developing an appropriate methodology.

OBJECTIVE: To test the feasibility of creating a valid and reliable checklist with the following features: appropriate for assessing both randomised and non-randomised studies; provision of both an overall score for study quality and a profile of scores not only for the quality of reporting, internal validity (bias and confounding) and power, but also for external validity. DESIGN: A pilot version was first developed, based on epidemiological principles, reviews, and existing checklists for randomised studies. Face and content validity were assessed by three experienced reviewers and reliability was determined using two raters assessing 10 randomised and 10 non-randomised studies. Using different raters, the checklist was revised and tested for internal consistency (Kuder-Richardson 20), test-retest and inter-rater reliability (Spearman correlation coefficient and sign rank test; kappa statistics), criterion validity, and respondent burden. MAIN RESULTS: The performance of the checklist improved considerably after revision of a pilot version. The Quality Index had high internal consistency (KR-20: 0.89) as did the subscales apart from external validity (KR-20: 0.54). Test-retest (r 0.88) and inter-rater (r 0.75) reliability of the Quality Index were good. Reliability of the subscales varied from good (bias) to poor (external validity). The Quality Index correlated highly with an existing, established instrument for assessing randomised studies (r 0.90). There was little difference between its performance with non-randomised and with randomised studies. Raters took about 20 minutes to assess each paper (range 10 to 45 minutes). CONCLUSIONS: This study has shown that it is feasible to develop a checklist that can be used to assess the methodological quality not only of randomised controlled trials but also non-randomised studies. It has also shown that it is possible to produce a checklist that provides a profile of the paper, alerting reviewers to its particular methodological strengths and weaknesses. Further work is required to improve the checklist and the training of raters in the assessment of external validity.

https://jech.bmj.com/content/jech/52/6/377.full.pdf

The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions.

There is confusion and misunderstanding about the concepts of knowledge translation, knowledge transfer, knowledge exchange, research utilization, implementation, diffusion, and dissemination. We review the terms and definitions used to describe the concept of moving knowledge into action. We also offer a conceptual framework for thinking about the process and integrate the roles of knowledge creation and knowledge application. The implications of knowledge translation for continuing education in the health professions include the need to base continuing education on the best available knowledge, the use of educational and other transfer strategies that are known to be effective, and the value of learning about plannedaction theories to be better able to understand and influence change in practice settings.

https://onlinelibrary.wiley.com/doi/pdf/10.1002/chp.47

Lost in knowledge translation: time for a map?

Such diverse thinkers as Lao-Tze, Confucius, and U.S. Defense Secretary Donald Rumsfeld have all pointed out that we need to be able to tell the difference between real and assumed knowledge. The systematic review is a scientific tool that can help with this difficult task. It can help, for example, with appraising, summarising, and communicating the results and implications of otherwise unmanageable quantities of data. This is important because quite often there are so many studies, and their results are often so conflicting, that no policymaker or practitioner could possibly carry out this task themselves.Systematic review methods have been widely used in health care, and are becoming increasingly common in the social sciences (fostered, for example, by the work of the Campbell Collaboration). 

This book outlines the rationale and methods of systematic reviews, giving worked examples from social science and other fields. It requires no previous knowledge, but takes the reader through the process stage by stage. It draws on examples from such diverse fields as psychology, criminology, education, transport, social welfare, public health, and housing and urban policy, among others.The book includes detailed sections on assessing the quality of both quantitative, and qualitative research; searching for evidence in the social sciences;meta-analytic and other methods of evidence synthesis; publication bias; heterogeneity; and approaches to dissemination.

Systematic Reviews in the Social Sciences: A Practical Guide

Doctors within the NHS are confronting major changes at work. While we endeavour to improve the quality of health care, junior doctors' hours have been reduced and the emphasis on continuing medical education has increased. We are confronted by a growing body of information, much of it invalid or irrelevant to clinical practice. This article discusses evidence based medicine, a process of turning clinical problems into questions and then systematically locating, appraising, and using contemporaneous research findings as the basis for clinical decisions. The computerisation of bibliographies and the development of software that permits the rapid location of relevant evidence have made it easier for busy clinicians to make best use of the published literature. Critical appraisal can be used to determine the validity and applicability of the evidence, which is then used to inform clinical decisions. Evidence based medicine can be taught to, and practised by, clinicians at all levels of seniority and can be used to close the gulf between good clinical research and clinical practice. In addition it can help to promote self directed learning and teamwork and produce faster and better doctors.

/pdf/evidence-based-medicine-an-approach-to-clinical-problem-pnanreboe5.pdf

Evidence based medicine: an approach to clinical problem-solving

Objective To determine the effect of home blood pressure monitoring on blood pressure levels and proportion of people with essential hypertension achieving targets. Design Meta-analysis of 18 randomised controlled trials. Participants 1359 people with essential hypertension allocated to home blood pressure monitoring and 1355 allocated to the “control” group seen in the healthcare system for 2-36 months. Main outcome measures Differences in systolic (13 studies), diastolic (16 studies), or mean (3 studies) blood pressures, and proportion of patients achieving targets (6 studies), between intervention and control groups. Results Systolic blood pressure was lower in people with hypertension who had home blood pressure monitoring than in those who had standard blood pressure monitoring in the healthcare system (standardised mean difference 4.2 (95% confidence interval 1.5 to 6.9) mm Hg), diastolic blood pressure was lower by 2.4 (1.2 to 3.5) mm Hg, and mean blood pressure was lower by 4.4 (2.0 to 6.8) mm Hg. The relative risk of blood pressure above predetermined targets was lower in people with home blood pressure monitoring (risk ratio 0.90, 0.80 to 1.00). When publication bias was allowed for, the differences were attenuated: 2.2 (−0.9 to 5.3) mm Hg for systolic blood pressure and 1.9 (0.6 to 3.2) mm Hg for diastolic blood pressure. Conclusions Blood pressure control in people with hypertension (assessed in the clinic) and the proportion achieving targets are increased when home blood pressure monitoring is used rather than standard blood pressure monitoring in the healthcare system. The reasons for this are not clear. The difference in blood pressure control between the two methods is small but likely to contribute to an important reduction in vascular complications in the hypertensive population.

https://www.bmj.com/content/bmj/329/7458/145.full.pdf

Blood pressure control by home monitoring: meta-analysis of randomised trials

This is the first in a series of eight articles analysing the gap between research and practice

Series editors: Andrew Haines and Anna Donald

There is increasing interest in implementing research findings in practice both because of a growing awareness of the gap between clinical practice and the findings of research and also because of the need to show that public investment in research results in benefits for patients. Improved understanding of the reasons for the uptake of research findings requires insights from a range of disciplines. In order to promote the uptake of research findings it is necessary to identify potential barriers to implementation and to develop strategies to overcome them. Specific interventions that can be used to promote change in practice include using clinical guidelines and computerised decision support systems, developing educational programmes, communicating research findings to patients, and developing strategies for organisational change.

Interest in how best to promote the uptake of research findings has been fuelled by a number of factors including the well documented disparities between clinical practice and research evidence of effective interventions. Examples include interventions in the management of cardiac failure, secondary prevention of heart disease,1 atrial fibrillation,2 menorrhagia,3 and pregnancy and childbirth.4 In the United Kingdom the advent of the NHS research and development programme has led to greater involvement of NHS personnel in setting priorities5 and to the establishment of a programme to evaluate different methods of promoting the implementation of research findings.6 The concept of pay back on research7 has also been developed, resulting in a framework that can be used to assess the benefits arising from research.

Relying on the passive diffusion of information to keep health professionals' knowledge up to date is doomed to failure in a global environment in which …

Making better use of research findings

/pdf/getting-research-findings-into-practice-4dd306wcar.pdf

Anna Donald

Papers

Evidence based medicine: an approach to clinical problem-solving

Blood pressure control by home monitoring: meta-analysis of randomised trials

Making better use of research findings

Getting research findings into practice

Blood pressure control by home monitoring: Meta-analysis of randomised trials