Showing papers by "Tanya Horsley published in 2011"

Checking reference lists to find additional studies for systematic reviews

Abstract: Background Checking reference lists to identify relevant studies for systematic reviews is frequently recommended by systematic review manuals and is often undertaken by review authors. To date, no systematic review has explicitly examined the effectiveness of checking reference lists as a method to supplement electronic searching. Objectives To investigate the effectiveness of checking reference lists for the identification of additional, relevant studies for systematic reviews. Effectiveness is defined as the proportion of relevant studies identified by review authors solely by checking reference lists. Search methods We searched the databases of The Cochrane Library (Issue 3, 2008), Library and Information Science abstracts (LISA) (1969 to July 2008) and MEDLINE (1966 to July 2008). We contacted experts in systematic review methods and examined reference lists of articles. Selection criteria Studies of any design which examined checking reference lists as a search method for systematic reviews in any area. The primary outcome was the additional yield of relevant studies (i.e. studies not found through any other search methodologies); other outcomes were publication types identified and data pertaining to the costs (e.g. cost-effectiveness, cost-efficiency) of checking reference lists. Data collection and analysis We summarized data descriptively. Main results We included 12 studies (in 13 publications) in this review, but interpretability and generalizability of these studies is difficult and the study designs used were at high risk of bias. The additional yield (calculated by dividing the additional 'unique' yield identified by checking reference lists by the total number of studies found to be eligible within the study) of relevant studies identified through checking reference lists ranged from 2.5% to 42.7%. Only two studies reported yield information by publication type (dissertations and systematic reviews). No cost data were reported although one study commented that it was impossible to isolate the time spent on reference tracking since this was done in parallel with the critical appraisal of each paper, and for that particular study costs were not specifically estimated. Authors' conclusions There is some evidence to support the use of checking reference lists for locating studies in systematic reviews. However, this evidence is derived from weak study designs. In situations where the identification of all relevant studies through handsearching and database searching is difficult, it would seem prudent that authors of reviews check reference lists to supplement their searching. The challenge, therefore, is for review authors to recognize those situations.

Teaching critical appraisal skills in healthcare settings

Abstract: Background Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity, results and relevance to an individual's work. Within the last decade critical appraisal has been added as a topic to many medical school and UK Royal College curricula, and several continuing professional development ventures have been funded to provide further training. This is an update of a Cochrane review first published in 2001. Objectives To assess the effects of teaching critical appraisal skills to health professionals on the process of care, patient outcomes and knowledge of health professionals. SEARCH METHODS. We updated the search (see Appendix 1 for search strategies by database) and used those search strategies to search the Cochrane Central Register of Controlled Trials (1997 to June 2011) and MEDLINE (from 1997 to June 2011). We also searched EMBASE, CINAHL and PsycINFO (up to January 2010). We searched LISA (up to January 2010), ERIC (up to January 2010), SIGLE (up to January 2010) and Web of Knowledge (up to January 2010). We also searched the Cochrane Database of Systematic Reviews (CDSR), Database of Abstracts of Reviews of Effects (DARE) and the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register up to January 2010. Selection criteria Randomised trials, controlled clinical trials, controlled before and after studies and interrupted time series analyses that examined the effectiveness of educational interventions teaching critical appraisal to health professionals. The outcomes included process of care, patient mortality, morbidity, quality of life and satisfaction. We included studies reporting on health professional knowledge/awareness only when based upon objective, standardised, validated instruments. We did not consider studies involving students. Data collection and analysis Two review authors independently extracted data and assessed risk of bias. We contacted authors of included studies to obtain missing data. Main results In total, we reviewed a total of 11,057 titles and abstracts, of which 148 appeared potentially relevant to the review. We included three studies involving 272 people in this review. None of the included studies evaluated process of care or patient outcomes. Statistically significant improvements in participants' knowledge were reported in domains of critical appraisal (variable approaches across studies) in two of the three studies. We determined risk of bias to be 'unclear' and as such considered this to be 'plausible bias that raises some doubt about the results'. Authors' conclusions Low-intensity critical appraisal teaching interventions in healthcare populations may result in modest gains. Improvements to research examining the effectiveness of interventions in healthcare populations are required; specifically rigorous randomised trials employing interventions using appropriate adult learning theories.

Blinded versus unblinded assessments of risk of bias in studies included in a systematic review

Abstract: Background The importance of appraising the risk of bias of studies included in systematic reviews is well-established. However, uncertainty remains surrounding the method by which risk of bias assessments should be conducted. Specifically, no summary of evidence exists as to whether blinded (i.e. the assessor is unaware of the study author’s name, institution, sponsorship, journal, etc.) versus unblinded assessments of risk of bias yield systematically different assessments in a systematic review. Objectives To determine whether blinded versus unblinded assessments of risk of bias yield systematically different assessments in a systematic review. Search methods We searched MEDLINE (1966 to September week 4 2009), CINAHL (1982 to May week 3 2008), All EBM Reviews (inception to 6 October 2009), EMBASE (1980 to 2009 week 40) and HealthStar (1966 to September week 4 2009) (all Ovid interface). We applied no restrictions regarding language of publication, publication status or study design. We examined reference lists of included studies and contacted experts for potentially relevant literature. Selection criteria We included any study that examined blinded versus unblinded assessments of risk of bias included within a systematic review. Data collection and analysis We extracted information from each of the included studies using a pre-specified 16-item form. We summarized the level of agreement between blinded and unblinded assessments of risk of bias descriptively. We calculated the standardized mean difference whenever possible. Main results We included six randomized controlled trials (RCTs). Four studies had unclear risk of bias and two had high risk of bias. The results of these RCTs were not consistent; two demonstrated no differences between blinded and unblinded assessments, two found that blinded assessments had significantly lower quality scores, and another observed significantly higher quality scores for blinded assessments. The remaining study did not report the level of significance. We pooled five studies reporting sufficient information in a meta-analysis. We observed no statistically significant difference in risk of bias assessments between blinded or unblinded assessments (standardized mean difference -0.13, 95% confidence interval -0.42 to 0.16). The mean difference might be slightly inaccurate, as we did not adjust for clustering in our meta-analysis. We observed inconsistency of results visually and noted statistical heterogeneity. Authors' conclusions Our review highlights that discordance exists between studies examining blinded versus unblinded risk of bias assessments at the systematic review level. The best approach to risk of bias assessment remains unclear, however, given the increased time and resources required to conceal reports effectively, it may not be necessary for risk of bias assessments to be conducted under blinded conditions in a systematic review.