Supporting 21st century health care and the practice of evidence-based medicine (EBM) requires ubiquitous access to clinical information and to knowledge-based resources to answer clinical questions. Many questions go unanswered, however, due to lack of skills in formulating questions, crafting effective search strategies, and accessing databases to identify best levels of evidence. This randomized trial was designed as a pilot study to measure the relevancy of search results using three different interfaces for the PubMed search system. Two of the search interfaces utilized a specific framework called PICO, which was designed to focus clinical questions and to prompt for publication type or type of question asked. The third interface was the standard PubMed interface readily available on the Web. Study subjects were recruited from interns and residents on an inpatient general medicine rotation at an academic medical center in the US. Thirty-one subjects were randomized to one of the three interfaces, given 3 clinical questions, and asked to search PubMed for a set of relevant articles that would provide an answer for each question. The success of the search results was determined by a precision score, which compared the number of relevant or gold standard articles retrieved in a result set to the total number of articles retrieved in that set. Participants using the PICO templates (Protocol A or Protocol B) had higher precision scores for each question than the participants who used Protocol C, the standard PubMed Web interface. (Question 1: A = 35%, B = 28%, C = 20%; Question 2: A = 5%, B = 6%, C = 4%; Question 3: A = 1%, B = 0%, C = 0%) 95% confidence intervals were calculated for the precision for each question using a lower boundary of zero. However, the 95% confidence limits were overlapping, suggesting no statistical difference between the groups. Due to the small number of searches for each arm, this pilot study could not demonstrate a statistically significant difference between the search protocols. However there was a trend towards higher precision that needs to be investigated in a larger study to determine if PICO can improve the relevancy of search results.

/pdf/utilization-of-the-pico-framework-to-improve-searching-3kqp6609k5.pdf

Utilization of the PICO framework to improve searching PubMed for clinical questions

Surgical site infections (SSIs) are among the most preventable health-care-associated infections and are a substantial burden to health-care systems and service payers worldwide in terms of patient morbidity, mortality, and additional costs. SSI prevention is complex and requires the integration of a range of measures before, during, and after surgery. No international guidelines are available and inconsistencies in the interpretation of evidence and recommendations of national guidelines have been identified. Given the burden of SSIs worldwide, the numerous gaps in evidence-based guidance, and the need for standardisation and a global approach, WHO decided to prioritise the development of evidence-based recommendations for the prevention of SSIs. The guidelines take into account the balance between benefits and harms, the evidence quality, cost and resource use implications, and patient values and preferences. On the basis of systematic literature reviews and expert consensus, we present 13 recommendations on preoperative preventive measures.

/pdf/new-who-recommendations-on-preoperative-measures-for-1tsqb4qtpf.pdf

New WHO recommendations on preoperative measures for surgical site infection prevention: an evidence-based global perspective.

The Coronavirus Disease 2019 (COVID-19) pandemic has deeply altered social and working environments in several ways. Social distancing policies, mandatory lockdowns, isolation periods, and anxiety of getting sick, along with the suspension of productive activity, loss of income, and fear of the future, jointly influence the mental health of citizens and workers. Workplace aspects can play a crucial role on moderating or worsening mental health of people facing this pandemic scenario. The purpose of this literature review is to deepen the psychological aspects linked to workplace factors, following the epidemic rise of COVID-19, in order to address upcoming psychological critical issues in the workplaces. We performed a literature search using Google Scholar, PubMed, and Scopus, selecting papers focusing on workers' psychological problems that can be related to the workplace during the pandemic. Thirty-five articles were included. Mental issues related to the health emergency, such as anxiety, depression, post-traumatic stress disorder (PTSD), and sleep disorders are more likely to affect healthcare workers, especially those on the frontline, migrant workers, and workers in contact with the public. Job insecurity, long periods of isolation, and uncertainty of the future worsen the psychological condition, especially in younger people and in those with a higher educational background. Multiple organizational and work-related interventions can mitigate this scenario, such as the improvement of workplace infrastructures, the adoption of correct and shared anti-contagion measures, including regular personal protective equipment (PPE) supply, and the implementation of resilience training programs. This review sets the basis for a better understanding of the psychological conditions of workers during the pandemic, integrating individual and social perspectives, and providing insight into possible individual, social, and occupational approaches to this "psychological pandemic".

COVID-19-Related Mental Health Effects in the Workplace: A Narrative Review.

Objective: This review aimed to determine if the use of the patient, intervention, comparison, outcome (PICO) model as a search strategy tool affects the quality of a literature search. Methods: A comprehensive literature search was conducted in PubMed, Embase, CINAHL, PsycINFO, Cochrane Library, Web of Science, Library and Information Science Abstracts (LISA), Scopus, and the National Library of Medicine (NLM) catalog up until January 9, 2017. Reference lists were scrutinized, and citation searches were performed on the included studies. The primary outcome was the quality of literature searches and the secondary outcome was time spent on the literature search when the PICO model was used as a search strategy tool, compared to the use of another conceptualizing tool or unguided searching. Results: A total of 2,163 records were identified, and after removal of duplicates and initial screening, 22 full-text articles were assessed. Of these, 19 studies were excluded and 3 studies were included, data were extracted, risk of bias was assessed, and a qualitative analysis was conducted. The included studies compared PICO to the PIC truncation or links to related articles in PubMed, PICOS, and sample, phenomenon of interest, design, evaluation, research type (SPIDER). One study compared PICO to unguided searching. Due to differences in intervention, no quantitative analysis was performed. Conclusions: Only few studies exist that assess the effect of the PICO model vis-a-vis other available models or even vis-a-vis the use of no model. Before implications for current practice can be drawn, well-designed studies are needed to evaluate the role of the tool used to devise a search strategy.  This article has been approved for the Medical Library Association’s Independent Reading Program .

/pdf/the-impact-of-patient-intervention-comparison-outcome-pico-3erk3xrgze.pdf

The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review

Use of test collections and evaluation measures to assess the effectiveness of information retrieval systems has its origins in work dating back to the early 1950s. Across the nearly 60 years since that work started, use of test collections is a de facto standard of evaluation. This monograph surveys the research conducted and explains the methods and measures devised for evaluation of retrieval systems, including a detailed look at the use of statistical significance testing in retrieval experimentation. This monograph reviews more recent examinations of the validity of the test collection approach and evaluation measures as well as outlining trends in current research exploiting query logs and live labs. At its core, the modern-day test collection is little different from the structures that the pioneering researchers in the 1950s and 1960s conceived of. This tutorial and review shows that despite its age, this long-standing evaluation method is still a highly valued tool for retrieval research.

/pdf/test-collection-based-evaluation-of-information-retrieval-6yfk2vuddi.pdf

Test Collection Based Evaluation of Information Retrieval Systems

The paradigm of evidence-based medicine (EBM) recommends that physicians formulate clinical questions in terms of the problem/population, intervention, comparison, and outcome. Together, these elements comprise a PICO frame. Although this framework was developed to facilitate the formulation of clinical queries, the ability of PICO structures to represent physicians’ information needs has not been empirically investigated. This paper evaluates the adequacy and suitability of PICO frames as a knowledge representation by analyzing 59 real-world primary-care clinical questions. We discovered that only two questions in our corpus contain all four PICO elements, and that 37% of questions contain both intervention and outcome. Our study reveals prevalent structural patterns for the four types of clinical questions: therapy, diagnosis, prognosis, and etiology. We found that the PICO framework is primarily centered on therapy questions, and is less suitable for representing other types of clinical information needs. Challenges in mapping natural language questions into PICO structures are also discussed. Although we point out limitations of the PICO framework, our work as a whole reaffirms its value as a tool to assist physicians practicing EBM.

/pdf/evaluation-of-pico-as-a-knowledge-representation-for-zmhi8o38t8.pdf

Evaluation of PICO as a Knowledge Representation for Clinical Questions

Test collections model use cases in ways that facilitate evaluation of information retrieval systems. This paper describes the use of search-guided relevance assessment to create a test collection for retrieval of spontaneous conversational speech. Approximately 10,000 thematically coherent segments were manually identified in 625 hours of oral history interviews with 246 individuals. Automatic speech recognition results, manually prepared summaries, controlled vocabulary indexing, and name authority control are available for every segment. Those features were leveraged by a team of four relevance assessors to identify topically relevant segments for 28 topics developed from actual user requests. Search-guided assessment yielded sufficient inter-annotator agreement to support formative evaluation during system development. Baseline results for ranked retrieval are presented to illustrate use of the collection.

/pdf/building-an-information-retrieval-test-collection-for-1xhbjrgdi5.pdf

Building an information retrieval test collection for spontaneous conversational speech

The CLEF-2006 Cross-Language Speech Retrieval (CL-SR) track included two tasks: to identify topically coherent segments of English interviews in a known-boundary condition, and to identify time stamps marking the beginning of topically relevant passages in Czech interviews in an unknown-boundary condition. Five teams participated in the English evaluation, performing both monolingual and cross-language searches of speech recognition transcripts, automatically generated metadata, and manually generated metadata. Results indicate that the 2006 English evaluation topics are more challenging than those used in 2005, but that cross-language searching continued to pose no unusual challenges when compared with monolingual searches of the same collection. Three teams participated in the monolingual Czech evaluation using a new evaluation measure based on differences between system-suggested and ground truth replay start times, with results that were broadly comparable to those observed for English.

/pdf/overview-of-the-clef-2006-cross-language-speech-retrieval-1zz526p6y5.pdf

Overview of the CLEF-2006 cross-language speech retrieval track

The task for the CLEF-2005 cross-language speech retrieval track was to identify topically coherent segments of English interviews in a known-boundary condition. Seven teams participated, performing both monolingual and cross-language searches of ASR transcripts, automatically generated metadata, and manually generated metadata. Results indicate that monolingual search technology is sufficiently accurate to be useful for some purposes (the best mean average precision was 0.13) and cross-language searching yielded results typical of those seen in other applications (with the best systems approximating monolingual mean average precision).

/pdf/overview-of-the-clef-2005-cross-language-speech-retrieval-2m2c582bcb.pdf

Overview of the CLEF-2005 cross-language speech retrieval track

Synthesizing and building on many ideas from the literature, this article presents an improved conceptual framework that clarifies the notion of relevance with its many elements, variables, criteria, and situational factors. Relevance is defined as a Relationship (R) between an Information Object (I) and an Information Need (N) (which consists of Topic, User, Problem/Task, and Situation/Context) with focus on R. This defines Relevance-as-is (conceptual relevance, strong relevance). To determine relevance, an Agent A (a person or system) operates on a representation I′ of the information object and a representation N′ of the information need, resulting in relevance-as-determined (operational measure of relevance, weak relevance, an approximation). Retrieval tests compare relevance-as-determined by different agents. This article discusses and compares two major approaches to conceptualizing relevance: the entity-focused approach (focus on elaborating the entities involved in relevance) and the relationship-focused approach (focus on explicating the relational nature of relevance). The article argues that because relevance is fundamentally a relational construct the relationship-focused approach deserves a higher priority and more attention than it has received. The article further elaborates on the elements of the framework with a focus on clarifying several critical issues on the discourse on relevance. © 2013 Wiley Periodicals, Inc.

Xiaoli Huang

Papers

Evaluation of PICO as a Knowledge Representation for Clinical Questions

Building an information retrieval test collection for spontaneous conversational speech

Overview of the CLEF-2006 cross-language speech retrieval track

Overview of the CLEF-2005 cross-language speech retrieval track

Relevance: An improved framework for explicating the notion