Showing papers in "BMC Medical Informatics and Decision Making in 2013"

Presenting quantitative information about decision outcomes: a risk communication primer for patient decision aid developers

Abstract: Making evidence-based decisions often requires comparison of two or more options. Research-based evidence may exist which quantifies how likely the outcomes are for each option. Understanding these numeric estimates improves patients’ risk perception and leads to better informed decision making. This paper summarises current “best practices” in communication of evidence-based numeric outcomes for developers of patient decision aids (PtDAs) and other health communication tools. An expert consensus group of fourteen researchers from North America, Europe, and Australasia identified eleven main issues in risk communication. Two experts for each issue wrote a “state of the art” summary of best evidence, drawing on the PtDA, health, psychological, and broader scientific literature. In addition, commonly used terms were defined and a set of guiding principles and key messages derived from the results. The eleven key components of risk communication were: 1) Presenting the chance an event will occur; 2) Presenting changes in numeric outcomes; 3) Outcome estimates for test and screening decisions; 4) Numeric estimates in context and with evaluative labels; 5) Conveying uncertainty; 6) Visual formats; 7) Tailoring estimates; 8) Formats for understanding outcomes over time; 9) Narrative methods for conveying the chance of an event; 10) Important skills for understanding numerical estimates; and 11) Interactive web-based formats. Guiding principles from the evidence summaries advise that risk communication formats should reflect the task required of the user, should always define a relevant reference class (i.e., denominator) over time, should aim to use a consistent format throughout documents, should avoid “1 in x” formats and variable denominators, consider the magnitude of numbers used and the possibility of format bias, and should take into account the numeracy and graph literacy of the audience. A substantial and rapidly expanding evidence base exists for risk communication. Developers of tools to facilitate evidence-based decision making should apply these principles to improve the quality of risk communication in practice.

“Many miles to go …”: a systematic review of the implementation of patient decision support interventions into routine clinical practice

Abstract: Two decades of research has established the positive effect of using patient-targeted decision support interventions: patients gain knowledge, greater understanding of probabilities and increased confidence in decisions. Yet, despite their efficacy, the effectiveness of these decision support interventions in routine practice has yet to be established; widespread adoption has not occurred. The aim of this review was to search for and analyze the findings of published peer-reviewed studies that investigated the success levels of strategies or methods where attempts were made to implement patient-targeted decision support interventions into routine clinical settings. An electronic search strategy was devised and adapted for the following databases: ASSIA, CINAHL, Embase, HMIC, Medline, Medline-in-process, OpenSIGLE, PsycINFO, Scopus, Social Services Abstracts, and the Web of Science. In addition, we used snowballing techniques. Studies were included after dual independent assessment. After assessment, 5322 abstracts yielded 51 articles for consideration. After examining full-texts, 17 studies were included and subjected to data extraction. The approach used in all studies was one where clinicians and their staff used a referral model, asking eligible patients to use decision support. The results point to significant challenges to the implementation of patient decision support using this model, including indifference on the part of health care professionals. This indifference stemmed from a reported lack of confidence in the content of decision support interventions and concern about disruption to established workflows, ultimately contributing to organizational inertia regarding their adoption. It seems too early to make firm recommendations about how best to implement patient decision support into routine practice because approaches that use a ‘referral model’ consistently report difficulties. We sense that the underlying issues that militate against the use of patient decision support and, more generally, limit the adoption of shared decision making, are under-investigated and under-specified. Future reports from implementation studies could be improved by following guidelines, for example the SQUIRE proposals, and by adopting methods that would be able to go beyond the ‘barriers’ and ‘facilitators’ approach to understand more about the nature of professional and organizational resistance to these tools. The lack of incentives that reward the use of these interventions needs to be considered as a significant impediment.

A systematic development process for patient decision aids

Abstract: The original version of the International Patient Decision Aid Standards (IPDAS) recommended that patient decision aids (PtDAs) should be carefully developed, user-tested and open to scrutiny, with a well-documented and systematically applied development process. We carried out a review to check the relevance and scope of this quality dimension and, if necessary, to update it. Our review drew on three sources: a) published papers describing PtDAs evaluated in randomised controlled trials and included in the most recent Cochrane Collaboration review; b) linked papers cited in the trial reports that described how the PtDAs had been developed; and c) papers and web reports outlining the development process used by organisations experienced in developing multiple PtDAs. We then developed an extended model of the development process indicating the various steps on which documentation is required, as well as a checklist to assess the frequency with which each of the elements was publicly reported. Key features common to all patient decision aid (PtDA) development processes include: scoping and design; development of a prototype; ‘alpha’ testing with patients and clinicians in an iterative process; ‘beta’ testing in ‘real life’ conditions (field tests); and production of a final version for use and/or further evaluation. Only about half of the published reports on the development of PtDAs that we reviewed appear to have been field tested with patients, and even fewer had been reviewed or tested by clinicians not involved in the development process. Very few described a distribution strategy, and surprisingly few (17%) described a method for reviewing and synthesizing the clinical evidence. We describe a model development process that includes all the original elements of the original IPDAS criterion, expanded to include consideration of format and distribution plans as well as prototype development. The case for including each of the elements outlined in our model development process is pragmatic rather than evidence-based. Optimal methods for ensuring that each stage of the process is carried out effectively require further development and testing.

Appraisal of the Karnofsky Performance Status and proposal of a simple algorithmic system for its evaluation

Abstract: For over 60 years, the Karnofsky Performance Status (KPS) has proven itself a valuable tool with which to perform measurement of and comparison between the functional statuses of individual patients. In recent decades conditions for patients have changed, and so too has the KPS undergone several adjustments since its initial development. The most important works regarding the KPS tend to focus upon a variety of issues, including but not limited to reliability, validity and health-related quality of life. Also discussed is the question of what quantity the KPS may in fact be said to measure. The KPS is increasingly used as a prognostic factor in patient assessment. Thus, questions regarding if and how it affects survival are relevant. In this paper, we propose an algorithm which uses a minimum of two and a maximum of three questions to facilitate an adequate and efficient evaluation of the KPS. This review honors the original intention of the discoverer and gives an overview of adaptations made in recent years. The proposed algorithm suggests specific updates with the goal of ensuring continued adequacy and expediency in the determination of the KPS.

Is the coverage of google scholar enough to be used alone for systematic reviews

Abstract: In searches for clinical trials and systematic reviews, it is said that Google Scholar (GS) should never be used in isolation, but in addition to PubMed, Cochrane, and other trusted sources of information. We therefore performed a study to assess the coverage of GS specifically for the studies included in systematic reviews and evaluate if GS was sensitive enough to be used alone for systematic reviews. All the original studies included in 29 systematic reviews published in the Cochrane Database Syst Rev or in the JAMA in 2009 were gathered in a gold standard database. GS was searched for all these studies one by one to assess the percentage of studies which could have been identified by searching only GS. All the 738 original studies included in the gold standard database were retrieved in GS (100%). The coverage of GS for the studies included in the systematic reviews is 100%. If the authors of the 29 systematic reviews had used only GS, no reference would have been missed. With some improvement in the research options, to increase its precision, GS could become the leading bibliographic database in medicine and could be used alone for systematic reviews.

Patient-centered medicine and patient-oriented research: improving health outcomes for individual patients

Abstract: Patient-centered medicine is developing alongside the concepts of personalized medicine and tailored therapeutics. The main objective of patient-centered medicine is to improve health outcomes of individual patients in everyday clinical practice, taking into account the patient’s objectives, preferences, values as well as the available economic resources. Patient-centered medicine implies a paradigm shift in the relationship between doctors and patients, but also requires the development of patient-oriented research. Patient-oriented research should not be based on the evaluation of medical interventions in the average patient, but on the identification of the best intervention for every individual patient, the study of heterogeneity and the assignment of greater value to observations and exceptions. The development of information-based technologies can help to close the gap between clinical research and clinical practice, a fundamental step for any advance in this field. Evidence-based medicine and patient centered medicine are not contradictory but complementary movements. It is not possible to practice patient-centered medicine that is not based on evidence, nor is it possible to practice evidence-based medicine at a distance from the individual patient.

Addressing health literacy in patient decision aids

Abstract: Effective use of a patient decision aid (PtDA) can be affected by the user’s health literacy and the PtDA’s characteristics. Systematic reviews of the relevant literature can guide PtDA developers to attend to the health literacy needs of patients. The reviews reported here aimed to assess: 1. a) the effects of health literacy / numeracy on selected decision-making outcomes, and b) the effects of interventions designed to mitigate the influence of lower health literacy on decision-making outcomes, and 2. the extent to which existing PtDAs a) account for health literacy, and b) are tested in lower health literacy populations. We reviewed literature for evidence relevant to these two aims. When high-quality systematic reviews existed, we summarized their evidence. When reviews were unavailable, we conducted our own systematic reviews. Aim 1: In an existing systematic review of PtDA trials, lower health literacy was associated with lower patient health knowledge (14 of 16 eligible studies). Fourteen studies reported practical design strategies to improve knowledge for lower health literacy patients. In our own systematic review, no studies reported on values clarity per se, but in 2 lower health literacy was related to higher decisional uncertainty and regret. Lower health literacy was associated with less desire for involvement in 3 studies, less question-asking in 2, and less patient-centered communication in 4 studies; its effects on other measures of patient involvement were mixed. Only one study assessed the effects of a health literacy intervention on outcomes; it showed that using video to improve the salience of health states reduced decisional uncertainty. Aim 2: In our review of 97 trials, only 3 PtDAs overtly addressed the needs of lower health literacy users. In 90% of trials, user health literacy and readability of the PtDA were not reported. However, increases in knowledge and informed choice were reported in those studies in which health literacy needs were addressed. Lower health literacy affects key decision-making outcomes, but few existing PtDAs have addressed the needs of lower health literacy users. The specific effects of PtDAs designed to mitigate the influence of low health literacy are unknown. More attention to the needs of patients with lower health literacy is indicated, to ensure that PtDAs are appropriate for lower as well as higher health literacy patients.

Clarifying values: an updated review

Abstract: Consensus guidelines have recommended that decision aids include a process for helping patients clarify their values. We sought to examine the theoretical and empirical evidence related to the use of values clarification methods in patient decision aids. Building on the International Patient Decision Aid Standards (IPDAS) Collaboration’s 2005 review of values clarification methods in decision aids, we convened a multi-disciplinary expert group to examine key definitions, decision-making process theories, and empirical evidence about the effects of values clarification methods in decision aids. To summarize the current state of theory and evidence about the role of values clarification methods in decision aids, we undertook a process of evidence review and summary. Values clarification methods (VCMs) are best defined as methods to help patients think about the desirability of options or attributes of options within a specific decision context, in order to identify which option he/she prefers. Several decision making process theories were identified that can inform the design of values clarification methods, but no single “best” practice for how such methods should be constructed was determined. Our evidence review found that existing VCMs were used for a variety of different decisions, rarely referenced underlying theory for their design, but generally were well described in regard to their development process. Listing the pros and cons of a decision was the most common method used. The 13 trials that compared decision support with or without VCMs reached mixed results: some found that VCMs improved some decision-making processes, while others found no effect. Values clarification methods may improve decision-making processes and potentially more distal outcomes. However, the small number of evaluations of VCMs and, where evaluations exist, the heterogeneity in outcome measures makes it difficult to determine their overall effectiveness or the specific characteristics that increase effectiveness.

An investigation of the effect of nurses’ technology readiness on the acceptance of mobile electronic medical record systems

Abstract: Adopting mobile electronic medical record (MEMR) systems is expected to be one of the superior approaches for improving nurses’ bedside and point of care services. However, nurses may use the functions for far fewer tasks than the MEMR supports. This may depend on their technological personality associated to MEMR acceptance. The purpose of this study is to investigate nurses’ personality traits in regard to technology readiness toward MEMR acceptance. The study used a self-administered questionnaire to collect 665 valid responses from a large hospital in Taiwan. Structural Equation modeling was utilized to analyze the collected data. Of the four personality traits of the technology readiness, the results posit that nurses are optimistic, innovative, secure but uncomfortable about technology. Furthermore, these four personality traits were all proven to have a significant impact on the perceived ease of use of MEMR while the perceived usefulness of MEMR was significantly influenced by the optimism trait only. The results also confirmed the relationships between the perceived components of ease of use, usefulness, and behavioral intention in the Technology Acceptance Model toward MEMR usage. Continuous educational programs can be provided for nurses to enhance their information technology literacy, minimizing their stress and discomfort about information technology. Further, hospital should recruit, either internally or externally, more optimistic nurses as champions of MEMR by leveraging the instrument proposed in this study. Besides, nurses’ requirements must be fully understood during the development of MEMR to ensure that MEMR can meet the real needs of nurses. The friendliness of user interfaces of MEMR and the compatibility of nurses’ work practices as these will also greatly enhance nurses’ willingness to use MEMR. Finally, the effects of technology personality should not be ignored, indicating that hospitals should also include more employees’ characteristics beyond socio-demographic profiles in their personnel databases.

Ten years of the International Patient Decision Aid Standards Collaboration: evolution of the core dimensions for assessing the quality of patient decision aids

Abstract: In 2003, the International Patient Decision Aid Standards (IPDAS) Collaboration was established to enhance the quality and effectiveness of patient decision aids by establishing an evidence-informed framework for improving their content, development, implementation, and evaluation. Over this 10 year period, the Collaboration has established: a) the background document on 12 core dimensions to inform the original modified Delphi process to establish the IPDAS checklist (74 items); b) the valid and reliable IPDAS instrument (47 items); and c) the IPDAS qualifying (6 items), certifying (6 items + 4 items for screening), and quality criteria (28 items). The objective of this paper is to describe the evolution of the IPDAS Collaboration and discuss the standardized process used to update the background documents on the theoretical rationales, evidence and emerging issues underlying the 12 core dimensions for assessing the quality of patient decision aids.

Establishing the effectiveness of patient decision aids: key constructs and measurement instruments

Abstract: Establishing the effectiveness of patient decision aids (PtDA) requires evidence that PtDAs improve the quality of the decision-making process and the quality of the choice made, or decision quality. The aim of this paper is to review the theoretical and empirical evidence for PtDA effectiveness and discuss emerging practical and research issues in the measurement of effectiveness. This updated overview incorporates: a) an examination of the instruments used to measure five key decision-making process constructs (i.e., recognize decision, feel informed about options and outcomes, feel clear about goals and preferences, discuss goals and preferences with health care provider, and be involved in decisions) and decision quality constructs (i.e., knowledge, realistic expectations, values-choice agreement) within the 86 trials in the Cochrane review; and b) a summary of the 2011 Cochrane Collaboration’s review of PtDAs for these key constructs. Data on the constructs and instruments used were extracted independently by two authors from the 86 trials and any disagreements were resolved by discussion, with adjudication by a third party where required. The 86 studies provide considerable evidence that PtDAs improve the decision-making process and decision quality. A majority of the studies (76/86; 88%) measured at least one of the key decision-making process or decision quality constructs. Seventeen different measurement instruments were used to measure decision-making process constructs, but no single instrument covered all five constructs. The Decisional Conflict Scale was most commonly used (n = 47), followed by the Control Preference Scale (n = 9). Many studies reported one or more constructs of decision quality, including knowledge (n = 59), realistic expectation of risks and benefits (n = 21), and values-choice agreement (n = 13). There was considerable variability in how values-choice agreement was defined and determined. No study reported on all key decision-making process and decision quality constructs. Evidence of PtDA effectiveness in improving the quality of the decision-making process and decision quality is strong and growing. There is not, however, consensus or standardization of measurement for either the decision-making process or decision quality. Additional work is needed to develop and evaluate measurement instruments and further explore theoretical issues to advance future research on PtDA effectiveness.

Development and evaluation of a de-identification procedure for a case register sourced from mental health electronic records

Abstract: Electronic health records (EHRs) provide enormous potential for health research but also present data governance challenges. Ensuring de-identification is a pre-requisite for use of EHR data without prior consent. The South London and Maudsley NHS Trust (SLaM), one of the largest secondary mental healthcare providers in Europe, has developed, from its EHRs, a de-identified psychiatric case register, the Clinical Record Interactive Search (CRIS), for secondary research. We describe development, implementation and evaluation of a bespoke de-identification algorithm used to create the register. It is designed to create dictionaries using patient identifiers (PIs) entered into dedicated source fields and then identify, match and mask them (with ZZZZZ) when they appear in medical texts. We deemed this approach would be effective, given high coverage of PI in the dedicated fields and the effectiveness of the masking combined with elements of a security model. We conducted two separate performance tests i) to test performance of the algorithm in masking individual true PIs entered in dedicated fields and then found in text (using 500 patient notes) and ii) to compare the performance of the CRIS pattern matching algorithm with a machine learning algorithm, called the MITRE Identification Scrubber Toolkit – MIST (using 70 patient notes – 50 notes to train, 20 notes to test on). We also report any incidences of potential breaches, defined by occurrences of 3 or more true or apparent PIs in the same patient’s notes (and in an additional set of longitudinal notes for 50 patients); and we consider the possibility of inferring information despite de-identification. True PIs were masked with 98.8% precision and 97.6% recall. As anticipated, potential PIs did appear, owing to misspellings entered within the EHRs. We found one potential breach. In a separate performance test, with a different set of notes, CRIS yielded 100% precision and 88.5% recall, while MIST yielded a 95.1% and 78.1%, respectively. We discuss how we overcome the realistic possibility – albeit of low probability – of potential breaches through implementation of the security model. CRIS is a de-identified psychiatric database sourced from EHRs, which protects patient anonymity and maximises data available for research. CRIS demonstrates the advantage of combining an effective de-identification algorithm with a carefully designed security model. The paper advances much needed discussion of EHR de-identification – particularly in relation to criteria to assess de-identification, and considering the contexts of de-identified research databases when assessing the risk of breaches of confidential patient information.

Do personal stories make patient decision aids more effective? A critical review of theory and evidence

Abstract: Patient decision aids support people to make informed decisions between healthcare options. Personal stories provide illustrative examples of others’ experiences and are seen as a useful way to communicate information about health and illness. Evidence indicates that providing information within personal stories affects the judgments and values people have, and the choices they make, differentially from facts presented in non-narrative prose. It is unclear if including narrative communications within patient decision aids enhances their effectiveness to support people to make informed decisions. A survey of primary empirical research employing a systematic review method investigated the effect of patient decision aids with or without a personal story on people’s healthcare judgements and decisions. Searches were carried out between 2005-2012 of electronic databases (Medline, PsycINFO), and reference lists of identified articles, review articles, and key authors. A narrative analysis described and synthesised findings. Of 734 citations identified, 11 were included describing 13 studies. All studies found participants’ judgments and/or decisions differed depending on whether or not their decision aid included a patient story. Knowledge was equally facilitated when the decision aids with and without stories had similar information content. Story-enhanced aids may help people recall information over time and/or their motivation to engage with health information. Personal stories affected both “system 1” (e.g., less counterfactual reasoning, more emotional reactions and perceptions) and “system 2” (e.g., more perceived deliberative decision making, more stable evaluations over time) decision-making strategies. Findings exploring associations with narrative communications, decision quality measures, and different levels of literacy and numeracy were mixed. The pattern of findings was similar for both experimental and real-world studies. There is insufficient evidence that adding personal stories to decision aids increases their effectiveness to support people’s informed decision making. More rigorous research is required to elicit evidence about the type of personal story that a) encourages people to make more reasoned decisions, b) discourages people from making choices based on another’s values, and c) motivates people equally to engage with healthcare resources.

Why mobile health app overload drives us crazy, and how to restore the sanity

Abstract: Smartphones and tablet computers have become an integral part of our lives. One of their key features is the possibility of installing third-party apps. These apps can be very helpful for improving health and healthcare. However, medical professionals and citizens are currently being overloaded with health apps. Consequently, they will have difficulty with finding the right app, and information and features are fragmented over too many apps, thereby limiting their usefulness. In order to combat health app overload, suppliers of apps need to do three things. One, join the open source movement, so that a few apps can work as gateway to medical information by incorporating information from different sources. Two, standardize content, so that the information provided via apps is readable. And third, in order to prevent information overload from occurring within an app, content should be personalized towards an individual’s characteristics and context. Suppliers of medical information and features need to join the open source movement and must make use of standardized medical information formats, in order to allow third parties to create valuable, mobile gateway apps. This can prevent the occurrence of health app overload. By going along in these trends, we can make health apps achieve the impact on healthcare quality and citizens’ health many of us envision.

Recognizing clinical entities in hospital discharge summaries using Structural Support Vector Machines with word representation features

Abstract: Background Named entity recognition (NER) is an important task in clinical natural language processing (NLP) research. Machine learning (ML) based NER methods have shown good performance in recognizing entities in clinical text. Algorithms and features are two important factors that largely affect the performance of ML-based NER systems. Conditional Random Fields (CRFs), a sequential labelling algorithm, and Support Vector Machines (SVMs), which is based on large margin theory, are two typical machine learning algorithms that have been widely applied to clinical NER tasks. For features, syntactic and semantic information of context words has often been used in clinical NER systems. However, Structural Support Vector Machines (SSVMs), an algorithm that combines the advantages of both CRFs and SVMs, and word representation features, which contain word-level back-off information over large unlabelled corpus by unsupervised algorithms, have not been extensively investigated for clinical text processing. Therefore, the primary goal of this study is to evaluate the use of SSVMs and word representation features in clinical NER tasks.

Deployment of assistive living technology in a nursing home environment: methods and lessons learned

Abstract: [Background] With an ever-growing ageing population, dementia is fast becoming the chronic disease of the 21st century. Elderly people affected with dementia progressively lose their autonomy as they encounter problems in their Activities of Daily Living (ADLs). Hence, they need supervision and assistance from their family members or professional caregivers, which can often lead to underestimated psy- chological and financial stress for all parties. The use of Ambient Assistive Living (AAL) technologies aims to empower people with dementia and relieve the burden of their caregivers. The aim of this paper is to present the approach we have adopted to develop and deploy a system for ambient assistive living in an operating nursing home, and evaluate its performance and usability in real conditions. Based on this approach, we emphasise on the importance of deployments in real world settings as opposed to prototype testing in laboratories. [Methods] We chose to conduct this work in close partnership with end-users (dementia patients) and specialists in dementia care (professional caregivers). Our trial was conducted during a period of 14 months within three rooms in a nursing home in Singapore, and with the participation of eight dementia patients and two caregivers. A technical ambient assistive living solution, consisting of a set of sensors and devices controlled by a software platform, was deployed in the collaborating nursing home. The trial was preceded by a pre-deployment period to organise several observation sessions with dementia patients and focus group discussions with professional caregivers. A process of ground truth and system's log data gathering was also planned prior to the trial and a system performance evaluation was realised during the deployment period with the help of caregivers. An ethical approval was obtained prior to real life deployment of our solution. [Results] Patients' observations and discussions allowed us to gather a set of requirements that a system for elders with mild-dementia should fulfil. In fact, our deployment has exposed more concrete require- ments and problems that need to be addressed, and which cannot be identified in laboratory testing. Issues that were neither forecasted during the design phase nor during the laboratory testing surfaced during deployment, thus affecting the effectiveness of the proposed solution. Results of the sys- tem performance evaluation show the evolution of system precision and uptime over the deployment phases, while data analysis demonstrates the ability to provide early detection of the degradation of patients' conditions. A qualitative feedback was collected from caregivers and doctors and a set of lessons learned emerged from this deployment experience. [Conclusion] Lessons learned from this study were very useful for our research work and can serve as inspira- tion for developers and providers of assistive living services. They confirmed the importance of real deployment to evaluate assistive solutions especially with the involvement of professional caregivers. They also asserted the need for larger deployments. Larger deployments will allow to conduct surveys on assistive solutions social and health impact, even though they are time and manpower consuming during their first phases.

Impact of unlinked deaths and coding changes on mortality trends in the Swiss National Cohort

Abstract: Background: Results of epidemiological studies linking census with mortality records may be affected by unlinked deaths and changes in cause of death classification. We examined these issues in the Swiss National Cohort (SNC). Methods: The SNC is a longitudinal study of the entire Swiss population, based on the 1990 (6.8 million persons) and 2000 (7.3 million persons) censuses. Among 1,053,393 deaths recorded 1991–2007 5.4% could not be linked using stringent probabilistic linkage. We included the unlinked deaths using pragmatic linkages and compared mortality rates for selected causes with official mortality rates. We also examined the impact of the 1995 change in cause of death coding from version 8 (with some additional rules) to version 10 of the International Classification of Diseases (ICD), using Poisson regression models with restricted cubic splines. Finally, we compared results from Cox models including and excluding unlinked deaths of the association of education, marital status, and nationality with selected causes of death. Results: SNC mortality rates underestimated all cause mortality by 9.6% (range 2.4% - 17.9%) in the 85+ population. Underestimation was less pronounced in years nearer the censuses and in the 75–84 age group. After including 99.7% of unlinked deaths, annual all cause SNC mortality rates were reflecting official rates (relative difference between �1.4% and +1.8%). In the 85+ population the rates for prostate and breast cancer dropped, by 16% and 21% respectively, between 1994 and 1995 coincident with the change in cause of death coding policy. For suicide in males almost no change was observed. Hazard ratios were only negligibly affected by including the unlinked deaths. A sudden decrease in breast (21% less, 95% confidence interval: 12% - 28%) and prostate (16% less, 95% confidence interval: 7% - 23%) cancer mortality rates in the 85+ population coincided with the 1995 change in cause of death coding policy. Conclusions: Unlinked deaths bias analyses of absolute mortality rates downwards but have little effect on relative mortality. To describe time trends of cause-specific mortality in the SNC, accounting for the unlinked deaths and for the possible effect of change in death certificate coding was necessary.

Coaching and guidance with patient decision aids: A review of theoretical and empirical evidence

Abstract: Background: Coaching and guidance are structured approaches that can be used within or alongside patient decision aids (PtDAs) to facilitate the process of decision making. Coaching is provided by an individual, and guidance is embedded within the decision support materials. The purpose of this paper is to: a) present updated definitions of the concepts “coaching” and “guidance”; b) present an updated summary of current theoretical and empirical insights into the roles played by coaching/guidance in the context of PtDAs; and c) highlight emerging issues and research opportunities in this aspect of PtDA design. Methods: We identified literature published since 2003 on shared decision making theoretical frameworks inclusive of coaching or guidance. We also conducted a sub-analysis of randomized controlled trials included in the 2011 Cochrane Collaboration Review of PtDAs with search results updated to December 2010. The sub-analysis was conducted on the characteristics of coaching and/or guidance included in any trial of PtDAs and trials that allowed the impact of coaching and/or guidance with PtDA to be compared to another intervention or usual care. Results: Theoretical evidence continues to justify the use of coaching and/or guidance to better support patients in the process of thinking about a decision and in communicating their values/preferences with others. In 98 randomized controlled trials of PtDAs, 11 trials (11.2%) included coaching and 63 trials (64.3%) provided guidance. Compared to usual care, coaching provided alongside a PtDA improved knowledge and decreased mean costs. The impact on some other outcomes (e.g., participation in decision making, satisfaction, option chosen) was more variable, with some trials showing positive effects and other trials reporting no differences. For values-choice agreement, decisional conflict, adherence, and anxiety there were no differences between groups. None of these outcomes were worse when patients were exposed to decision coaching alongside a PtDA. No trials evaluated the effect of guidance provided within PtDAs. Conclusions: Theoretical evidence continues to justify the use of coaching and/or guidance to better support patients to participate in decision making. However, there are few randomized controlled trials that have compared the effectiveness of coaching used alongside PtDAs to PtDAs without coaching, and no trials have compared the PtDAs with guidance to those without guidance.

Delivering patient decision aids on the Internet: definitions, theories, current evidence, and emerging research areas

Abstract: Background: In 2005, the International Patient Decision Aids Standards Collaboration identified twelve quality dimensions to guide assessment of patient decision aids. One dimension—the delivery of patient decision aids on the Internet—is relevant when the Internet is used to provide some or all components of a patient decision aid. Building on the original background chapter, this paper provides an updated definition for this dimension, outlines a theoretical rationale, describes current evidence, and discusses emerging research areas. Methods: An international, multidisciplinary panel of authors examined the relevant theoretical literature and empirical evidence through 2012. Results: The updated definition distinguishes Internet-delivery of patient decision aids from online health information and clinical practice guidelines. Theories in cognitive psychology, decision psychology, communication, and education support the value of Internet features for providing interactive information and deliberative support. Dissemination and implementation theories support Internet-delivery for providing the right information (rapidly updated), to the right person (tailored), at the right time (the appropriate point in the decision making process). Additional efforts are needed to integrate the theoretical rationale and empirical evidence from health technology perspectives, such as consumer health informatics, user experience design, and human-computer interaction. Despite Internet usage ranging from 74% to 85% in developed countries and 80% of users searching for health information, it is unknown how many individuals specifically seek patient decision aids on the Internet. Among the 86 randomized controlled trials in the 2011 Cochrane Collaboration’s review of patient decision aids, only four studies focused on Internet-delivery. Given the limited number of published studies, this paper particularly focused on identifying gaps in the empirical evidence base and identifying emerging areas of research. Conclusions: As of 2012, the updated theoretical rationale and emerging evidence suggest potential benefits to delivering patient decision aids on the Internet. However, additional research is needed to identify best practices and quality metrics for Internet-based development, evaluation, and dissemination, particularly in the areas of interactivity, multimedia components, socially-generated information, and implementation strategies.

Using electronic technology to improve clinical care – results from a before-after cluster trial to evaluate assessment and classification of sick children according to Integrated Management of Childhood Illness (IMCI) protocol in Tanzania

Abstract: Background: Poor adherence to the Integrated Management of Childhood Illness (IMCI) protocol reduces the potential impact on under-five morbidity and mortality. Electronic technology could improve adherence; however there are few studies demonstrating the benefits of such technology in a resource-poor settings. This study estimates the impact of electronic technology on adherence to the IMCI protocols as compared to the current paper-based protocols in Tanzania. Methods: In four districts in Tanzania, 18 clinics were randomly selected for inclusion. At each site, observers documented critical parts of the clinical assessment of children aged 2 months to 5 years. The first set of observations occurred during examination of children using paper-based IMCI (pIMCI) and the next set of observations occurred during examination using the electronic IMCI (eIMCI). Children were re-examined by an IMCI expert and the diagnoses were compared. A total of 1221 children (671 paper, 550 electronic) were observed. Results: For all ten critical IMCI items included in both systems, adherence to the protocol was greater for eIMCI than for pIMCI. The proportion assessed under pIMCI ranged from 61% to 98% compared to 92% to 100% under eIMCI (p < 0.05 for each of the ten assessment items). Conclusions: Use of electronic systems improved the completeness of assessment of children with acute illness in Tanzania. With the before-after nature of the design, potential for temporal confounding is the primary limitation. However, the data collection for both phases occurred over a short period (one month) and so temporal confounding was expected to be minimal. The results suggest that the use of electronic IMCI protocols can improve the completeness and consistency of clinical assessments and future studies will examine the long-term health and health systems impact of eIMCI.

Information technology systems in public sector health facilities in developing countries: the case of South Africa

Abstract: Background The public healthcare sector in developing countries faces many challenges including weak healthcare systems and under-resourced facilities that deliver poor outcomes relative to total healthcare expenditure. Global references demonstrate that information technology has the ability to assist in this regard through the automation of processes, thus reducing the inefficiencies of manually driven processes and lowering transaction costs. This study examines the impact of hospital information systems implementation on service delivery, user adoption and organisational culture within two hospital settings in South Africa.

Evaluation of data completeness in the electronic health record for the purpose of patient recruitment into clinical trials: a retrospective analysis of element presence

Abstract: Computerized clinical trial recruitment support is one promising field for the application of routine care data for clinical research. The primary task here is to compare the eligibility criteria defined in trial protocols with patient data contained in the electronic health record (EHR). To avoid the implementation of different patient definitions in multi-site trials, all participating research sites should use similar patient data from the EHR. Knowledge of the EHR data elements which are commonly available from most EHRs is required to be able to define a common set of criteria. The objective of this research is to determine for five tertiary care providers the extent of available data compared with the eligibility criteria of randomly selected clinical trials. Each participating study site selected three clinical trials at random. All eligibility criteria sentences were broken up into independent patient characteristics, which were then assigned to one of the 27 semantic categories for eligibility criteria developed by Luo et al. We report on the fraction of patient characteristics with corresponding structured data elements in the EHR and on the fraction of patients with available data for these elements. The completeness of EHR data for the purpose of patient recruitment is calculated for each semantic group. 351 eligibility criteria from 15 clinical trials contained 706 patient characteristics. In average, 55% of these characteristics could be documented in the EHR. Clinical data was available for 64% of all patients, if corresponding data elements were available. The total completeness of EHR data for recruitment purposes is 35%. The best performing semantic groups were ‘age’ (89%), ‘gender’ (89%), ‘addictive behaviour’ (74%), ‘disease, symptom and sign’ (64%) and ‘organ or tissue status’ (61%). No data was available for 6 semantic groups. There exists a significant gap in structure and content between data documented during patient care and data required for patient eligibility assessment. Nevertheless, EHR data on age and gender of the patient, as well as selected information on his disease can be complete enough to allow for an effective support of the manual screening process with an intelligent preselection of patients and patient data.

User needs elicitation via analytic hierarchy process (AHP). A case study on a Computed Tomography (CT) scanner

Abstract: The rigorous elicitation of user needs is a crucial step for both medical device design and purchasing. However, user needs elicitation is often based on qualitative methods whose findings can be difficult to integrate into medical decision-making. This paper describes the application of AHP to elicit user needs for a new CT scanner for use in a public hospital. AHP was used to design a hierarchy of 12 needs for a new CT scanner, grouped into 4 homogenous categories, and to prepare a paper questionnaire to investigate the relative priorities of these. The questionnaire was completed by 5 senior clinicians working in a variety of clinical specialisations and departments in the same Italian public hospital. Although safety and performance were considered the most important issues, user needs changed according to clinical scenario. For elective surgery, the five most important needs were: spatial resolution, processing software, radiation dose, patient monitoring, and contrast medium. For emergency, the top five most important needs were: patient monitoring, radiation dose, contrast medium control, speed run, spatial resolution. AHP effectively supported user need elicitation, helping to develop an analytic and intelligible framework of decision-making. User needs varied according to working scenario (elective versus emergency medicine) more than clinical specialization. This method should be considered by practitioners involved in decisions about new medical technology, whether that be during device design or before deciding whether to allocate budgets for new medical devices according to clinical functions or according to hospital department.

Analyzing the impact of social factors on homelessness: a Fuzzy Cognitive Map approach

Abstract: The forces which affect homelessness are complex and often interactive in nature. Social forces such as addictions, family breakdown, and mental illness are compounded by structural forces such as lack of available low-cost housing, poor economic conditions, and insufficient mental health services. Together these factors impact levels of homelessness through their dynamic relations. Historic models, which are static in nature, have only been marginally successful in capturing these relationships. Fuzzy Logic (FL) and fuzzy cognitive maps (FCMs) are particularly suited to the modeling of complex social problems, such as homelessness, due to their inherent ability to model intricate, interactive systems often described in vague conceptual terms and then organize them into a specific, concrete form (i.e., the FCM) which can be readily understood by social scientists and others. Using FL we converted information, taken from recently published, peer reviewed articles, for a select group of factors related to homelessness and then calculated the strength of influence (weights) for pairs of factors. We then used these weighted relationships in a FCM to test the effects of increasing or decreasing individual or groups of factors. Results of these trials were explainable according to current empirical knowledge related to homelessness. Prior graphic maps of homelessness have been of limited use due to the dynamic nature of the concepts related to homelessness. The FCM technique captures greater degrees of dynamism and complexity than static models, allowing relevant concepts to be manipulated and interacted. This, in turn, allows for a much more realistic picture of homelessness. Through network analysis of the FCM we determined that Education exerts the greatest force in the model and hence impacts the dynamism and complexity of a social problem such as homelessness. The FCM built to model the complex social system of homelessness reasonably represented reality for the sample scenarios created. This confirmed that the model worked and that a search of peer reviewed, academic literature is a reasonable foundation upon which to build the model. Further, it was determined that the direction and strengths of relationships between concepts included in this map are a reasonable approximation of their action in reality. However, dynamic models are not without their limitations and must be acknowledged as inherently exploratory.

Balancing the presentation of information and options in patient decision aids: an updated review

Abstract: Standards for patient decision aids require that information and options be presented in a balanced manner; this requirement is based on the argument that balanced presentation is essential to foster informed decision making. If information is presented in an incomplete/non-neutral manner, it can stimulate cognitive biases that can unduly affect individuals’ knowledge, perceptions of risks and benefits, and, ultimately, preferences. However, there is little clarity about what constitutes balance, and how it can be determined and enhanced. We conducted a literature review to examine the theoretical and empirical evidence related to balancing the presentation of information and options. A literature search related to patient decision aids and balance was conducted on Medline, using MeSH terms and PubMed; this search supplemented the 2011 Cochrane Collaboration’s review of patient decision aids trials. Only English language articles relevant to patient decision making and addressing the balance of information and options were included. All members of the team independently screened clusters of articles; uncertainties were resolved by seeking review by another member. The team then worked in sub-groups to extract and synthesise data on theory, definitions, and evidence reported in these studies. A total of 40 articles met the inclusion criteria. Of these, six explained the rationale for balancing the presentation of information and options. Twelve defined “balance”; the definition of “balance” that emerged is as follows: “The complete and unbiased presentation of the relevant options and the information about those options—in content and in format—in a way that enables individuals to process this information without bias”. Ten of the 40 articles reported assessing the balance of the relevant decision aid. All 10 did so exclusively from the users’ or patients’ perspective, using a five-point Likert-type scale. Presenting information in a side-by-side display form was associated with more respondents (ranging from 70% to 96%) judging the information as “balanced”. There is a need for comparative studies investigating different ways to improve and measure balance in the presentation of information and options in patient decision aids.

Incorporating patient and family preferences into evidence-based medicine.

Abstract: Clinicians are encouraged to practice evidence-based medicine (EBM) as well as patient-centered medicine. At times, these paradigms seem to be mutually exclusive and difficult to reconcile. It can become even more challenging when trying to include the preferences of the patient’s family members. This paper discusses the basis for this quandary, providing examples of the real-world impact it has on diagnosis-seeking and treatment decision-making behaviors and how it might inform implementation of EBM practices. To further explore the role of friends and family in health-care decision making and to understand how patients and families introduce other considerations that may or may not be congruent with a strictly EBM approach, data from two research studies that examined healthcare–seeking behaviors are presented. Both studies explore how family and friends not only can influence health-care decisions but also may be a source of conflict for the patient and/or clinician. Illness is a biological and social process. Clinicians who engage in EBM need to acknowledge the social and cultural factors that affect the health-care encounter, understand the important role of those factors in health-care decision making, and expand the paradigm of EBM to incorporate sociocultural influences more explicitly. Moreover, recognition of the influences family members and other caregivers have within the clinical encounter—by offering opinions and participating in treatment-related decision making—is needed and could lead to more efficient and effective health care.

An evaluation of data quality in Canada's Continuing Care Reporting System (CCRS): secondary analyses of Ontario data submitted between 1996 and 2011.

Abstract: Evidence informed decision making in health policy development and clinical practice depends on the availability of valid and reliable data. The introduction of interRAI assessment systems in many countries has provided valuable new information that can be used to support case mix based payment systems, quality monitoring, outcome measurement and care planning. The Continuing Care Reporting System (CCRS) managed by the Canadian Institute for Health Information has served as a data repository supporting national implementation of the Resident Assessment Instrument (RAI 2.0) in Canada for more than 15 years. The present paper aims to evaluate data quality for the CCRS using an approach that may be generalizable to comparable data holdings internationally. Data from the RAI 2.0 implementation in Complex Continuing Care (CCC) hospitals/units and Long Term Care (LTC) homes in Ontario were analyzed using various statistical techniques that provide evidence for trends in validity, reliability, and population attributes. Time series comparisons included evaluations of scale reliability, patterns of associations between items and scales that provide evidence about convergent validity, and measures of changes in population characteristics over time. Data quality with respect to reliability, validity, completeness and freedom from logical coding errors was consistently high for the CCRS in both CCC and LTC settings. The addition of logic checks further improved data quality in both settings. The only notable change of concern was a substantial inflation in the percentage of long term care home residents qualifying for the Special Rehabilitation level of the Resource Utilization Groups (RUG-III) case mix system after the adoption of that system as part of the payment system for LTC. The CCRS provides a robust, high quality data source that may be used to inform policy, clinical practice and service delivery in Ontario. Only one area of concern was noted, and the statistical techniques employed here may be readily used to target organizations with data quality problems in that (or any other) area. There was also evidence that data quality was good in both CCC and LTC settings from the outset of implementation, meaning data may be used from the entire time series. The methods employed here may continue to be used to monitor data quality in this province over time and they provide a benchmark for comparisons with other jurisdictions implementing the RAI 2.0 in similar populations.

On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop

Abstract: Background: Clinical decision support (CDS) for electronic prescribing systems (computerized physician order entry) should help prescribers in the safe and rational use of medicines. However, the best ways to alert users to unsafe or irrational prescribing are uncertain. Specifically, CDS systems may generate too many alerts, producing unwelcome distractions for prescribers, or too few alerts running the risk of overlooking possible harms. Obtaining the right balance of alerting to adequately improve patient safety should be a priority. Methods: A workshop funded through the European Regional Development Fund was convened by the University Hospitals Birmingham NHS Foundation Trust to assess current knowledge on alerts in CDS and to reach a consensus on a future research agenda on this topic. Leading European researchers in CDS and alerts in electronic prescribing systems were invited to the workshop. Results: We identified important knowledge gaps and suggest research priorities including (1) the need to determine the optimal sensitivity and specificity of alerts; (2) whether adaptation to the environment or characteristics of the user may improve alerts; and (3) whether modifying the timing and number of alerts will lead to improvements. We have also discussed the challenges and benefits of using naturalistic or experimental studies in the evaluation of alerts and suggested appropriate outcome measures. Conclusions: We have identified critical problems in CDS, which should help to guide priorities in research to evaluate alerts. It is hoped that this will spark the next generation of novel research from which practical steps can be taken to implement changes to CDS systems that will ultimately reduce alert fatigue and improve the design of future systems.

Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention

Abstract: We describe and evaluate the development and use of a Clinical Decision Support (CDS) intervention; an alert, in response to an identified medical error of overuse of a diagnostic laboratory test in a Computerized Physician Order Entry (CPOE) system. CPOE with embedded CDS has been shown to improve quality of care and reduce medical errors. CPOE can also improve resource utilization through more appropriate use of laboratory tests and diagnostic studies. Observational studies are necessary in order to understand how these technologies can be successfully employed by healthcare providers. The error was identified by the Test Utilization Committee (TUC) in September, 2008 when they noticed critical care patients were being tested daily, and sometimes twice daily, for B-Type Natriuretic Peptide (BNP). Repeat and/or serial BNP testing is inappropriate for guiding the management of heart failure and may be clinically misleading. The CDS intervention consists of an expert rule that searches the system for a BNP lab value on the patient. If there is a value and the value is within the current hospital stay, an advisory is displayed to the ordering clinician. In order to isolate the impact of this intervention on unnecessary BNP testing we applied multiple regression analysis to the sample of 41,306 patient admissions with at least one BNP test at LVHN between January, 2008 and September, 2011. Our regression results suggest the CDS intervention reduced BNP orders by 21% relative to the mean. The financial impact of the rule was also significant. Multiplying by the direct supply cost of $28.04 per test, the intervention saved approximately $92,000 per year. The use of alerts has great positive potential to improve care, but should be used judiciously and in the appropriate environment. While these savings may not be generalizable to other interventions, the experience at LVHN suggests that appropriately designed and carefully implemented CDS interventions can have a substantial impact on the efficiency of care provision.

Quality of prenatal and maternal care: Bridging the know-do gap (QUALMAT study): an electronic clinical decision support system for rural Sub-Saharan Africa.

Abstract: Despite strong efforts to improve maternal care, its quality remains deficient in many countries of Sub-Saharan Africa as persistently high maternal mortality rates testify. The QUALMAT study seeks to improve the performance and motivation of rural health workers and ultimately quality of primary maternal health care services in three African countries Burkina Faso, Ghana, and Tanzania. One major intervention is the introduction of a computerized Clinical Decision Support System (CDSS) for rural primary health care centers to be used by health care workers of different educational levels.