Showing papers presented at "American Medical Informatics Association Annual Symposium in 2002"

A security architecture for query tools used to access large biomedical databases.

Abstract: Disseminating information from large biomedical databases can be crucial for research. Often this data will be patient-specific, and therefore require that the privacy of the patient be protected. In response to this requirement, HIPAA released regulations for the dissemination of patient data. In many cases, the regulations are so restrictive as to render data useless for many purposes. We propose in this paper a model for obfuscation of data when served to a client application, that will make it extremely unlikely that an individual will be identified. At Partners Healthcare Inc, with over 1.4 million patients and 400 research clinician users, we implemented this model. Based on the results, we believe that a web-client could be made generally available using the proposed data obfuscation scheme that could allow general usage of large biomedical databases of patient information without risk to patient privacy.

Characteristics and override rates of order checks in a practitioner order entry system.

Abstract: Order checks are important error prevention tools when used in conjunction with practitioner order entry systems. We studied characteristics of order checks generated in a sample of consecutively entered orders during a 4 week period in an electronic medical record at VA Puget Sound. We found that in the 42,641 orders where an order check could potentially be generated, 11% generated at least one order check and many generated more than one order check. The rates at which the ordering practitioner overrode 'Critical drug interaction' and 'Allergy-drug interaction' alerts in this sample were 88% and 69% respectively. This was in part due to the presence of alerts for interactions between systemic and topical medications and for alerts generated during medication renewals. Refinement in order check logic could lead to lower override rates and increase practitioner acceptance and effectiveness of order checks.

Analysis of identifier performance using a deterministic linkage algorithm.

Abstract: As part of developing a record linkage algorithm using de-identified patient data, we analyzed the performance of several demographic variables for making linkages between patient registry records from two hospital registries and the Social Security Death Master File. We analyzed samples from each registry totaling 6,000 record-pairs to establish a linkage gold-standard. Using Social Security Number as the exclusive linkage variable resulted in substantial linkage error rates of 4.7% and 9.2%. The best single variable combination for finding links was Social Security Number, phonetically compressed first name, birth month, and gender. This found 87% and 88% of the links without any false links. We achieved sensitivities of 90% to 92% while maintaining 100% specificity using combinations of social security number, gender, name, and birth date fields. This represents an accurate method for linking patient records to death data and is the basis for a more generalized de-identified linkage algorithm.

Test of the technology acceptance model for the internet in pediatrics.

Abstract: There is growing recognition of the importance of the Internet and, more generally, information technology to pediatric care. However, acceptance of these technologies has been low. Attitudes of physicians can play a pivotal role in the adoption session. This study tests the extension to a widely used model in the information systems literature: the Technology Acceptance Model (TAM). Data were collected in a survey of pediatricians to see how well the extended model, TAM2, fits in the medical arena. Our results partially confirm the model; significant parts of the model were not confirmed. The primary factors in pediatricians' acceptance of technology applications relate to their usefulness and job relevance. Little weight is given to ease of use and social factors. We discuss possible explanations for the discrepancies and suggest future research.

Clinician's assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements.

Abstract: Electronic medical record alerts and reminders are increasingly employed as a means of decreasing medical errors and increasing the quality and cost-effectiveness of care. However, clinicians indicate that alerts and reminders can be either help or hindrance. Discerning the elements that determine which they will be, and the requirements of a helpful alert or reminder, was the focus of this study. We convened three focus groups, comprised of a total of 16 participants. During analysis, five themes emerged: Efficiency, Usefulness, Information Content, User Interface, and Workflow. In addition there were some New Ideas and Surprises. Specific usability and usefulness requirements emerged from within the themes and these are described.

Combat medical informatics: present and future.

Abstract: In this paper, we describe a U.S. Army concept to monitor soldier physiologic status and provide computer-based medical support to increase the likelihood of soldier survival on the battlefield. Supported by an underlying platform of complex wearable computerized systems, the "Warfighter Physiological Status Monitoring" (WPSM) concept consists of an array of biosensors embedded in the soldier's uniform integrated with a database management system and a decision support system that will provide assistance in casualty prevention and casualty management. We discuss the main components of the WPSM, its present status, key requirements and outstanding challenges, and near- and far-term research directions.

A study of abbreviations in MEDLINE abstracts.

Abstract: Abbreviations are widely used in writing, and the understanding of abbreviations is important for natural language processing applications. Abbreviations are not always defined in a document and they are highly ambiguous. A knowledge base that consists of abbreviations with their associated senses and a method to resolve the ambiguities are needed. In this paper, we studied the UMLS coverage, textual variants of senses, and the ambiguity of abbreviations in MEDLINE abstracts. We restricted our study to three-letter abbreviations which were defined using parenthetical expressions. When grouping similar expansions together and representing senses using groups, we found that after ignoring senses where the total number of occurrences within the corresponding group was less than 100, 82.8% of the senses matched the UMLS, covered over 93% of occurrences that were considered, and had an average of 7.74 expansions for each sense. Abbreviations are highly ambiguous: 81.2% of the abbreviations were ambiguous, and had an average of 16.6 senses. However, after ignoring senses with occurrences of less than 5, 64.6% of the abbreviations were ambiguous, and had an average of 4.91 senses.

Computerized physician order entry and communication: reciprocal impacts.

Abstract: Participant observation, focus group and oral history techniques were used to collect data from four distinctly different sites across the U.S. Data were examined initially to identify success factors for computerized physician order entry (CPOE) implementation. These data, reexamined for communication issues, revealed significant impacts on communication channels and relationships unanticipated by the implementers. Effects on physician-nurse interactions, pharmacy roles, and patient communications that vary by time and location were noted. The importance of robust bi-directional information channels between administration and staff was demonstrated to be potentially "mission-critical." The recommendation for implementers is "Plan to be surprised." Careful planning and pre-work are important but, no matter how much an institution prepares for the upheaval of CPOE, unforeseen consequences are inevitable. The presence of a prepared and capable implementation support group is essential.

Accuracy of three classifiers of acute gastrointestinal syndrome for syndromic surveillance.

Abstract: ICD-9-coded emergency department (ED) diagnoses and free-text triage diagnoses are routinely collected data elements that have potential value for public health surveillance and early detection of epidemics. We constructed and measured performance of three classifiers for the detection of cases of acute gastrointestinal syndrome of public health significance: one used ICD-9-coded ED diagnosis as input data; the other two used free-text triage diagnosis. We measured the performance of these classifiers against the expert classification of cases based on review of ED reports. The sensitivity of the ICD-9-code classifier was 0.32, and the specificity was 0.99. The sensitivity of a naive Bayes classifier using triage diagnoses was 0.63, the specificity was 0.94, and the area under the ROC curve was 0.82. A bigram Bayes classifier had sensitivity 0.38, specificity 0.94, and area under the ROC of 0.69. We conclude that a naive Bayes classifier of free-text triage diagnosis data provides more sensitive and earlier detection of cases of acute gastrointestinal syndrome than either a bigram Bayes classifier or an ICD-9 code classifier. The sensitivity achieved should be sufficient for syndromic surveillance system designed to detect moderate to large epidemics.

Exploring text mining from MEDLINE.

Abstract: We present a text mining application that exploits the MeSH heading subheading combinations present in MEDLINE records. The process begins with a user specified pair of subheadings. Co-occurring concepts qualified by these subheadings are regarded as being conceptually related and thus extracted. A parallel process using SemRep, a linguistic tool, also extracts conceptually related concept pairs from the titles of MEDLINE records. The pairs extracted via MeSH and the pairs extracted via SemRep are compared to yield a high confidence subset. These pairs are then combined to project a summary view associated with the selected subheading pair. For each concept the "diversity" in the set of related concepts is assessed. We suggest that this summary and the diversity indicators will be useful a health care practitioner or researcher. We illustrate this application with the subheading pair "drug therapy" and "therapeutic use" which approximates the treatment relationship between Drugs and Diseases.

The SNOMED clinical terms development process: refinement and analysis of content.

Abstract: SNOMED Clinical Terms is a comprehensive concept-based health care terminology that was created by merging SNOMED RT and Clinical Terms Version 3. Following the mapping of concepts and descriptions into a merged database, the terminology was further refined by adding new content, modeling the relationships of individual concepts, and reviewing the hierarchical structure. A quality control process was performed to ensure integrity of the data. Additional features such as subsets, qualifiers, and mappings to other coding systems were added or updated to facilitate usability. We then analyzed the content of the completed work. This paper describes the refinement processes and compares the actual content of SNOMED CT with the early data obtained from analysis of the description mapping process. As predicted, the majority of concepts in SNOMED CT originated from SNOMED RT or CTV3, but not both.

Identification of patient name references within medical documents using semantic selectional restrictions.

Abstract: De-identification of a patient's personal data from medical records is a protective legal requirement imposed before medical documents can be used for research purposes or transferred to other healthcare providers (e.g., teachers, students, tele-consultations). This de-identification process is tedious if performed manually, and is known to be quite faulty in direct search and replace strategies [9]. In this paper, we report on the identification step of this process. The proposed algorithm is based on estimating the fitness of candidate patient name references to a set of semantic selectional restrictions. The semantic restrictions place tight contextual requirements upon candidate words in the report text and are determined automatically from a manually tagged corpus of training reports. Maximum entropy classifiers are used to provide a probabilistic measure of the belief of a given candidate token to a given semantic restriction. We report on the design and preliminary evaluation of the system within the do-main of pediatric urology.

A successful technique for removing names in pathology reports using an augmented search and replace method.

Abstract: The ability to access large amounts of de-identified clinical data would facilitate epidemiologic and retrospective research. Previously described de-identification methods require knowledge of natural language processing or have not been made available to the public. We take advantage of the fact that the vast majority of proper names in pathology reports occur in pairs. In rare cases where one proper name is by itself, it is preceded or followed by an affix that identifies it as a proper name (Mrs., Dr., PhD). We created a tool based on this observation using substitution methods that was easy to implement and was largely based on publicly available data sources. We compiled a Clinical and Common Usage Word (CCUW) list as well as a fairly comprehensive proper name list. Despite the large overlap between these two lists, we were able to refine our methods to achieve accuracy similar to previous attempts at de-identification. Our method found 98.7% of 231 proper names in the narrative sections of pathology reports. Three single proper names were missed out of 1001 pathology reports (0.3%, no first name/last name pairs). It is unlikely that identification could be implied from this information. We will continue to refine our methods, specifically working to improve the quality of our CCUW and proper name lists to obtain higher levels of accuracy.

Theoretical, empirical and practical approaches to resolving the unmet information needs of clinical information system users.

Abstract: We hypothesize that when clinicians review clinical data in an electronic medical record, the information needs that arise are predictable, based on a number of situational factors. Because our theory does not say, exactly, what those needs are, we are using an empirical approach (observation) to detecting and categorizing them. For each need, we can construct an "infobutton" that links the clinical data to an on-line information resource. We have constructed an Infobutton Manager to match the data being reviewed by clinicians with context-appropriate infobuttons. This paper describes how the theory, observations, and practical solutions can come together to improve clinician decision making by resolving information needs.

Automatic extraction of gene and protein synonyms from MEDLINE and journal articles.

Abstract: Genes and proteins are often associated with multiple names, and more names are added as new functional or structural information is discovered. Because authors often alternate between these synonyms, information retrieval and extraction benefits from identifying these synonymous names. We have developed a method to extract automatically synonymous gene and protein names from MEDLINE and journal articles. We first identified patterns authors use to list synonymous gene and protein names. We developed SGPE (for synonym extraction of gene and protein names), a software program that recognizes the patterns and extracts from MEDLINE abstracts and full-text journal articles candidate synonymous terms. SGPE then applies a sequence of filters that automatically screen out those terms that are not gene and protein names. We evaluated our method to have an overall precision of 71% on both MEDLINE and journal articles, and 90% precision on the more suitable full-text articles alone

Using binning to maintain confidentiality of medical data.

Abstract: Biomedical informatics in general and pharmacogenomics in particular require a research platform that simultaneously enables discovery while protecting research subjects' privacy and information confidentiality. The development of inexpensive DNA sequencing and analysis technologies promises unprecedented database access to very specific information about individuals. To allow analysis of this data without compromising the research subjects' privacy, we must develop methods for removing identifying information from medical and genomic data. In this paper, we build upon the idea that binned database records are more difficult to trace back to individuals. We represent symbolic and numeric data hierarchically, and bin them by generalizing the records. We measure the information loss due to binning using an information theoretic measure called mutual information. The results show that we can bin the data to different levels of precision and use the bin size to control the tradeoff between privacy and data resolution.

Role grouping as an extension to the description logic of Ontylog, motivated by concept modeling in SNOMED.

Abstract: Several clinical terminologies now utilize description logic to model the logical definitions of concepts. Recent editions of the Systematized Nomenclature of Medicine (SNOMED) have been developed using the description logic Ontylog. A significant design criterion for SNOMED is to keep concept expressions simple enough to be broadly usable by clinicians, while maintaining faithful representation of concept meaning. Motivated by this criterion, "role grouping" has been developed as an extension to the description logic Ontylog. This paper describes the problems that motivated the creation of role grouping, outlines the semantics of role grouping, illustrates the benefits of this construct with examples from SNOMED Clinical Terms, and provides an algorithm for determining normal forms for expressions involving role groups.

Evaluation of the UMLS as a terminology and knowledge resource for biomedical informatics.

Abstract: Objectives : Terminology and knowledge resources are essential components of interoperability among disparate systems. This paper evaluates whether names and relationships needed in biomedical informatics are present in the UMLS. Methods : Terms for five broad categories of concepts were extracted from LocusLink and mapped to the UMLS Metathesausus. Relationships between gene products and the other four categories (phenotype, molecular function, biological process, and cellular componen t) were searched for in the Metathesaurus. All gene products in the Gene Ontology database were also mapped to the UMLS in order to evaluate its global coverage of the domain. Results : The coverage of concepts ranged from 2% (gene product symbols) to 44% (molecular functions). The coverage of relationships ranged from 60% for Gene productBiological process to 83% for Gene productMolecular function. Discussion : Terminology and ontology issues are discussed, as well as the need for integrating additional resources to the UMLS.

Free-text medical document retrieval via phrase-based vector space model.

Abstract: Many information retrieval systems are based on vector space model (VSM) that represents a document as a vector of index terms. Concepts have been proposed to replace word stems as the index terms to improve retrieval accuracy. However, past research revealed that such systems did not outperform the traditional stem-based systems. Incorporating conceptual similarity derived from knowledge sources should have the potential to improve retrieval accuracy. Yet the incompleteness of the knowledge source precludes significant improvement. To remedy this problem, we propose to represent documents using phrases. A phrase consists of multiple concepts and word stems. The similarity between two phrases is jointly determined by their conceptual similarity and their common word stems. The document similarity can in turn be derived from phrase similarities. Using OHSUMED as a test collection and UMLS as the knowledge source, our experiment results reveal that phrase-based VSM yields a 16% increase of retrieval accuracy compared to the stem-based model.

Robustness of empirical search strategies for clinical content in MEDLINE.

Abstract: BACKGROUND: It is important for clinical end users of MEDLINE to be able to retrieve articles that are both scientifically sound and directly relevant to clinical practice. The use of methodologic search filters (such as "random allocation" for sound studies of medical interventions) has been advocated to improve the accuracy of searching for such studies. Methodologic search filters have been tested in previous MEDLINE files but indexing continues to evolve and the operating characteristics of these search filters in current MEDLINE files are unknown. OBJECTIVE: To determine the robustness of empirical search strategies developed in 1991 for detecting clinical content in MEDLINE in the year 2000. DESIGN: A survey based on a hand search of 171 core health care journals using predetermined quality indicators for scientific merit and clinical relevance. METHODS: 6 trained, experienced research assistants read all issues of 171 journals for the publishing year 2000. Each article was rated using purpose and quality indicators and categorized into clinically relevant original studies, review articles, general papers, or case reports. The original and review articles were then categorized as 'pass' or 'fail' for methodologic rigor in the areas of therapy/quality improvement, diagnosis, prognosis, causation, economics, clinical prediction, and qualitative and review articles. Search strategies developed in 1991 were tested in the 2000 database. MAIN OUTCOME MEASURES: Sensitivity, specificity, precision, and accuracy of the search strategies. RESULTS: Search strategies developed in 1991 generally performed at least as well in 2000 for both best single terms and combinations of terms for high-sensitivity MEDLINE searches for studies of treatment, prognosis, etiology and diagnosis. For example, the accuracy of "clinical trial (pt)" rose from 91.6% to 94.4% (P<0.05) for retrieving high-quality studies of treatments. CONCLUSION: Most MEDLINE search strategies developed in 1991 are robust when searching in the publishing year 2000.

The sublanguage of cross-coverage.

Abstract: At Columbia-Presbyterian Medical Center, free-text "Signout" notes are typed into the electronic record by clinicians for the purpose of cross-coverage. We plan to "unlock" information about adverse events contained in these notes in a subsequent project using Natural Language Processing (NLP). To better understand the requirements for parsing, Signout notes were compared to other common medical notes (ambulatory clinic notes and discharge summaries) on a series of quantitative metrics. They are shorter (mean length 59.25 words vs. 144.11 and 340.85 for ambulatory and discharge notes respectively) and use more abbreviations (26.88% vs. 20.07% and 3.57%). Despite being terser, Signout notes use less ambiguous abbreviations (8.34% vs. 9.09% and 18.02%). Differences were found using Relative Entropy and Squared Chi-square Distance in a novel fashion to compare these medical corpora. Signout notes appear to constitute a unique sublanguage of medicine. The implications for parsing free-text cross-coverage notes into coded medical data are discussed.

Initializing the VA medication reference terminology using UMLS metathesaurus co-occurrences.

Abstract: We developed and evaluated a UMLS Metathesaurus Co-occurrence mining algorithm to connect medications and diseases they may treat. Based on 16 years of co-occurrence data, we created 977 candidate drug-disease pairs for a sample of 100 ingredients (50 commonly prescribed and 50 selected at random). Our evaluation showed that more than 80% of the candidate drug-disease pairs were rated "APPROPRIATE" by physician raters. Additionally, there was a highly significant correlation between the overall frequency of citation and the likelihood that the connection was rated "APPROPRIATE." The drug-disease pairs were used to initialize term definitions in an ongoing effort to build a medication reference terminology for the Veterans Health Administration. Co-occurrence mining is a valuable technique for initializing term definitions in a large-scale reference terminology creation project.

Informatics tools to monitor progress and outcomes of patients with drug resistant tuberculosis in Peru.

Abstract: Multi-drug resistant tuberculosis (MDR-TB) is an important and growing problem in many developing countries. New strategies have been developed to combat the disease but require complex treatment regimens and close monitoring of patients' bacteriology results. We describe a web-based medical record system deployed in Peru to support the management of MDR-TB. Web-based analyses have been developed to track drug sensitivity test results, patterns of sputum smear and culture results and time to conversion from positive to negative cultures. Individual and aggregate drug requirements can also be monitored in real time. Multiple analyses can be linked together and data can be graphed or downloaded to spreadsheets. Over 1200 patients are currently in the system. We argue that such a web-based clinical and epidemiological management system is an important component for successful implementation of complex health interventions in resource poor areas.

Asking questions: information needs in a surgical intensive care unit.

Abstract: Even in the information-rich environment of hospitals, health-care providers face challenges in addressing their various information needs Through a study of a patient-care team in a tertiary care Surgical Intensive Care Unit (SICU), we expanded our understanding of health-care providers' information needs in two important ways First, the study focused on a patient-care team instead of individual health-care providers Second, information needs were examined in a particular organizational setting, the SICU, which had not been previously studied We found that organizational information was extremely important to SICU team members Furthermore, the first resource that team members utilized was not electronic or paper but rather human: another team member

The implementation of a Personal Digital Assistant (PDA) based patient record and charting system: lessons learned.

Abstract: Personal Digital Assistants (PDAs) offer many potential advantages to clinicians. A number of systems have begun to appear for all types of PDAs that allow for the recording and tracking of patient information. PDAs allow information to be both entered and accessed at the point of care. They also allow information entered away from a central repository to be added or "synced" with data through the use of a wireless or wired connection. Few systems, however, have been designed to work in the client/server environment. Even fewer have been designed as point of care additions to already existing enterprise systems. This paper describes the issues encountered in deploying such a system for use in the University of Washington Neonatal Intensive Care Unit (NICU). The lessons learned could be applied to other institutions that will seek to add handheld technology to information systems in the future.

Evaluation of the Diagnostic Accuracy of Chest X-rays Acquired Using a Digital Camera for Low-cost Teleradiology

Abstract: Camera for Low-Cost Teleradiology Agnieszka Szot, MD, MS1"3, Darius Jazayeri, MEng2, Francine L. Jacobson, MD3, Lucila Ohno-Machado, MD, PhD"13, Laura M. Smeaton, MS4, Hamish S. Fraser, MBChB, MSc2 'Decision Systems Group, Brigham and Women's Hospital, 2Partners in Health, Boston, 3Department of Radiology, Brigham and Women's Hospital, 4Center for Biostatistics in AIDS Research, Harvard School ofPublic Health

Impact of a computerized alert during physician order entry on medication dosing in patients with renal impairment.

Abstract: Computerized assistance to clinicians during physician order entry can provide protection against medical errors. However, computer systems that provide too much assistance may adversely affect training of medical students and residents. Trainees may rely on the computer to automatically perform complex calculations and create appropriate orders and are thereby deprived of an important educational exercise. An alternative strategy is to provide a critique at the completion of an order, requiring the trainee to enter the entire order but displaying an alert if an error is made. While this approach preserves the educational components of order-writing, the potential for errors exists if the computerized critique does not induce clinicians to correct the order. The goal of this study was to determine (a) the frequency with which errors are made by trainees in an environment in which renal dosing adjustment calculation for antimicrobials are done by the system after the user has entered an order, and (b) the frequency with which prompts to clinicians regarding these errors leads to correction of those orders.

The Chronus II temporal database mediator.

Abstract: Clinical databases typically contain a significant amount of temporal information. This information is often crucial in medical decision-support systems. Although temporal queries are common in clinical systems, the medical informatics field has no standard means for representing or querying temporal data. Over the past decade, the temporal database community has made a significant amount of progress in temporal systems. Much of this research can be applied to clinical database systems. This paper outlines a temporal database mediator called Chronus II. Chronus II extends the standard relational model and the SQL query language to support temporal queries. It provides an expressive general-purpose temporal query language that is tuned to the querying requirements of clinical decision support systems. This paper describes how we have used Chronus II to tackle a variety of clinical problems in decision support systems developed by our group.

Terminology extraction from text to build an ontology in surgical intensive care.

Abstract: In many medical fields, the maintenance of unabiguous terminologies, the comparison and aggregation of different terminologies go through the building of formal specialized clinical terminologies, the ontologies. In this paper, we describe the building of an ontology in the surgical intensive care medical domain. We considered textual reports as the main source of information and a natural language processing tool, the SYNTEX software, is used to build the ontology. We have tested the possibility for an expert to build a sizeable ontology in a reasonable time. The quality of the ontology has been evaluated according to its capacity to cover the ICD-10 terminology in the field. Examples of coding activity with the ontology are proposed and discussed.

The lexical properties of the gene ontology.

Abstract: The Gene Ontology (GO) is a construct developed for the purpose of annotating molecular information about genes and their products. The ontology is a shared resource developed by the GO Consortium, a group of scientists who work on a variety of model organisms. In this paper we investigate the nature of the strings found in the Gene Ontology and evaluate them for their usefulness in natural language processing (NLP). We extend previous work that identified a set of properties that reliably identifies natural language phrases in the Unified Medical Language System (UMLS). The results indicate that a large percentage (79%) of GO terms are potentially useful for NLP applications. Some 35% of the GO terms were found in a corpus derived from the MEDLINE bibliographic database, and 27% of the terms were found in the current edition of the UMLS.