Showing papers in "Health Services Research in 2006"

Teamwork as an essential component of high-reliability organizations.

Abstract: Organizations are increasingly becoming dynamic and unstable. This evolution has given rise to greater reliance on teams and increased complexity in terms of team composition, skills required, and degree of risk involved. High-reliability organizations (HROs) are those that exist in such hazardous environments where the consequences of errors are high, but the occurrence of error is extremely low. In this article, we argue that teamwork is an essential component of achieving high reliability particularly in health care organizations. We describe the fundamental characteristics of teams, review strategies in team training, demonstrate the criticality of teamwork in HROs and finally, identify specific challenges the health care community must address to improve teamwork and enhance reliability.

Creating High Reliability in Health Care Organizations

Abstract: In the years 1999 and 2001, landmark reports from the Institute of Medicine (IOM) made deficiencies in quality of care and patient safety inescapably visible to health care professionals and the public (Institute of Medicine 1999, 2001). What have we accomplished since these reports? Are we safer; and if so, how do we know? Many say we lack empiric evidence to demonstrate improved safety (Wachter 2004; Brennan et al. 2005; Leape and Berwick 2005), with few measures to broadly evaluate our progress with improvements. Current publicly reported performance measures are likely insufficient for providers to evaluate safety. In many hospitals, these performance measures apply to <10 percent of a hospitals' discharges (Jha et al. 2005). We need scientifically sound and feasible measures of patient safety. In light of these challenges, health care has turned to “high-reliability organizations” (HRO) (e.g., aviation), who achieved a high degree of safety or reliability despite operating in hazardous conditions (Weick and Sutcliffe 2001). Exactly what does reliability mean in health care and how do we know if we are reliable? These answers remain elusive. Reliability is often presented as a defect rate in units of 10 and generally represents the number of defects per opportunity for that defect. In health care, an opportunity for a defect usually translates to a population of patients at risk for the medical error or adverse event. For example, within a health care institution, failure to use evidence-based interventions may occur in five of 10 patients, or a catheter-related blood stream infection (CRBSI) in four of 1,000 catheter days (McGlynn et al. 2003; CDC 2004). A fundamental principle in measuring reliability is focusing on defects that can be validly measured as rates, which is not possible for most patient safety defects. Rates need a clearly defined numerator (defect) and denominator (population at risk) and must be devoid of reporting biases (see framework below). In addition to valid measures, HRO and health care safety experts recognize that the context in which work occurs, called “organizational culture,” has important influences on patient safety (Donald and Canter 1994 Hofmann and Stetzer 1996; Zohar 2000; Barling Loughlin and Kelloway 2002; Hofmann, Morgeson, and Gerras 2003; Sexton, Thomas, and Pronovost 2005). For example, the ability of staff to raise concerns or senior leaders to listen and act on those concerns can influence safety. In health care, communication failures are a leading contributing factor in all types of sentinel events reported to the Joint Commission on the Accreditation of Health Care Organizations (http://www.jcaho.org), with poor communication often occurring between the caregivers who interact most often—physicians and nurses (Sexton, Helmreich, and Thomas 2000). Valid measures of safety climate constructs can be made by systematically eliciting frontline caregivers' perceptions of the organizations commitment to safety (i.e., “safety climate”) using questionnaires (Sexton et al. 2004). Early evidence demonstrates that safety climate is responsive to interventions (Pronovost 2005). As such, strategies to improve reliability must occur in a culture that is conducive to change. A clear framework to measure safety within a health care organization is lacking. Federal agencies, organizations, and some institutions have developed “score cards” or performance measurement reports. However, some have scores of measures and most measures either lack validity (e.g., overall hospital mortality) or target-specific patient populations (e.g., congestive heart failure), preventing generalizability of results to the entire organization (Thomas and Hofer 1999; Hayward and Hofer 2004; Lilford et al. 2004). A comprehensive approach to evaluate an organization's progress with patient safety efforts has not been clearly articulated. In this paper, we describe a comprehensive approach for health care organizations to measure patient safety and then present an example of how this approach was applied to eliminate CRBSIs and improve safety culture in intensive care units (ICUs) in the state of Michigan (Pronovost and Goeschel 2005). Through this improvement example, we hope to highlight the importance of balancing the use of scientifically sound and feasible measures of patient safety with wisdom from front-line staff, noting that both are necessary and equally important.

Operational Failures and Interruptions in Hospital Nursing

Abstract: A growing body of evidence suggests more nursing time per patient results in better patient outcomes (Aiken et al. 2002; Kovner et al. 2002; Needleman et al. 2002). Despite this recognition, increasing patient loads (Aiken, Clarke, and Sloane 2001) and a developing nursing shortage (Buerhaus, Staiger, and Auerbach 2000) make it difficult for nurses to spend as much time with their patients as they would like. To date, much of the discussion regarding nursing time per patient has focused on increasing nurse staffing levels (e.g., Wilson 2004). However, there has been less attention on ensuring that work systems provide supplies, medications, equipment, and information in a timely and accurate fashion. We propose that units lose valuable caregiver time owing to ineffective supply systems, and therefore productive time can be reclaimed by improving work systems. In addition, supply problems interrupt patient care, potentially increasing patient safety risks. We suspect that similar issues affect residents, aids, therapists, and other health care professionals. To explore these propositions, we examined nursing work environments with a particular focus on the systems supplying information, equipment, and materials necessary for patient care. Although specific to nurses, this study may interest other health care professionals who work under similar conditions. We start by reviewing literature describing the content of nursing work. The 2004 Institute of Medicine report characterized the process for planning and managing nursing work as follows: assess patients, identify desired outcomes for these patients, plan and implement treatments to achieve these outcomes, and re-evaluate patients to ensure that the treatments achieved the intended outcomes. As acknowledged in the report, however, this linear description fails to capture complexities inherent in providing patient care. As a result, many newly graduated nurses find that the practice of nursing differs markedly from what they learned in school, and consequently many new nurses leave the profession (Kramer 1974; Godinez et al. 2001; Roberts, Jones, and Lynn 2004). In addition, nurses lament patients' lack of understanding about the nursing role (Ajiboye 2004). Patients often expect more direct care time from their nurses than is possible, resulting in patient dissatisfaction (Staniszewska and Ahmed 1998). This suggests that a more accurate description of nursing would be valuable for both nursing students as well as the public. What makes nursing work complex? One source of complexity lies in the continuously changing conditions of patients for whom nurses care (Benner, Hooper-Kyriakidis, and Stannard 1999). As new information about their patients becomes evident, nurses must solve problems in realtime, often changing which problem they are solving, and where in the problem-solving process they are (Taylor 1997). This requires nurses to modify their planned sequence of care before they can complete a full cycle from initial assessment to posttreatment evaluation (Taylor 1997). Thus, the nature of patient care requires nurses to move among assessment, planning, implementation, and evaluation in a back-and-forth manner rather than sequentially progressing through the steps to completion. A second source of complexity stems from the coordination role that nurses play, ensuring that their patients receive ordered services from other health care workers (e.g., blood tests, radiology tests,and physical therapy) (McCloskey et al. 1996). As an indication of the time nurses spend coordinating care—as opposed to providing patient care—studies find that the average nurse only spends between 31 and 44 percent of her time on direct patient care activities, but between 34 and 49 percent on coordination-related activities (Minyard, Wall, and Turner 1986; Hendrickson, Doddato, and Kovner 1990; Quist 1992; Lundgren and Segesten 2001). Consequently, as nurses conduct their work, they must be continually mindful of what other people are doing (Page 2004). This creates a tightly coupled system (Perrow 1984), increasing the cognitive load on nurses (Beaudoin and Edgar 2003). In summary, factors inherent to caring for patients, such as the need to respond to new information and the need to interact with the larger system of care, increase the complexity of nursing work. In addition to these unavoidable sources of complexity, care is also complicated by avoidable factors unrelated to patients' conditions. Disruptions in the supply of materials or information have received recent attention. For example, when interviewed about productivity, nurses talked about the negative impact of poorly functioning supply systems (McNeese-Smith 1999; Beaudoin and Edgar 2003). Research shows that nurses frequently experience operational failures (Tucker 2004), which are also called hassles (Beaudoin and Edgar 2003) or glitches (Uhlig et al. 2002). These breakdowns interfere with work, reducing employee effectiveness by increasing the time required to complete tasks. One study of nursing work found that, on average, nurses spend 42 minutes of each 8-hour shift resolving operational failures such as missing medications and broken or missing equipment (Tucker 2004). Other studies estimate that nurses spend from 10 percent (Linden and English 1994) to 25 percent (Miller, Deets, and Miller 1997) of their time looking for other staff members. Operational failures can also cause interruptions, as shown in a study that examined interruptions encountered by one hospital nurse during a 10-hour period. This study found that the nurse was interrupted 43 times, including 10 instances when necessary materials, equipment, or personnel were unavailable (Potter et al. 2004). For clarity, in this paper we call these workflow problems “operational failures,” defined as the inability of the work system to reliably provide information, services, and supplies when, where, and to whom needed (Tucker 2004). Examples of operational failures include the pharmacy inadvertently sending only half of the prescribed dose of a patient's medication, broken or missing equipment, and stocked-out supply items. Furthermore, several articles have illustrated the dire consequences for patients—including medication error, procedures carried out on the wrong patient, and hospital-acquired infection—when health care workers received incomplete or incorrect information, services, and items (Bates 2002; Chassin and Becher 2002; Gerberding 2002; Cleary 2003; Volpp and Grande 2003). Given these sources of complexity, it is perhaps not surprising that health care organizations have been characterized as complex adaptive systems, where employees face high levels of uncertainty in their daily work (McDaniel and Driebe 2001). Organizational theorists suggest that under such conditions, resilient employees quickly implement positive adaptive behaviors that are matched to the immediate situation (Mallak 1998). Thus, the ability to flexibly respond to challenging situations, using materials at hand, can enable health care employees to function effectively (Weick 1993). Research suggests that experienced nurses are skilled at responding to changing patient conditions (Benner and Tanner 1987; Hansten and Washburn 2000), as well as compensating for operational failures (Tucker and Edmondson 2003). In this paper, we propose that these two very different sources of complexity warrant different strategies for mitigating potentially negative effects. The first cause is inherent to medical work: new information about a patient becomes evident, triggering a change in caregiver work plans. The inevitability of changing patient conditions suggests that benefits can be gained by designing work to be robust to interruption. The second cause results from faulty work systems: a glitch or error occurs and the caregiver has to compensate for the operational failure. Interruptions due to work system failures are, at least theoretically, avoidable and therefore work systems can be improved to reduce future occurrences. Reducing the time nurses have to spend responding to faulty work systems will allow more time for patient care.

Fair and Just Culture, Team Behavior, and Leadership Engagement: The Tools to Achieve High Reliability

Abstract: Background. Disparate health care provider attitudes about autonomy, teamwork, and administrative operations have added to the complexity of health care delivery and are a central factor in medicine’s unacceptably high rate of errors. Other industries have improved their reliability by applying innovative concepts to interpersonal relationships and administrative hierarchical structures (Chandler 1962). In the last 10 years the science of patient safety has become more sophisticated, with practical concepts identified and tested to improve the safety and reliability of care. Objective. Three initiatives stand out as worthy regarding interpersonal relationships and the application of provider concerns to shape operational change: The development and implementation of Fair and Just Culture principles, the broad use of Teamwork Training and Communication, and tools like WalkRounds that promote the alignment of leadership and frontline provider perspectives through effective use of adverse event data and provider comments. Methods. Fair and Just Culture, Teamwork Training, and WalkRounds are described, and implementation examples provided. The argument is made that they must be systematically and consistently implemented in an integrated fashion. Conclusions. There are excellent examples of institutions applying Just Culture principles, Teamwork Training, and Leadership WalkRounds——but to date, they have not been comprehensively instituted in health care organizations in a cohesive and interdependent manner. To achieve reliability, organizations need to begin thinking about the relationship between these efforts and linking them conceptually.

Who searches the internet for health information

Abstract: Objective. To determine what types of consumers use the Internet as a source of health information. Data Sources. A survey of consumer use of the Internet for health information conducted during December 2001 and January 2002. Study Design. We estimated multivariate regression models to test hypotheses regarding the characteristics of consumers that affect information seeking behavior. Data Collection. Respondents were randomly sampled from an Internet-enabled panel of over 60,000 households. Our survey was sent to 12,878 panel members, and 69.4 percent of surveyed panel members responded. We collected information about respondents' use of the Internet to search for health information and to communicate about health care with others using the Internet or e-mail within the last year. Principal Findings. Individuals with reported chronic conditions were more likely than those without to search for health information on the Internet. The uninsured, particularly those with a reported chronic condition, were more likely than the privately insured to search. Individuals with longer travel times for their usual source of care were more likely to use the Internet for health-related communication than those with shorter travel times. Conclusions. Populations with serious health needs and those facing significant barriers in accessing health care in traditional settings turn to the Internet for health information.

Use of geocoding and surname analysis to estimate race and ethnicity

Abstract: Objective. To review two indirect methods, geocoding and surname analysis, for estimating race/ethnicity as a means for health plans to assess disparities in care. Study Design. Review of published articles and unpublished data on the use of geocoding and surname analyses. Principal Findings. Few published studies have evaluated use of geocoding to estimate racial and ethnic characteristics of a patient population or to assess disparities in health care. Three of four studies showed similar estimates of the proportion of blacks and one showed nearly identical estimates of racial disparities, regardless of whether indirect or more direct measures (e.g., death certificate or CMS data) were used. However, accuracy depended on racial segregation levels in the population and region assessed and geocoding was unreliable for identifying Hispanics and Asians/Pacific Islanders. Similarly, several studies suggest surname analyses produces reasonable estimates of whether an enrollee is Hispanic or Asian/Pacific Islander and can identify disparities in care. However, accuracy depends on the concentrations of Asians or Hispanics in areas assessed. It is less accurate for women and more acculturated and higher SES persons due intermarriage, name changes, and adoption. Surname analysis is not accurate for identifying African Americans. Recent unpublished analyses suggest plans can successfully use a combined geocoding/surname analyses approach to identify disparities in care in most regions. Refinements based on Bayesian methods may make geocoding/surname analyses appropriate for use in areas where the accuracy is currently poor, but validation of these preliminary results is needed. Conclusions. Geocoding and surname analysis show promise for estimating racial/ ethnic health plan composition of enrollees when direct data on major racial and ethnic groups are lacking. These data can be used to assess disparities in care, pending availability of self-reported race/ethnicity data.

Implementing the Institute of Medicine Definition of Disparities: An Application to Mental Health Care

Abstract: In health care, the term “disparities” refers to the unequal treatment of patients on the basis of race or ethnicity, and sometimes on the basis of gender or other patient characteristics. A consensus has emerged that eliminating disparities should be a major goal of health policy, but the empirical and policy literature fails to agree on what a “disparity” is, and how it should be measured. Empirical research often estimates coefficients of race/ethnicity variables without relating these coefficients to an explicit definition of disparity. The recent Institute of Medicine (IOM) report, Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care (IOM 2002), defines a disparity as a difference in treatment provided to members of different racial (or ethnic) groups that is not justified by the underlying health conditions or treatment preferences of patients.1 This definition recognizes the role of socioeconomic differences associated with race/ethnicity as mediators of disparities. To implement the IOM definition, we developed a new method for adjusting for health status that can be used with any model, including nonlinear models that quantify use of health care. We compare the magnitude of disparities estimated by this method to a residual race/ethnicity effect that is often interpreted as meaning a disparity.2

Obtaining Data on Patient Race, Ethnicity, and Primary Language in Health Care Organizations: Current Challenges and Proposed Solutions

Abstract: Numerous studies document that racial and ethnic minorities often receive lower quality care than nonminorities. Although aggregate national data are important, sample sizes often limit their usefulness to only broad racial and ethnic groups. In addition, the data in these surveys may come from records rather than direct interviews of individuals and the information may be based on the observation of the person filling out the record. All these factors leave the quality and consistency of the data questionable. Although much information on health care comes from health care organizations (HCOs) (hospitals, health plans, and medical groups), data on race, ethnicity, and language are often not available or are incomplete (Ver Ploeg and Perrin 2004). In this paper, we focus on the collection of race, ethnicity, and language data by HCOs. Valid and reliable data are fundamental building blocks for identifying differences in care and developing targeted interventions to improve the quality of care delivered to specific population groups. There have been clear calls to action to systematically document disparities and tailor interventions to improve the quality of care. In fact, the drive toward measuring quality is based on the idea that performance measures can help patients, consumers, providers, and purchasers understand what high-quality health care is and increase demand for it. The capacity to measure and monitor quality of care for various racial/ethnic populations rests on the ability both to measure quality of care in general and to conduct similar measurements across different racial/ethnic groups (Fremont and Lurie 2004; Lurie, Jung, and Lavizzo-Mourey 2005). The Institute of Medicine (IOM) Crossing the Quality Chasm report focuses on the quality gap, identifies current practices that impede quality care, and explores how systems approaches can be used to implement change. The subsequent IOM report, Unequal Treatment: Confronting Racial and Ethnic Disparities in Healthcare places its focus on disparities in health care and concludes that racial and ethnic minorities are less likely to receive even routine medical procedures and experience lower quality health care. Further, these two reports and others urge collecting data on patient race, ethnicity, and language. The report by the National Research Council of the National Academies, Eliminating Health Disparities: Measurement, and Data Needs, speaks directly to the importance of collecting valid and reliable data to reduce disparities and improve quality. Reflecting this mandate, efforts are underway by America's Health Insurance Plans (AHIP) to improve the collection of race, ethnicity, and primary language data in health plans and by the Health Research and Educational Trust (HRET), the research and educational affiliate of the American Hospital Association, to improve data collection in hospitals. The public appears to support collecting this information. Attempts to eliminate the collection of race and ethnicity data in California under Proposition 54 were soundly defeated when opposition arose (Torrassa 2003). A national survey (Robert Wood Johnson Foundation 2003) of adults found that over 50 percent of the respondents favor legislation allowing race/ethnicity data collection when told of its benefits. A recent study conducted by the authors (Baker et al. 2005, 2006) of patients' attitudes towards health care providers collecting information about their race and ethnicity found that 80 percent agreed that health care providers should collect information on patients' race and ethnicity, but many felt uncomfortable giving this information. We discuss findings from this study in more detail in the section entitled “Barriers to Collecting Data Directly from Patients.” In this article, we provide an overview of why HCOs should collect race, ethnicity, and language data and review current practices. We discuss the rationale for collecting this information directly from patients and/or enrollees (i.e., self-report), describe foreseeable obstacles, and explicate the mechanics, even the art, of overcoming them. We propose recommendations for standardizing data collection practices and discuss policy implications.

Organizational silence and hidden threats to patient safety.

Abstract: Organizational silence refers to a collective-level phenomenon of saying or doing very little in response to significant problems that face an organization. The paper focuses on some of the less obvious factors contributing to organizational silence that can serve as threats to patient safety. Converging areas of research from the cognitive, social, and organizational sciences and the study of sociotechnical systems help to identify some of the underlying factors that serve to shape and sustain organizational silence. These factors have been organized under three levels of analysis: (1) individual factors, including the availability heuristic, self-serving bias, and the status quo trap; (2) social factors, including conformity, diffusion of responsibility, and microclimates of distrust; and (3) organizational factors, including unchallenged beliefs, the good provider fallacy, and neglect of the interdependencies. Finally, a new role for health care leaders and managers is envisioned. It is one that places high value on understanding system complexity and does not take comfort in organizational silence.

Development of an easy-to-use Spanish Health Literacy test.

Abstract: Objective. The study was intended to develop and validate a health literacy test, termed theShortAssessmentofHealthLiteracyforSpanish-speakingAdults (SAHLSA), for the Spanish-speaking population. Study Design. The design of SAHLSA was based on the Rapid Estimate of Adult Literacy in Medicine (REALM), known as the most easily administered tool for assessing health literacy in English. In addition to the word recognition test in REALM, SAHLSA incorporates a comprehension test using multiple-choice questions designed by an expert panel. Data Collection. Validation of SAHLSA involved testing and comparing the tool with other health literacy instruments in a sample of 201 Spanish-speaking and 202 English-speaking subjects recruited from the Ambulatory Care Center at UNC Health Care. Principal Findings. With only the word recognition test, REALM could not differentiate the level of health literacy in Spanish. The SAHLSA significantly improved the differentiation. Item response theory analysis was performed to calibrate the SAHLSA and reduce the instrument to 50 items. The resulting instrument, SAHLSA-50, was correlated with the Test of Functional Health Literacy in Adults, another health literacy instrument, at r 5 0.65. The SAHLSA-50 score was significantly and positively associated with the physical health status of Spanish-speaking subjects (po.05), holding constant age and years of education. The instrument displayed good internal reliability (Cronbach’s a 5 0.92) and test–retest reliability (Pearson’s r 5 0.86). Conclusions. The new instrument, SAHLSA-50, has good reliability and validity. It could be used in the clinical or community setting to screen for low health literacy among Spanish speakers.

Quality Improvement Implementation and Hospital Performance on Quality Indicators

Abstract: Significant opportunities exist for improving the quality of care delivered in U.S. hospitals. As many as one-fourth of hospital deaths might be preventable; nearly 180,000 people die each year partly as a result of iatrogenic conditions. Moreover, as much as one-third of some hospital procedures expose patients to risks without improving their health; one-third of drugs prescribed are not indicated; and one-third of laboratory tests showing abnormal results do not get followed up by clinicians (Dubois and Brook 1988; Brook et al. 1990; Leape 1994; Institute of Medicine 2000). Many believe that quality improvement (QI) represents a promising strategy for improving hospital quality of care. QI is a systemic approach to planning and implementing continuous improvement in performance. QI emphasizes continuous examination and improvement of work processes by teams of organizational members trained in basic statistical techniques and problem solving tools and empowered to make decisions based on their analysis of the data. The systemic focus of QI complements a growing recognition in the field that the quality of the care delivered by clinicians depends substantially on the performance capability of the organizational systems in which they work. While individual clinician competence remains important, many increasingly see the capability of organizational systems to prevent errors, coordinate care among settings and practitioners, and ensure that relevant, accurate information is available when needed as critical elements in providing high quality care (Institute of Medicine 2000). Reflecting the growing emphasis on organizational systems of care, the Joint Commission on Accreditation of Healthcare Organizations, the National Committee for Quality Assurance, and the Peer Review Organizations of the Centers for Medicare and Medicaid have all encouraged hospitals to use QI methods. Although QI holds promise for improving quality of care, hospitals that adopt QI often struggle with its implementation (Shortell, Bennett, and Byck 1998). By implementation, we refer to the transition period, following a decision to adopt a new idea or practice, when intended users put that new idea or practice into use—for example, when clinical and nonclinical staff begin applying QI principles and practices to improve clinical care processes (Klein and Sorra 1996; Rogers 2003). Successful implementation is critical to the effectiveness of a QI initiative (Blumenthal and Kilo 1998; Shortell, Bennett, and Byck 1998). However, QI implementation is demanding on individuals and organizations. It requires sustained leadership, extensive training and support, robust measurement and data systems, realigned incentives and human resources practices, and cultural receptivity to change (Shortell, Bennett, and Byck 1998; Ferlie and Shortell 2001; Institute of Medicine 2001; Meyer et al. 2004). In addition, the systemic nature of many quality problems implies that the effectiveness of a QI initiative may depend on its implementation across many conditions, disciplines, and departments. This too often proves challenging (Gustofson et al. 1997; Blumenthal and Kilo 1998; Meyer et al. 2004). If successful, though, implementing QI in this manner may create a durable infrastructure for enhancing quality organization-wide. In the present study, we examine the association of several dimensions of QI implementation in hospitals and hospital performance on selected indicators of clinical quality. To do so, we combine data from a national survey of hospital QI practices with a group of carefully screened and validated measures indicative of patient safety in hospital settings. In taking this approach, we seek to address several problems associated with existing research on hospital QI and quality of care. First, previous studies do not adequately account for differences in how hospitals implement QI. Consequently, the relative advantage of different implementation strategies remains unknown. Second, previous studies of hospital QI typically make use of small samples. This restricts the generalization of study findings to larger populations of hospitals and limits the extent to which study findings could be used to develop managerial or policy recommendations. Finally, data availability has precluded previous studies from examining a broad range of hospital quality indicators. This, in turn, has limited the opportunity to link specific QI structures and practices with a set of quality indicators that broadly reflect quality at the institutional level. Study results will provide policy makers, accrediting bodies, and consumers with more precise information about how different approaches to QI implementation in hospital settings relate to a range of hospital-level quality indicators. Such information would facilitate the development of QI standards and benchmarks that make use of hospital-level quality indicators that are not only widely available, but also potentially amenable to change through the systematic application of QI practices. For instance, information on QI practices could be useful in the design of financial incentive programs to “make quality pay,” such as the recent CMS program to reward hospitals scoring in the top 10th percentile on various measures. Finally, such information would help hospital managers and clinicians target those approaches to QI implementation that provide the greatest value for resources expended.

Rural Residents with Disabilities Confront Substantial Barriers to Obtaining Primary Care

Abstract: Rural residents often confront significant barriers when seeking health care, including limited numbers of primary care and specialist physicians nearby, the absence of sophisticated inpatient and diagnostic services, lack of public transportation, and inadequate or absent health insurance coverage, compounded by widespread poverty, low rates of employment-related health insurance, and fragile socioeconomic infrastructures (Ricketts 1999, 2005; Auchincloss and Hadden 2002; Gamm et al. 2002; Hart et al. 2002; Moscovice and Stensland 2002; Slifkin 2002; Arcury, Gesler et al. 2005; Goins et al. 2005; Larson and Hill 2005). Given the nature of these well-documented impediments, certain subpopulations or rural residents, such as elderly individuals (Goins et al. 2005) and ethnic minorities (Glover et al. 2004; Probst et al. 2004), may face considerable hurdles when seeking health care services. We wondered about the experiences of a subpopulation that might be especially disadvantaged by physical, economic, and health care delivery system barriers: working-age, community dwelling rural residents with disabilities. In general, persons with disabilities report lower satisfaction with their health care than do others (Rosenbach 1995; Rosenbach, Acamache, and Khandker 1995; Iezzoni, Davis et al. 2002, 2003, 2004; Jha et al. 2002). Many reasons might explain such findings, including the greater need for complex health services, inadequate communication with clinicians, problematic attitudes of health care professionals and office staff toward disabling conditions, written health information in inaccessible formats, physically inaccessible care settings, and difficulties obtaining reliable transportation to health care facilities. In addition, compared with nondisabled populations, persons with disabilities on average face considerable socioeconomic disadvantages, such as higher rates of poverty and unemployment, lower educational attainment, and comparable to slightly higher rates of missing or inadequate health insurance (Hanson et al. 2003; Harris Interactive 2004; Kaiser Family Foundation 2004; Iezzoni and O'Day 2006). These problems compromise the health care experiences of persons with disabilities even in communities with extensive resources (Reis et al. 2004; Iezzoni and O'Day 2006). Sparse health care options in some rural communities could exacerbate such difficulties. A limited literature suggests that rural residents with disabilities do have more problems with their health care than do nondisabled individuals (Lishner et al. 1996). However, relatively little in-depth information exists about how working-age, community dwelling rural residents with disabilities perceive their health care experiences. “Understanding the perspective of the individual … is a key component to forming a complete picture of rural health care access” (Goins et al. 2005, p. 207). Learning these individual perceptions is critical to crafting patient-centered solutions—reforms that consider persons' values, preferences, and expectations (Institute of Medicine 2001; Berwick 2002). To begin exploring this issue, we conducted four focus groups in 2000–2001 with working-age adults with diverse disabilities living in two rural areas.

Failure of ICD-9-CM codes to identify patients with comorbid chronic kidney disease in diabetes.

Abstract: Objective To determine prevalence of chronic kidney disease (CKD) in patients with diabetes, and accuracy of International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes to identify such patients.

Improving Patient Safety in Hospitals: Contributions of High-Reliability Theory and Normal Accident Theory

Abstract: The Institute of Medicine (IOM) report To Err Is Human introduced many patient safety advocates to the idea of developing hospitals into high-reliability organizations (HROs) (Kohn, Corrigan, and Donaldson 2000). The HRO model is appealing, in part, because it helps health care organizations incorporate lessons learned from high hazard industries, such as aviation and nuclear power. In contrast, normal accident theory (NAT), another research perspective that examines similar industries, did not receive such widespread attention from the health care sector. Although high reliability theory (HRT) and NAT were first cast as competing perspectives, they are now considered complementary (Perrow 1999a; Weick 2004). The two sets of HRT and NAT assumptions, concepts, and empirical predictions are best viewed as providing distinctive frames for understanding patient safety (Weick 2004).1 HRT and NAT are bodies of theory, research, and recommendations for practice and policy that evolved essentially in parallel. Hence, there are instances where these approaches diverge in their assumptions and in the organizational features they treat as critical, rather than offering competing hypotheses. Each frame poses significant questions and offers valuable insights into the pursuit of patient safety. Previous studies compared the two perspectives by applying them to disasters (e.g., Roberts 1990) or near disasters (e.g., Sagan 1993), but we apply them to five popular patient safety practices. We aim to identify distinctive contributions that HRT and NAT make to understanding the organizational conditions affecting patient safety in hospitals and the prospects for transforming hospitals into HROs. To accomplish this, like Snook (2000) we expand NAT beyond its original system-level focus to include processes and interactions among units and individuals. Moreover, we apply NAT to understanding incidents and component failure accidents in hospitals, not just to system accidents.

Benefit-cost in the California treatment outcome project: does substance abuse treatment "pay for itself"?

Abstract: Objective. To examine costs and monetary benefits associated with substance abuse treatment. Data Sources. Primary and administrative data on client outcomes and agency costs from 43 substance abuse treatment providers in 13 counties in California during 2000‐ 2001. Study Design. Using a social planner perspective, the estimated direct cost of treatmentwascomparedwiththe associatedmonetarybenefits,includingtheclient’scostsof medical care, mental health services, criminal activity, earnings, and (from the government’s perspective) transfer program payments. The cost of the client’s substance abuse treatmentepisode wasestimatedbymultiplyingthe number ofdaysthatthe clientspent in each treatment modality by the estimated average per diem cost of that modality. Monetary benefits associated with treatment were estimated using a pre‐posttreatment admission study design, i.e., each client served as his or her own control. Data Collection. Treatment cost data were collected from providers using the Drug Abuse Treatment Cost Analysis Program instrument. For the main sample of 2,567 clients, information on medical hospitalizations, emergency room visits, earnings, and transfer payments was obtained from baseline and 9-month follow-up interviews, and linked to information on inpatient and outpatient mental health services use and criminal activity from administrative databases. Sensitivity analyses examined administrative data outcomes for a larger cohort (N 56,545) and longer time period (1 year). Principal Findings. On average, substance abuse treatment costs $1,583 and is associated with a monetary benefit to society of $11,487, representing a greater than 7:1 ratioofbenefitstocosts.Thesebenefitswereprimarilybecauseofreducedcostsofcrime and increased employment earnings. Conclusions. Even without considering the direct value to clients of improved health and quality of life, allocating taxpayer dollars to substance abuse treatment may be a wise investment.

Impact of institutional review board practice variation on observational health services research.

Abstract: Ethical oversight of research involving human subjects is essential in order to insure that the values of respect for persons, beneficence, and social justice (United States Department of Health & Human Services 1978) are maintained. That function is currently served by the Institutional Review Board (IRB) system, based on the prospective and ongoing local review of the proposed research at every site involved in the conduct of a given project. Many papers critical of current IRB procedures have been written in the past decade. Criticisms include: that IRBs are generally ill equipped to review social science research (American Association of University Professors 2000), resulting in barriers to the effective conduct of such research; that IRB members do not use a systematic way of assessing the risk/benefit ratio when evaluating protocols (Reynolds 2002a); that IRB decisions may frequently be based more on institutional risk aversion than on subject risk and adequate protection (Rogers et al. 1999); that IRBs are more concerned with the content of the consent document than with the consent process (Lynn, Johnson, and Levine 1994); and that IRBs are typically made up of researchers and physicians who are biased toward quantitative research (Tod, Nicolson, and Allmark 2002). The high degree of inconsistency across IRBs, which delays and complicates multicenter studies, has long been observed (Benson 1989; Lux, Edwards, and Osborne 2000; Burman et al. 2001; Silverman, Hull, and Sugarman 2001; Stair et al. 2001; Hirshon et al. 2002). While some have called for centralizing the IRB process to reduce variability, delays, and duplication of effort (Edgar and Rothman 1995; Christian et al. 2002), and to allow national-level discussion of difficult ethical issues (Lind 1992) and “moral consistency” (Moreno 1998), others focus on the advantages of local review (e.g., familiarity with locally relevant issues pertinent to human subjects) (Freedman 1994; Moreno 1998; Levine 2000; Reynolds 2002b). Observational health services research is particularly sensitive to the issues arising from multiple IRB reviews. In order to be generalizable, research on health care delivery, physician practice patterns, and other health care systems issues must involve many and widely varying practice settings. As a result, observational health services research studies almost invariably undergo multiple reviews in the current local-IRB system. However, observational research budgets are typically very modest compared with clinical trials and are often unable to absorb the delays and unexpected expenses that can arise from multiple resubmissions and conflicting reviews. Wolf, Croughan, and Lo (2002) discuss the challenges of human subjects protection in multisite observational research, in the context of practice-based research networks. They point out that “… much of practice-based research has involved medical record review, interviews, or surveys. These types of research customarily present minimal risk provided that informed consent is appropriately obtained and confidentiality is protected. Such research therefore should require less scrutiny than multisite clinical trials of unproven interventions.” They recommend that articles should be published clarifying “how regulations developed for clinical intervention research may not fit practice-based research … and suggest[ing] how IRB policies or federal regulations need to be revised.” Other studies have provided case examples of the variability and delays associated with multisite IRB reviews (While 1995; Lux, Edwards, and Osborne 2000; Silverman, Hull, and Sugarman 2001; Stair et al. 2001; Hirshon et al. 2002). Two of these involved randomized clinical trials (Silverman, Hull, and Sugarman 2001; Stair et al. 2001), two involved observational health services research (While 1995; Hirshon et al. 2002), and the type of research involved in the fifth was not described (Lux, Edwards, and Osborne 2000). Of the two involving observational health services research, only one discusses the reasons for the delays and the nature of the variable responses, and that study involved IRB review at only three sites. Studies of the IRB review process in multisite observational health services research using larger samples and providing more detailed enumeration of the components of delay and variation are needed in order to make informed recommendations for change. This study undertakes to do so.

The Effect of Rural Hospital Closures on Community Economic Health

Abstract: Hospitals are generally considered to be the locus of rural health care systems. Not only are important health services based at hospitals, but many of a community's health care personnel are either directly employed by or supported by the local hospital. Further, hospitals are often considered vital to local economies as they bring outside dollars into the communities via third-party payors, provide jobs, stimulate local purchasing, and help attract industry and retirees (Doeksen et al. 1997). As such, the closure of a hospital can have detrimental effects on a rural community. The rapid succession of hospital closures throughout the 1980s and 1990s helped stimulate legislation, such as creation of Critical Access Hospitals (hospitals that accept certain restrictions and are reimbursed 101 percent of cost from Medicare), designed to ensure the financial viability of small rural hospitals. The number of small rural hospitals that have chosen to convert to CAH status has risen beyond expectations; as on August 2004, 959 small rural hospitals (over 40 percent of all rural hospitals) have opted out of Prospective Payment System by converting (Flex Monitoring Team 2005). In some policy circles, concern has been expressed about the effect on the Prospective Payment System of so many hospitals taking advantage of the protection of cost-based reimbursement (MEDPAC 2003). In light of these concerns, this is an opportune time to more accurately assess the economic importance of small rural hospitals to their communities, and to estimate the potential impact of their closure, should favorable reimbursement policies be changed. The effect of hospital closures on the health of community members has been relatively well documented and is not the focus of this study. For example, Reif, Des Harnais, and Bernard (1999) study six communities experiencing a hospital closure and conclude that hospital closures decrease access to health care, whereas Rosenbach and Dayhoff (1995) find that per-capita Medicare expenditures grew at a slower rate in communities experiencing a closure. Fleming et al. (1995) find that residents of communities with a hospital closure experienced a mean increase in travel time to care of about 30 minutes. Rather, we are concerned with the relationship between a hospital closure and the local economic conditions before and after the closure. In general, hospital closure is perceived to have negative economic effects on a rural community (Hart, Pirani, and Rosenblatt 1991a), although few studies have directly measured the effect. A number of studies have attempted to estimate the role of hospitals in their local economies as evidence of the direct and indirect impact a closure would have, by either comparing the closure communities' economies to those of control groups, or through input/output (I/O) analysis. In one of the earliest studies, Christianson and Faulkner modeled the contribution of rural hospitals to local economies and found an estimated $686,405 to $1,083,282 (US$ in 1978) in community income was generated directly and indirectly by hospital expenditures; income multiplier estimates were less than 2 for 90 percent of the communities (Christianson and Faulkner 1981). McDermott et al. (1991) used hospital survey data to estimate the economic impact of a hospital on its host community and found that the combined induced and direct effects, on average, were $54,739 per hospital bed (1991). Studies using I/O analysis, which uses observed data on business and consumer purchase patterns to estimate the direct and indirect/induced effects of a change in one sector of the economy on others, have found similar results. For example, Doeksen, Gerald, and Altobelli (1990) simulated the effect of a hospital closure in rural Oklahoma and estimated that over a 5-year period approximately 78 jobs, $1.7 million in income, $452,100 in retail sales, and $9,100 in sales tax revenue would be lost because of the closure. Similar conclusions were reached using data from three Texas communities (Doeksen, Loewen, and Strawn 1990). Cordes et al. (1999) extended the literature by examining the role of the hospital in the economy and differentiating hospitals by bed size. They found that the estimated economic multipliers increased in magnitude with hospital bed size, but did not specifically estimate the effect of closure using I/O analysis. While each of these studies suggests that a hospital closure would have negative economic consequences for rural communities, other research has indicated little to no effect on the rural community because of hospital closure. Pearson and Tajalli (2003) found that hospital closure does not appear to cause short- or long-term harm to the economies of their rural host communities. Their findings were based on a pretest/posttest model of data for 24 Texas rural communities where a hospital closed and a group of control communities. Five economic indicators were examined for trends and none were found to have had a statistically significant change following closure of the hospital. Similarly, Probst et al. (1999) compared economic indicators in closure communities to a control group of nonclosure communities and failed to find a statistically significant difference in income trends in the closure counties relative to the comparison counties. Stensland et al. (2002) examined the effect of 42 hospital closures in rural Appalachian communities and concluded that the closure had no effect on short-term or long-term economic growth of those areas. Predominantly, the literature on the economic effects of hospital closures has relied on I/O analysis. Whereas I/O analysis has been useful in furthering the methodology of measuring hospital closure effects to include spending induced by the hospital business, the technique is limited in many ways. First, it is not designed to calculate “amenity” effects of a hospital closure—the absence of a local hospital may discourage retirees and businesses from moving into the community. Secondly, because of a lack of data on these small rural markets, I/O analysis for rural areas often relies on national purchasing trends, rather than local purchasing patterns, to calculate economic multipliers. Third, I/O treats the study region (often a county) as an isolated economy and tends to ignore market area considerations, which may lead to over- or under-estimation of the effects. Finally, I/O analysis does not offer measures of precision in the estimates. The concept of standard errors (SEs) is critical in ascertaining the degree of confidence one has in the results, and I/O has no such ability. In this paper, we estimate the effect of hospital closure on the local economy using multivariate regression methods that do not require the use of a control group consisting of communities not experiencing a hospital closure. We posit that the closure of a hospital negatively affects the economic health of a community, and we extend the hospital closure literature in two new dimensions. First, we differentiate between the impact of a hospital closure in a community where another hospital remains open and closure in a community with no other proximal access to hospital services. This distinction is important because many of the ways that a closure can affect local economies, such as the amenity effect, can be mitigated by the presence of a near-by alternative hospital. Second, our analysis considers whether the economic conditions in communities where a hospital has closed can be attributed to the closure, or whether poor economic conditions preceded (and perhaps contributed to) the closure. Our methodology allows this assessment without the necessity of identifying appropriate controls, a difficult task as there may be intrinsic differences between financially struggling communities where hospitals ultimately close and those where they remain open.

Is Some Provider Advice on Smoking Cessation Better Than No Advice? An Instrumental Variable Analysis of the 2001 National Health Interview Survey

Abstract: Despite much increased awareness of the health hazards of smoking, 23 percent of American adult population remain smokers in 2001 (National Center for Health Statistics 2003a). Although the majority (about two-thirds) of smoking adults had some contact with health care providers annually, physicians were advising patients to quit in only one out of five encounters with smokers (Thorndike et al. 1998), suggesting missed opportunities of provider intervention. What adds to the concern, other than the low rate of advice, is that health care providers are more likely to advise patients who are heavier smokers and/or who already have tobacco-related conditions such as heart disease, cancer, and chronic obstructive pulmonary diseases (COPD) (e.g., Fiore et al. 1990; Gilpin et al. 1993; Jaen et al. 1997). By doing so, providers largely miss out on the opportunities of converting lighter smokers for whom quitting might be easier, and of preventing the damages of smoking. Among a number of reasons for which providers fail to provide advice during clinical encounters with smoking patients, the lack of confidence in the effectiveness of their advice has proven to be a great barrier (Demers et al. 1990). This study aims to estimate the effectiveness of provider advice on smoking cessation in the real world under time and other resource constraints and in an unselected patient population. A number of studies based on randomized controlled trials (RCTs) have tested the efficacy of smoking cessation advice delivered by health care providers (e.g., Kottke et al. 1988; Fiore et al. 2000), usually under strictly controlled conditions. The lack of flexibility in the treatment conditions thus makes it hard to generalize the evidence from RCTs to the experience of community practitioners who develop intervention strategies to suit their own practice styles and the needs of their own patients. In addition, clinical trials often adopt restrictive criteria when recruiting participating patients and providers and are usually conducted in nonrepresentative clinical settings such as academic medical centers. Therefore, intervention outcomes as seen in RCTs might not be replicable among the patient population in community settings. Furthermore, what RCTs are designed to identify, namely, the average efficacy of a well-defined intervention across a specific patient population, is usually not adequate to inform the decisions of providers in busy community practices. As providers have to choose between to advise and not to advise when the next smoking patient steps into their office, they could be better informed by knowing how effective it would be if they increase their advice incrementally. To inform policy and practice, this study estimates the effect of health provider advice, delivered in routine care settings, on the success of smoking cessation by their patients, using nationally representative survey data of patients receiving care from their providers. To address the selection in provider advice of smoking cessation in day-to-day practice—for example, heavier smokers and/or patients with smoking-related conditions are more likely to be advised—we use the behavior of the same provider in advising diet and physical activities as instrumental variables (IVs) for smoking cessation advice (see, e.g., McClellan and Newhouse [2000] for an overview of the application of the IV method in health services research and the other articles in the same special issue of Health Services Research 35[5]). We model smoking cessation outcome and provider advice jointly. The effect of “some advice” (versus no advice) is predicted based on the model, and the robustness of the effect size against the crucial assumption for the identification is examined by additional analyses including a simulation exercise.

Racial variation in end-of-life intensive care use: a race or hospital effect?

Abstract: One in five Americans die in the hospital using intensive care unit (ICU) services, and these hospitalizations consume over 80 percent of all terminal hospitalization costs (Angus et al. 2004). Variations in end-of-life ICU admission exist by geography (Wennberg and Cooper 1999) and age (Yu et al. 2000; Levinsky et al. 2001; Angus et al. 2004), but less is known about racial or ethnic differences in ICU use at the end of life (Degenholtz et al. 2003). Paradoxically, racial variations in end-of-life care appear to follow an entirely different pattern than is observed for other medical services in the United States. While minorities, and blacks in particular, generally are treated less intensively than whites (Smedley et al. 2002), including lower rates of invasive cardiac procedures (Ayanian et al. 1993; Whittle et al. 1993), surgical treatment for lung cancer (Bach et al. 1999), and renal transplantation (Epstein et al. 2000), blacks appear to receive higher rates of intensive treatment at the end of life. For example, blacks are more likely to die in the hospital (Pritchard et al. 1998) and less likely to use hospice (Greiner et al. 2003) and have higher overall spending in their last 12 months than whites (Hogan et al. 2001; Levinsky et al. 2001; Shugarman et al. 2004). As dying patients can be considered more or less equivalent in their illness severity (Fisher et al. 2003a,b), studies of end-of-life treatment variations are less subject to confounding by inadequate risk-adjustment methodology. Many scholars have tried to explain these phenomena by citing differences in patient preferences. Indeed, several studies report that blacks and Hispanics prefer more aggressive life-sustaining treatment than whites (Garrett et al. 1993; O'Brien et al. 1995; McKinley et al. 1996; Diringer et al. 2001), and that physicians' preferences for end-of-life treatment follow the same pattern by race as patients' preferences (Mebane et al. 1999). However, we also know that treatment preferences for care at the end of life do not reliably predict actual treatment (Teno et al. 1997; Pritchard et al. 1998), and so it seems implausible that preferences alone drive observed patterns of care. Because minority populations often live in different neighborhoods and access different providers than majority populations, observed differences in treatment patterns may be a function of physician or hospital and not race per se (Kahn et al. 1994; Skinner et al. 2003; Bach et al. 2004; Bradley et al. 2004; Barnato et al. 2005). Indeed, at the hospital referral region, racial differences in end-of-life Medicare spending are driven more by region of residence than by race (Baicker et al. 2004). If dying patients seek care at the nearest hospital rather than at the hospital that provides the type of care patterns they prefer, and minority populations use systematically different hospitals than majority populations due to residential segregation in the United States, then prevailing hospital practice patterns may drive end-of-life treatment intensity rather than patient preferences. Clarifying this issue is critical to inform any policy designed to better match patients' preferences with treatment. In this study we explore the relationship between race/ethnicity and ICU admission and costs among patients who died in the hospital. We asked two questions. Do differences exist? And, if so, can they be explained by provider rather than by race and ethnicity?

Making noncatastrophic health care processes reliable: Learning to walk before running in creating high-reliability organizations.

Abstract: Health care clinicians successfully apply proven medical evidence in common acute, chronic, or preventive care processes less than 80 percent of the time. This low level of reliability at the basic process level means that health care's efforts to improve reliability start from a different baseline from most other industries, and therefore may require a different approach. This paper describes The Institute for Healthcare Improvement's (IHI) current approach to improving health care reliability, including a useful nomenclature for levels of reliability, and a focus on improving reliability of basic health care processes before moving on to more sophisticated high reliability organization concepts. Early IHI work with a community of health care reliability innovators has identified four themes in health care settings that help to explain at least a portion of the gap in process reliability between health care and other industries. These include extreme dependence on hard work and personal vigilance, a focus on mediocre benchmark outcomes rather than process, great tolerance of provider autonomy, and failure to create systems that are specifically designed to reach articulated reliability goals. This paper describes our recommendations for the initial steps health care organizations' might take, based on these four themes, as they begin to move toward higher reliability.

Sensemaking of patient safety risks and hazards.

Abstract: In order for organizations to become learning organizations, they must make sense of their environment and learn from safety events. Sensemaking, as described by Weick (1995), literally means making sense of events. The ultimate goal of sensemaking is to build the understanding that can inform and direct actions to eliminate risk and hazards that are a threat to patient safety. True sensemaking in patient safety must use both retrospective and prospective approach to learning. Sensemaking is as an essential part of the design process leading to risk informed design. Sensemaking serves as a conceptual framework to bring together well established approaches to assessment of risk and hazards: (1) at the single event level using root cause analysis (RCA), (2) at the processes level using failure modes effects analysis (FMEA) and (3) at the system level using probabilistic risk assessment (PRA). The results of these separate or combined approaches are most effective when end users in conversation-based meetings add their expertise and knowledge to the data produced by the RCA, FMEA, and/or PRA in order to make sense of the risks and hazards. Without ownership engendered by such conversations, the possibility of effective action to eliminate or minimize them is greatly reduced.

Patient Turnover and Nursing Staff Adequacy

Abstract: Objective To assess the relative validity of patient turnover adjustments and the difference in nurse staffing using measures that adjust for patient turnover and severity versus those that do not.

Estimating the Effect of Smoking Cessation on Weight Gain: An Instrumental Variable Approach

Abstract: Healthy People 2010 identifies tobacco use and obesity as important focus areas for the United States public health community (U.S. Department of Health and Human Services 2000). The two priorities share an important link. Over the past 25 years, a number of studies have documented the changes in body weight that result from smoking cessation. The research has generally concluded that smoking cessation is associated with a statistically significant increase in body weight among quitters (Coates and Li 1983; Manley and Boland 1983; Albanes et al. 1987; Klesges et al. 1989; Shimokata, Muller, and Andres 1989; Moffart and Owens 1991; Williamson et al. 1991; Flegal et al. 1995; Klesges et al. 1997, 1998; Froom, Melamed, and Benbassat 1998; Mizoue et al. 1998; Froom et al. 1999; Hudmond et al. 1999). Estimates of postcessation weight gain range considerably. A review of the smoking and body weight literature by Klesges et al. (1989) found that estimates of weight gain associated with cessation range from 0.2 to 8.2 kg depending on the sample, study design, and follow-up period. Through their meta-analysis of these studies, the authors concluded that smokers gain an average of 2.9 kg after quitting. In these studies researchers took the basic approach of comparing average weight gain of quitters to that of continuing smokers. Typically the analyses controlled for differences across the two groups with variables including age, sex, and baseline body weight. A problem with this approach is that the two groups are likely to differ in unmeasured variables—such as general concern for health or present versus future orientation—that may be related to weight gain. These differences could confound estimates of the true causal effect of cessation on weight gain. Unmeasured confounders could cause the estimates derived from comparing quitters to continuing smokers to be biased either upwards or downwards. Quitters are known to be less concerned on average about weight gain than continuing smokers, particularly in the case of women (Meyers et al. 1997). This suggests quitters may be more prone to weight gain than continuing smokers, implying that the comparison between the two groups would be biased upwards from the true causal effect. On the other hand, quitters have been shown to be different from continuing smokers in three ways that could produce a downward bias in estimated weight gain. First, a primary motivation for quitting smoking is general concern about health (McCaul et al. 2006). Second, quitters are more future-oriented on average (Bickel, Odum, and Madden 1999). Third, quitters are less impulsive on average (Wurtman 1993; Cinciripini et al. 2003; Terracciano and Costa 2004). All three of these factors could mean that quitters, compared with continuing smokers, are less prone to weight gain. One other factor could produce a bias, but it is not clear in which direction. Quitters tend to be less depressed and exhibit less negative affect at the time of successful quitting than continuing smokers (Cinciripini et al. 2003). These differences in emotional states have been associated in various studies with both weight loss and weight gain (Wurtman 1993; Barefoot et al. 1998). Indeed, the Diagnostic and Statistical Manual of the American Psychiatric Association, 4th Edition (DSM-IV) cites “significant weight loss/gain” as one of the nine symptoms of depression (American Psychiatric Association 1994, p. 161). Depending on their relative magnitudes, the confounding factors described above could produce an overestimate or underestimate of the true effect of cessation on weight gain. It is also theoretically possible that the cofounders offset each other, causing the conventional estimates to be accurate. Thus, we do not know whether previous estimates are too high, too low, or just right. The only way to answer this question definitively is to examine a situation in which people have been randomized to be quitters or continuing smokers, thus ensuring that unmeasured confounders are equal on average across the groups being compared. In this study we adopt a variant of this approach by applying an instrumental variables (IVs) method to previously published randomized cessation trials.

How adults' access to outpatient physician services relates to the local supply of primary care physicians in the rural southeast.

Abstract: Objective To examine how access to outpatient medical care varies with local primary care physician densities across primary care service areas (PCSAs) in the rural Southeast, for adults as a whole and separately for the elderly and poor.

Geographic Variation in the Prescription of Schedule II Opioid Analgesics among Outpatients in the United States

Abstract: Effective management of pain often requires the use of opioid analgesics (American Pain Society 1999, 2002; World Health Organization 2000; American Pain Society 2004). Because of their potential for abuse, opioid analgesics are regulated under federal narcotics and controlled substances laws (Joranson et al. 2000). The Controlled Substances Act of 1970 authorizes the Drug Enforcement Administration to supervise the manufacturing and distribution of legal narcotics and places all substances regulated under existing federal law into one of five schedules. Schedule II is reserved for drugs or substances with (a) high potential for abuse, (b) currently accepted medical use in treatment in the United States, and (c) potential for severe psychological or physical dependence if abused. Currently, 11 oral opioid analgesics have a Schedule II assignment (Drug Facts 2002). In addition to federal regulations, many states have enacted programs to monitor the use of opioid analgesics. Although details vary by state, prescription monitoring programs typically collect prescribing and dispensing data from pharmacies, review and analyze the data, and disseminate information to appropriate law enforcement and regulatory authorities (Joranson et al. 2000). Recent analyses suggest that the use of opioid analgesics has grown considerably over the last decade. From 1990 to 1996, there were steady increases in the use of morphine, fentanyl, oxycodone, and hydromorphone (Joranson et al. 2000). The same pattern persisted from 1997 to 2002, with marked increases in the use of fentanyl and oxycodone (Gilson et al. 2004). Considerable attention has been given to the use and abuse of controlled-release oxycodone hydrochloride (Clancy 2000; Gold 2000; Graettinger 2000; Ordway 2000; Tough 2001). Notably, abuse and diversion of controlled-release oxycodone has been concentrated in certain geographic areas, with abuse in rural Maine, Kentucky, Virginia, and West Virginia bringing national attention to the problem (Clines and Meier 2001; Rogers 2001; Rosenberg 2001; Drug Enforcement Administration 2002). Geographic variations in the use of other prescription medications have been previously examined. In particular, significant geographic variation has been documented in the use of stimulant medication in children (Zito et al. 1997; Wennberg and Wennberg 2000; Cox et al. 2003), antihypertensive medications in the Veterans Affairs health system (Lopez et al. 2004), and lipid lowering drugs, proton pump inhibitors, antianxiety drugs, and antihistamines among adults in Michigan (Wennberg and Wennberg 2000). By contrast, an analysis of medication use for five conditions (depression, asthma, congestive heart failure, rheumatoid/osteoarthritis, and upper respiratory infection) in 11 California regions found relatively little geographic variation (DuBois, Batchlor, and Wade 2002). To date, no study has explored geographic variation in the use of opioid analgesics. Examining geographic variation in the use of opioid analgesics is particularly important given the presence of state policies that may limit the prescription of these drugs. That is, geographic variation may yield important insights about the effects of these state policies. Evidence from the 1989 National Ambulatory Medical Care Survey (NAMCS) suggests that physicians in states with multiple-copy prescription programs are significantly less likely to prescribe opioid analgesics during an office visit (Wastila and Bishop 1996). Although a nationally representative sample, the observed NAMCS sample visits that occurred in states with multiple-copy prescription programs were likely heavily weighted toward states with especially large populations. Consequently, the generalizability of the findings to other states is unclear. Prior work has also identified other factors related to the medical and nonmedical use of abusable prescription drugs. A study using the 1987 National Medical Expenditure Survey found that female gender, age less than 35 years, socioeconomic status, and diagnosis were independently and positively associated with the probability of narcotic analgesic use (Simoni-Wastila 2000). Using the 1991 National Household Survey on Drug Abuse (NHSDA), Simoni-Wastila, Ritter, and Strickler (2004) identified female gender, age less than 35 years, annual income greater than $40,000, poor health status, and use of illicit drugs in the previous year as independent predictors of nonmedical use of prescription drugs. Simoni-Wastila and Strickler (2004) found that female gender and single marital status were positively and independently associated with problem use of narcotic analgesics, whereas age less than 25 years and illicit drug use in the previous year were negatively associated. In the present study, we used a large, outpatient pharmaceutical claims database of commercially insured individuals to build upon prior work in two ways. First, we examined state-level prevalence of and geographic variations in the use of Schedule II oral opioid analgesics. Second, we investigated the influence of prescription monitoring programs and a variety of other factors on county-level claim rates for all opioid analgesics and for controlled-release oxycodone alone. Based on prior work, we hypothesized that the presence of a prescription monitoring program would be negatively and independently associated with claim rates for opioid analgesics, whereas female gender, age less than 35 years, and prior use of illicit drugs would be positively and independently associated.

Does date stamping ICD-9-CM codes increase the value of clinical information in administrative data?

Abstract: Hospital report cards have become an integral part of the evolving health care landscape. One of the principal barriers to the widespread implementation of health outcomes measurement is the cost of collecting the clinical data required for risk adjustment (Chassin and Galvin 1998). To circumvent this problem, many performance-profiling systems rely on administrative rather than clinical data. Third-party payers are making hospital report cards based on such administrative data available to patients. “Hospital Comparison Tools,” which uses administrative data to benchmark hospital performance, is an example of a web-based service offered to members of many major health care plans such as Blue Cross/Blue Shield and Aetna. The Institute of Medicine has identified the importance of these data for assessing health care quality in Envisioning the National HealthCare Quality Report, by stating that “Administrative data, such as Medicare claims, represent one of the most practical and cost-effective data sources on selected components of healthcare quality available today” (Hurtado, Swift, and Corrigan 2001). Hospital report cards are based on risk-adjusted mortality rates that are calculated from administrative data. The clinical information used in risk adjustment is captured in the primary and secondary diagnoses coded using the International Classification of Diseases (ICD-9-CM) system. These administrative data sets, however, fail to distinguish between conditions present at admission (preexisting conditions) and conditions that developed subsequent to admission (complications). This distinction is critically important because misclassifying complications as preexisting conditions can lead to the overestimation of the risk of mortality, effectively giving lower quality hospitals “credit for the complications that occurred under their care” (Jollis and Romano 1998). For example, a hospital with a high postoperative myocardial infarction rate after coronary artery bypass grafting will have an inappropriately high-predicted mortality rate and, therefore, a low risk-adjusted mortality rate if patients with postoperative myocardial infarctions are wrongly assumed to have had their myocardial infarction prior to hospital admission. Inaccurate risk adjustment may yield incorrect conclusions regarding hospital quality. The inability to distinguish accurately between preexisting conditions and complications in administrative data may greatly reduce the face validity and value of hospital quality report cards. Unfortunately, ICD-9-CM codes in most administrative data sets are not “date stamped” to indicate whether they represent secondary diagnoses that were present prior to hospital admission or complications that developed subsequent to hospital admission. Only two states, California and New York, use a “condition present at admission” (CPAA) field to indicate for each recorded diagnosis as to whether or not it was present at the time of admission. Although in theory, the information from the CPAA field should lead to fewer errors, the extent to which complications are misclassified as preexisting conditions is largely unknown. In practice, the large number of ICD-9-CM codes—over 14,000—makes it impossible to perform risk adjustment using administrative data without first mapping ICD-9-CM codes to a smaller number of diagnostic categories (i.e., congestive heart failure [CHF], myocardial infarction, renal disease). The best-known mapping algorithms used for this purpose are the Deyo (1992) and Dartmouth–Manitoba (Romano, Roos, and Jollis 1993) adaptations of the Charlson index (Charlson et al. 1987), and the Elixhauser comorbidity measure (Elixhauser et al. 1998). These mapping algorithms are designed to exclude ICD-9-CM codes in a patient record that are likely to represent complications. The Deyo and Dartmouth–Manitoba adaptations of the Charlson index rely on the linkage of hospital data “across multiple episodes of care” to differentiate between preexisting conditions and complications. The Elixhauser algorithm, on the other hand, uses information only from the current admission: by design, many ICD-9-CM codes that could represent either complications or preexisting conditions were excluded from the Elixhauser algorithm in an effort to avoid inadvertently identifying a complication as a preexisting condition. Whether the addition of CPAA modifiers for each secondary diagnosis leads to more accurate identification of preexisting conditions versus complications is not well understood. Previous studies have been limited by small sample sizes and the use of narrowly defined population groups. Roos et al. (1997) investigated the extent to which the Dartmouth–Manitoba adaptation of the Charlson index misclassified complications in patients undergoing coronary artery bypass grafting surgery (CABG), pacemaker, and hip fracture surgery (n=7,187) using an administrative data set with date stamp information. These investigators found that the proportion of diagnoses—myocardial infarction, CHF, and cerebrovascular disease, etc.—that were correctly mapped, varied between 9.5 and 100 percent. The same study showed that in a much larger study population based on patients undergoing 17 surgical procedures, complications represented 11.1 percent of all of the diagnoses. However, in this larger patient group, findings were not reported for specific diagnostic categories. Southern, Quan, and Ghali (2004) showed, using an administrative data set with date stamp information, that the prevalence of specific diagnoses using the Deyo adaptation of the Charlson index was very similar regardless of whether or not date stamp information was used in patients with acute myocardial infarctions (n=4,833). However, prevalence rates across diagnoses could be similar if complications misclassified as preexisting diagnoses were offset by “missed diagnoses.”1 Neither of these studies explored the potential for diagnoses to be “missed” by the Charlson index—which can occur when some ICD-9-CM codes for a preexisting condition are present only on the current admission record, and not on a record from a previous hospitalization. The goal of our study was to quantify the misclassification rate of the Dartmouth–Manitoba adaptation of the Charlson index and of the Elixhauser algorithm using date stamp information as the “gold standard.” Our study was based on a cohort of 178,838 patients admitted for one of seven major surgical procedures or medical conditions: coronary artery bypass grafting, coronary angioplasty, carotid endarterectomy, abdominal aortic aneurysm (AAA) repair, total hip replacement, acute myocardial infarction, and stroke. The Dartmouth–Manitoba adaptation of the Charlson index and the Elixhauser algorithm were used to map ICD-9-CM codes to diagnostic categories. We compared the results of using these mapping algorithms with and without the use of date stamp information. We estimated both the proportion of complications that were misclassified as preexisting conditions and the proportion of preexisting conditions that were “missed” using these mapping algorithms. This study was conducted using the California State Inpatient Database (SID) because for each recorded diagnosis, California data indicates whether or not it was present on admission through the use of a “CPAA” field. Evaluating the importance of the datestamp is important if other states and the Medicare program are to consider adding date stamps to their administrative data. Currently, date-stamped diagnoses are not present in any of the state discharge databases other than California and New York, nor are they present in the Medicare or Medicaid databases. Adding date stamp information to hospital discharge data sets will be expensive because every secondary diagnosis will have to be evaluated by hospital coders to determine whether it was present on admission. However, misclassifying complications as preexisting conditions may seriously bias quality hospital measurement and may compromise our ability to improve health care quality in this country. If our findings show that the addition of “date stamp” information to ICD-9-CM codes leads to more accurate identification of preexisting conditions, health care policy makers will need to consider mandating date stamping of ICD-9-CM codes by the states and by the federal government.

Using public reports of patient satisfaction for hospital quality improvement.

Abstract: Although hospitals across the country routinely measure and report patient experience and satisfaction survey data internally, until recently few comparative public reports of hospital patient satisfaction have been available. A recent review identified nine states, cities, or regions that publicly reported comparative data on hospital patient experience and satisfaction (Barr et al. 2004). Similar public reports have been found on other websites; however, only five continue to report regularly (Shearer, Cronin, and Feeney 2004). Voluntary national efforts to publicly report on hospital quality include pilot projects that have tested the use of a standardized instrument (the Hospital CAHPS Survey) to measure patient perspectives on hospital care (Centers for Medicare & Medicaid Services 2005). The intent of public reporting is, not only to provide information for consumers, but also to stimulate quality improvement (QI) efforts in hospitals. Yet, there have been few formal studies about the impact of public reports on hospital QI. Several evaluations of public reports on hospital clinical measures suggest that facilities do make changes in response to these reports. In Wisconsin, a recent evaluation found that, among low scoring hospitals, those involved in public reporting were significantly more likely to report improvement activities in areas included in the public report than were comparison hospitals not involved in public reporting (Hibbard, Stockard, and Tusler 2003). Hospitals in Pennsylvania and New Jersey (Bentley and Nash 1998), Missouri (Longo et al. 1997), and Cleveland (Rosenthal et al. 1998) used public reports of performance to develop new approaches to improve clinical indicators. What remains unclear is whether public reports of patient experience will similarly result in efforts that can lead to improvement. Only one report in the literature discusses the impact of a hospital patient satisfaction public report on hospitals (Draper, Cohen, and Buchan 2001). Moreover, it is unknown how individual hospitals use these public reports to make changes that would improve their ratings. In order to listen to QI messages and adopt practices for QI, hospitals must be able to identify areas where they need improvement (Halm and Siu 2005) and have a way to track changes. Factors both internal and external to the hospital may affect their adoption of QI (Scanlon et al. 2001), and an organizational structure and culture that supports QI is critical to its adoption (Shortell et al. 1995; Berwick 2003; Bradley et al. 2003). Understanding the process through which hospitals respond to the public release of comparative data based on patient experience can help answer questions about the impact of public reporting on hospitals.

The impact of health plan delivery system organization on clinical quality and patient satisfaction.

Abstract: Evidence of the quality gap between best practice and the current reality of everyday medical care is widely documented and acknowledged (Institute of Medicine [IOM] 2001a, b, 2002, Leatherman and McCarthy 2002; McGlynn et al. 2003). Two IOM reports, Crossing the Quality Chasm (2002) and Leadership by Example (2001a), link the defects in quality largely to system problems rather than individual errors or actions. These reports have helped to focus attention on the need to identify the characteristics that differentiate high-performing care delivery systems from those that do less well. Studies to assess variations in care processes, costs, outcomes, and patient perceptions of care across organizations and delivery system types have not been conclusive or consistent in determining whether one type of delivery system or care delivery organization delivers higher quality or lower cost care than others (Scitovsky and McCall 1980; Nobrega et al. 1982; Miller 1992; Himmelstein et al. 1999; Miller and Luft 1994, 2002; Singh and Kalavar 2004). Shortell and Schmittdiel (2004) suggested that organized delivery systems, especially large, multispecialty practices, characterized by patient-care teams, defined patient populations, aligned financial and payment incentives, partnership between medicine and management, information technology, and accountability, have the potential to provide superior performance in terms of clinical quality and safety although they concluded that studies have yet to demonstrate superiority in the quality, efficiency, or costs of care. Casalino et al. (2003a) demonstrated that physician organizations with strong external incentives, clinical information technology, substantial health maintenance organization (HMO) penetration, a high percentage of patients with utilization management delegated to the group, and owned or affiliated with a hospital, health system or health plan used more recommended care management processes (CMPs), which have been shown to be linked to higher quality care (Wagner et al. 2001). Shortell et al. (2005) found that high performing physician organizations were significantly more likely than low-performing physician organizations to engage in formally organized quality improvement initiatives and external reporting of quality data. Chuang, Luft, and Dudley (2004) posited that health plans affiliated with group- or staff-model delivery systems deliver higher quality care than other plans because of greater integration across specialties and sites of care; decreased conflict among clinical protocols; more consistency of incentives and goals; and larger scale and more stable enrollment populations. Other studies (Levin 2001, Casalino et al. 2003b) indicated that group practices provide more recommended treatments for chronic disease and have lower mortality rates from congestive heart failure and other disease. On the other hand, a study (Baker et al. 2004) of health plans in California concluded that the impact on Healthplan Employer Data and Information Set (HEDIS®)-based quality scores could be owing to more efficient administrative and data reporting systems than to what the physicians themselves did. The study was not conclusive about the impact of the physician group on quality scores. Similarly, existing evidence comparing patient perceptions of care in health maintenance organizations (HMOs) with those in fee-for-service settings is mixed (Miller and Luft 2002, Roohan et al. 2003; Lin, Xirasagar, and Laditka 2004). It should be noted that most existing research on patient perceptions of care does not separate HMOs by the type of delivery system used to deliver care. In the present study, we examine whether the extent to which a health plan utilizes a staff or group model of care delivery is associated with better clinical performance and patient satisfaction. We also examine other organizational characteristics, including geographic location of the plan, affiliation with a national managed care firm, and for-profit status, that may be associated with the performance of health plans (Himmelstein et al. 1999, NCQA 2004).

Parent‐Identified Barriers to Pediatric Health Care: A Process‐Oriented Model

Abstract: Children's health care services access, utilization, and outcomes in the U.S. are characterized by disparities across vulnerability factors such as socioeconomic status (SES), race/ethnicity, and language (Newacheck, Hughes, and Stoddard 1996; Smedley et al. 2003). While much research has documented the associations between variables such as insurance status, race/ethnicity, education, and English language ability on health care access and quality, less is known about the processes by which these associations arise. Qualitative, patient-centered research methods hold great promise for expanding our knowledge in this area. Building on Andersen and Aday's behavioral model of health care access (Aday and Andersen 1974; Andersen and Aday 1978), Aday's model of vulnerability (Aday 1993, 1994), and the noncategorical approach (Stein et al. 1993) to pediatric quality-of-care measurement, Seid et al. (2003) have proposed a conceptual model to organize examinations of how health care structures and processes affect health-related quality of life for vulnerable children. Earlier versions of this model have been used to generate a parent-report primary care measure (Seid et al. 2001), examine the effects of language, race/ethnicity, and access to care on parents' reports of primary care experience (Seid, Stevens, and Varni 2003), and compare child health services access and primary care experiences on both sides of the U.S.–Mexico border (Seid et al. 2003). However useful for describing relationships between vulnerability factors and access to and quality of care, these quantitative studies fell short in elucidating the processes by which these relationships might arise. For example, Seid, Stevens, and Varni (2003) documented, in a sample drawn from 18 elementary schools in an urban school district, that insurance status, language, and presence of a regular provider of care were significantly related to scores on a parent-report measure of pediatric primary care experiences called the Parent's Perceptions of Primary Care survey or P3C (Seid et al. 2001). A closer look at these data reveals that more than half (56 percent) of those with the lowest P3C scores (Z-scores less than −1.96) were insured, 43.5 percent completed the survey in English, and 39 percent reported having a regular doctor. In other words, despite the significant associations between vulnerability factors (insurance status, language, having a regular source of care) and the quality of primary care reported via the P3C score, a substantial number of children without these vulnerability markers experienced poor primary care. Conversely, a substantial number of children with vulnerability markers experienced better primary care.

Variation in Hospital Length of Stay: Do Physicians Adapt Their Length of Stay Decisions to What Is Usual in the Hospital Where They Work?

Abstract: A persistent finding in health services research is that hospital utilization varies widely (Paul-Shaheen, Clark, and Williams 1987; Ashton et al. 1999; Wennberg 1999). These variations have been observed between geographic areas, hospitals, and physicians. The variation within these units of analysis has been found to be smaller than between the units, for different types of services, numbers of admissions, and length of stay (Wennberg and Gittelsohn 1982; Read et al. 1983; Westert, Nieboer, and Groenewegen 1993; Arndt, Bradbury, and Golec 1995; O'Connor et al. 1999). Several explanations have been sought for the variation between and similarities within units; a summary can be found in Table 1. Wennberg and Gittelsohn (1975) suggested that an explanation lies with the physicians themselves. First, they theorized that professional uncertainty explains whether the specific procedure or diagnosis will have high or low variation. Second, they hypothesized that the judgment and preferences of groups of physicians give rise to a unique pattern over time, which has been termed a “surgical signature.”Chassin (1993) suggested that variation is caused by a difference in the prevalence of physicians who are enthusiastic about certain procedures. The “surgical signature” and enthusiasm hypotheses assume that physicians have a preference for certain procedures, but the behavioral mechanisms that produce different practice styles remain unclear. Table 1 Summary of Explanations for Variation Westert and Groenewegen (1999) offered an alternative to this preference-centered approach, emphasizing incentives and environmental conditions that influence the behavior of physicians by providing opportunities and constraints. Westert (1992) applied this approach in a model of local standards that predicts similarities among physicians who share a common work environment and thus a social system and similar constraints. We tested whether variations in medical practice are indeed related to the hospital in which physicians practice. This would imply that variation within hospitals is small compared with variation between hospitals for physicians treating similar patients. A second implication would be that physicians working in more than one hospital conform to the usual practice of each hospital (Westert 1992). This implication cannot be deduced from a preference-centered approach, as the preferences of an individual would not change when working in another hospital. Indications were found by Westert, Nieboer, and Groenewegen (1993) that multihospital physicians in the Netherlands have a patient length of stay close to the usual practice in the hospital where the surgery was performed. Griffiths, Waters, and Acheson (1979), using data from the British National Health Service, also found that average postoperative stays were similar among physicians who practice in the same hospitals, while the average between hospitals was significantly different for physicians who practice in more than one hospital. In the present study, we tested these implications with data from the United States, where it is quite common for physicians to work in more than one hospital. As a result, the implications can be tested more reliably than was carried out by Westert with 23 physicians in five hospitals (only four working in more than one hospital) and Griffiths with nine physicians in eight hospitals (six working in more than one hospital). Furthermore, we used a different methodology. We took length of stay, which is a well-defined and important indicator in hospital management, as the outcome variable.