Book•
Engineering a Safer World: Systems Thinking Applied to Safety
13 Jan 2012-
TL;DR: A new approach to safety, based on systems thinking, that is more effective, less costly, and easier to use than current techniques was proposed by Leveson et al. as discussed by the authors, which is more suited to today's complex, sociotechnical, software-intensive world.
Abstract: A new approach to safety, based on systems thinking, that is more effective, less costly, and easier to use than current techniques.Engineering has experienced a technological revolution, but the basic engineering techniques applied in safety and reliability engineering, created in a simpler, analog world, have changed very little over the years. In this groundbreaking book, Nancy Leveson proposes a new approach to safety?more suited to today's complex, sociotechnical, software-intensive world?based on modern systems thinking and systems theory. Revisiting and updating ideas pioneered by 1950s aerospace engineers in their System Safety concept, and testing her new model extensively on real-world examples, Leveson has created a new approach to safety that is more effective, less expensive, and easier to use than current techniques.Arguing that traditional models of causality are inadequate, Leveson presents a new, extended model of causation (Systems-Theoretic Accident Model and Processes, or STAMP), then shows how the new model can be used to create techniques for system safety engineering, including accident analysis, hazard analysis, system design, safety in operations, and management of safety-critical systems. She applies the new techniques to real-world events including the friendly-fire loss of a U.S. Blackhawk helicopter in the first Gulf War; the Vioxx recall; the U.S. Navy SUBSAFE program; and the bacterial contamination of a public water supply in a Canadian town. Leveson's approach is relevant even beyond safety engineering, offering techniques for ?reengineering? any large sociotechnical system to improve safety and manage risk.
Citations
More filters
TL;DR: This report assesses how HIS can be designed, developed, implemented, monitored, and maintained to maximize safety and identifies seven publications presenting results of assessments of e-iatrogenic risk.
Abstract: of “Roadmap for Provision of Safer Healthcare Information Systems: Preventing eIatrogenesis” Joan S. Ash, Ph.D., M.L.S., M.B.A.; Charles M. Kilo, M.D., M.P.H.; Michael Shapiro, M.A., M.S.; Joseph Wasserman, B.A.; Carmit McMullen, Ph.D.; William Hersh, M.D. BACKGROUND AND METHODS e-Iatrogenesis, defined as “patient harm caused at least in part by the application of health information technology” (Weiner et al., 2007), is of increasing concern as more and more hospitals are implementing health information systems (HIS). This report assesses how HIS can be designed, developed, implemented, monitored, and maintained to maximize safety. We specifically focus on hospital electronic health records (EHRs), clinical decision support (CDS), and computerized provider order entry (CPOE) systems. This white paper is intended to provide background for an Institute of Medicine (IOM) report on how the use of health information technology affects the safety of patient care by answering the following IOM-posed questions: • What are the risks of health care information systems that arise from workflow and related issues? • How have organizations acted to implement health care information systems safely? • What are the impacts of customization on safety? • What is the industry approach to managing change and customization? A recent literature review by Harrington et al. (2011) has summarized the EHR safety literature, so we first reviewed all papers cited in their report. Of their 43 references, we identified 37 that were relevant to the scope of this article. We analyzed the bibliographies of these selected papers and performed a reverse bibliography search on the articles deemed most relevant and published since 2000. In total, we identified over 100 sources relevant to the scope of this report. We then targeted topics for which published evidence was lacking and conducted several interviews with experts to help fill the knowledge gaps. 1 Full commissioned paper can be found in this project’s Public Access File. C-2 HEALTH IT AND PATIENT SAFETY PREPUBLICATION COPY: UNCORRECTED PROOFS RESULTS What Are the Risks of Health Information Systems That Arise from Workflow and Related Issues? We found seven publications (Chuo and Hicks, 2008; Joint Commission on Accreditation of Healthcare Organizations, 2008; Magrabi et al., 2010; Myers et al., 2011; Santell et al., 2009; Walsh et al., 2006; Zhan et al., 2006) presenting results of assessments of e-iatrogenic risk. All are studies of large databases of reported errors and they consistently indicate low levels of HISrelated risk, under 1 percent of all errors. All point to the need for human diligence when using HIS. Specifically, they indicate that HIS-related errors are due to inadequate staffing levels, lack of user experience, mislabeled bar-codes on medications, human distraction, inaccurate data entry, system downtime, and missing data. How Have Organizations Acted to Implement Health Information Systems Safely? Prior to implementation, health care organizations can mitigate risk. There is a large literature base devoted to the risks inherent in commercial EHR systems, and also warnings about their impact on workflow. Many publications offer guidance to hospitals about assessing workflow, selecting systems for purchase, conducting simulation tests, training, and other mechanisms for ensuring safe HIS implementation. Numerous publications exist to guide the implementation process itself, but there are also several pointing to the risks of rapid implementation without appropriate preparation. Finally, after implementation, continuous monitoring and improvement can mitigate safety risks. What Are the Impacts of Customization on Safety? The literature indicates that customization of the EHR to fit local situations seems to be necessary for many reasons, but there is scant research on how much customization or what form of customization is needed to optimize EHR use and what the risks are from either too much or too little customization. The content of CDS likewise needs adaptation, especially to avoid alert fatigue. Any customization must be done with care so that system upgrades can be accommodated. What Is the Industry Approach to Managing Change and Customization? The current industry approach is fragmented; a report sponsored by the Agency for Healthcare Research and Quality describes a wide variety of vendor practices related to usability of systems (McDonnell et al., 2010). Because purchasers must usually customize systems to fit local workflows and regulations, HIS safety depends on a combination of industry and local diligence.
462 citations
27 Jan 2020
TL;DR: The proposed auditing framework is intended to contribute to closing the accountability gap in the development and deployment of large-scale artificial intelligence systems by embedding a robust process to ensure audit integrity.
Abstract: Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited for harm by investigators from outside the organizations deploying the algorithms. However, it remains challenging for practitioners to identify the harmful repercussions of their own systems prior to deployment, and, once deployed, emergent issues can become difficult or impossible to trace back to their source. In this paper, we introduce a framework for algorithmic auditing that supports artificial intelligence system development end-to-end, to be applied throughout the internal organization development life-cycle. Each stage of the audit yields a set of documents that together form an overall audit report, drawing on an organization's values or principles to assess the fit of decisions made throughout the process. The proposed auditing framework is intended to contribute to closing the accountability gap in the development and deployment of large-scale artificial intelligence systems by embedding a robust process to ensure audit integrity.
373 citations
Cites background or methods from "Engineering a Safer World: Systems ..."
...For example, a power plant can be consistently productive while causing harm to the environment through pollution [42]....
[...]
...Following Leveson’s work on safety engineering [42], we stress that careful attention must be paid to the distinction between the designers’ mental models of the artificial intelligence system and the user’s mental model....
[...]
...intelligence systems increase, new approaches are necessary to understand risk [42]....
[...]
...To design our audit procedure, we suggest complementing formal risk assessment methodologies with ideas from responsible innovation, which stresses four key dimensions: anticipation, reflexivity, inclusion and responsiveness [73], as well as system-theoretic concepts that help grapple with increasing complexity and coupling of artificial intelligence systems with the external world [42]....
[...]
...Complex systems tend to drift toward unsafe conditions unless constant vigilance is maintained [42]....
[...]
25 May 2015
TL;DR: In this article, enabling technologies and solutions for the electrified transportation are discussed in terms of power electronics, electric machines, electrified powertrain architectures, energy storage systems (ESSs), and controls and software.
Abstract: In order to achieve lower fuel consumption and less greenhouse gas (GHG) emissions, we need higher efficiency vehicles with improved performance. Electrification is the most promising solution to enable a more sustainable and environmentally friendly transportation system. Electrified transportation vision includes utilizing more electrical energy to power traction and nontraction loads in the vehicle. In electrified powertrain applications, the efficiency of the electrical path, and the power and energy density of the components play important roles to improve the electric range of the vehicle to run the engine close to its peak efficiency point and to maintain lower energy consumption with less emissions. In general, the electrified powertrain architecture, design and control of the powertrain components, and software development are coupled to facilitate an efficient, high-performance, and reliable powertrain. In this paper, enabling technologies and solutions for the electrified transportation are discussed in terms of power electronics, electric machines, electrified powertrain architectures, energy storage systems (ESSs), and controls and software.
340 citations
Cites background from "Engineering a Safer World: Systems ..."
...Safety [69], ratified in 2011, has recognized the need to properly address safety of electrical and/or electronic components, recognizing the rapidly increasing role of these components...
[...]
TL;DR: The study suggests that, whilst the selection of an analysis method is subject to trade-offs that practitioners and researchers must make, the SCM remains a viable model for accident analysis.
264 citations
TL;DR: A comprehensive survey of existing approaches to industrial facility design and risk assessment that consider both safety and security and a comparative analysis of the different approaches identified in the literature is provided.
256 citations
Cites background from "Engineering a Safer World: Systems ..."
...System theoretic accident model and processes (STAMP) is an accident causality model developed by Leveson [104] that accounts for new causal factors associated with software, human decision making, new technology, social and organizational design, and increasing complexity....
[...]