Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors

Summary Of the more than 1,300 studies identi-fied as potentially addressing the effect of teacher professional development on student achievement in three key con-tent areas, nine meet What Works Clear-inghouse evidence standards, attesting to the paucity of rigorous studies that directly examine this link. This report finds that teachers who receive sub-stantial professional development—an average of 49 hours in the nine studies—can boost their students’ achievement by about 21 percentile points. How does teacher professional development affect student achievement? The connection seems intuitive. But demonstrating it is difficult. Examining more than 1,300 studies identified as potentially addressing the effect of teacher professional development on student achieve-ment in three key content areas, this report finds nine that meet What Works Clearing-house evidence standards. That only nine meet standards attests to the paucity of rigorous studies that directly assess the effect of in-service teacher professional development on student achievement in mathematics, science, and reading and English/language arts.But the results of those studies—that average control group students would have increased their achievement by 21 percentile points if their teacher had received substantial profes-sional development—indicates that provid-ing professional development to teachers had a moderate effect on student achievement across the nine studies. The effect size was fairly consistent across the three content areas reviewed. All nine studies focused on elementary school teachers and their students. About half fo-cused on lower elementary grades (kindergar-ten and first grade), and about half on upper elementary grades (fourth and fifth grades).Six studies were published in peer-reviewed journals; three were unpublished doctoral dissertations. The studies were not particularly recent, ranging from 1986 to 2003.Five studies were randomized controlled trials that meet evidence standards without reserva-tions. Four studies meet evidence standards with reservations (one randomized controlled trial with group equivalence problems and three quasi-experimental designs).Four focused on student achievement in read-ing and English/language arts—unsurprising given the large literature in this content area. Two studies focused on mathemat-ics, two on mathematics and reading and

https://ies.ed.gov/ncee/edlabs/regions/southwest/pdf/rel_2007033_sum.pdf

Reviewing the evidence on how teacher professional development affects student achievement

Generalized Linear Models (2nd ed.)

This meta-analysis on surgical treatment of obesity supports the American College of Physicians' clinical guideline in this issue. Surgery is more effective than nonsurgical treatment for weight lo...

http://www.paxysarkia.net/Download/Article_metaanalysis_surgical%20treatment%20of%20obesity.pdf

Meta-analysis: surgical treatment of obesity.

ContextThe increasing prevalence and associated sociodemographic disparities
of morbid obesity are serious public health concerns. Bariatric surgical procedures
provide greater and more durable weight reduction than behavioral and pharmacological
interventions for morbid obesity.ObjectiveTo examine trends for elective bariatric surgical procedures, patient
characteristics, and in-hospital complications from 1998 to 2003 in the United
States.Design, Setting, and PatientsThe Nationwide Inpatient Sample was used to identify bariatric surgery
admissions from 1998-2002 (with preliminary data for 12 states from 2003)
using International Classification of Diseases, Ninth Revision, codes for foregut surgery with a confirmatory diagnosis of obesity
or by diagnosis related group code for obesity surgery. Annual estimates and
trends were determined for procedures, patient characteristics, and adjusted
complication rates.Main Outcome MeasuresTrends in bariatric surgical procedures, patient characteristics, and
complications.ResultsThe estimated number of bariatric surgical procedures increased from
13 365 in 1998 to 72 177 in 2002 (P<.001).
Based on preliminary state-level data (1998-2003), the number of bariatric
surgical procedures is projected to be 102 794 in 2003. Gastric bypass
procedures accounted for more than 80% of all bariatric surgical procedures.
From 1998 to 2002, there were upward trends in the proportion of females (81%
to 84%; P = .003), privately insured patients
(75% to 83%; P = .001), patients from ZIP
code areas with highest annual household income (32% to 60%, P<.001), and patients aged 50 to 64 years (15% to 24%; P<.001). Length of stay decreased from 4.5 days in 1998 to 3.3 days
in 2002 (P<.001). The adjusted in-hospital mortality
rate ranged from 0.1% to 0.2%. The rates of unexpected reoperations for surgical
complications ranged from 6% to 9% and pulmonary complications ranged from
4% to 7%. Rates of other in-hospital complications were low.ConclusionsThese findings suggest that use of bariatric surgical procedures increased
substantially from 1998 to 2003, while rates of in-hospital complications
were stable and length of stay decreased. However, disparities in the use
of these procedures, with disproportionate and increasing use among women,
those with private insurance, and those in wealthier ZIP code areas should
be explored further.

https://jamanetwork.com/journals/jama/articlepdf/201708/joc50128.pdf

Trends in Bariatric Surgical Procedures

The relationship of surgeon and hospital volume to outcome after gastric bypass surgery in Pennsylvania: a 3-year summary.

OBJECTIVE: This study explores the alignment between physicians’ confidence in their diagnoses and the “correctness” of these diagnoses, as a function of clinical experience, and whether subjects were prone to over-or underconfidence. DESIGN: Prospective, counterbalanced experimental design. SETTING: Laboratory study conducted under controlled conditions at three academic medical centers. PARTICIPANTS: Seventy-two senior medical students, 72 senior medical residents, and 72 faculty internists. INTERVENTION: We created highly detailed, 2-to 4-page synopses of 36 diagnostically challenging medical cases, each with a definitive correct diagnosis. Subjects generated a differential diagnosis for each of 9 assigned cases, and indicated their level of confidence in each diagnosis. MEASUREMENTS AND MAIN RESULTS: A differential was considered “correct” if the clinically true diagnosis was listed in that subject’s hypothesis list. To assess confidence, subjects rated the likelihood that they would, at the time they generated the differential, seek assistance in reaching a diagnosis. Subjects’ confidence and correctness were “mildly” aligned (k=.314 for all subjects, .285 for faculty, .227 for residents, and .349 for students). Residents were overconfident in 41% of cases where their confidence and correctness were not aligned, whereas faculty were overconfident in 36% of such cases and students in 25%. CONCLUSIONS: Even experienced clinicians may be unaware of the correctness of their diagnoses at the time they make them. Medical decision support systems, and other interventions designed to reduce medical errors, cannot rely exclusively on clinicians’ perceptions of their needs for such support.

https://deepblue.lib.umich.edu/bitstream/handle/2027.42/74850/j.1525-1497.2005.30145.x.pdf;sequence=1

Do physicians know when their diagnoses are correct? Implications for decision support and error reduction.

Many statistical procedures used in educational research are described as requiring that dependent variables follow a normal distribution, implying an interval scale of measurement. Despite the desirability of interval scales, many dependent variables possess an ordinal scale of measurement in which the differences among values composing the scale are unequal in terms of what is being measured, permitting only a rank ordering of scores. This means that data possessing an ordinal scale will not satisfy the assumption of normality needed in many statistical procedures and may produce biased statistical results that threaten the validity of inferences. This article shows how the measurement technique known as item response theory can be used to rescale ordinal data to an interval scale. The authors provide examples of rescaling using student performance data and argue that educational researchers should routinely consider rescaling ordinal data using item response theory.

https://www.jstor.org/stable/pdfplus/3516068.pdf

Rescaling Ordinal Data to Interval Data in Educational Research

Computer decision support as a source of interpretation error: the case of electrocardiograms.

The process of clinical decision support is linked to the validity of clinicians' confidence in their judgments. Clinicians who are appropriately confident-highly confident when they are correct and less confident when they are incorrect-will access computer-based and other information resources only when they are needed. Clinicians who are consistently underconfident will rely on external resources when they are not needed. Those who are overconfident, who believe they are correct when in fact they are not, will be prone to medical errors. An extensive literature indicates a general tendency toward overconfidence in human judgment. This study explores the relationship between confidence and "correctness", across three levels of clinical experience, in the task domain of diagnosis in internal medicine. We created detailed synopses of 36 diagnostically challenging cases and divided them into four equivalent sets of nine cases each. We asked 216 subjects at three experience levels (72 senior medical students, 72 senior medical residents, and 72 faculty attendings) to generate a differential diagnosis for each of the nine cases in one randomly-assigned set, and simultaneously to indicate their level of confidence in each of their diagnoses. We then examined the relationship between the correctness of these diagnoses (the appearance of the correct diagnosis anywhere in the hypothesis list) and these confidence judgments, for all subjects and separately for subjects at each experience level. Results indicate a small but statistically significant relationship associating correctness with higher confidence for all subjects (Kendall's tau b =.-106;p <.0001). This statistical relationship is strongest for the students ( tau b =.-121;p <.001), somewhat lesser but still significant for the faculty-level attendings ( tau b =.-103;p <.005), and non-significant ( tau b =.-041 ) for the residents. (The negative correlations are a coding artifact.) Subjects in this study showed a tendency toward underconfidence: they had low confidence in correct diagnoses more often than they had high confidence when wrong. Nonetheless, they were overconfident and thus "error prone" for 17% of cases overall. The medical students were possibly overmatched by the difficulty of the cases, so their concordance between confidence and correctness may have resulted from an awareness that they were often guessing. The relatively low concordance seen in the residents and attendings makes a strong argument that decision support systems to reduce medical errors should include both "push" and "pull" models. In sum, these results indicate that medical decision support systems cannot rely exclusively on clinicians' perceptions of their information needs, as such perceptions will frequently be incorrect.

Guido G. Gatti

Papers

The relationship of surgeon and hospital volume to outcome after gastric bypass surgery in Pennsylvania: a 3-year summary.

Do physicians know when their diagnoses are correct? Implications for decision support and error reduction.

Rescaling Ordinal Data to Interval Data in Educational Research

Computer decision support as a source of interpretation error: the case of electrocardiograms.

Are clinicians correct when they believe they are correct? Implications for medical decision support.