Who Is an Efficient and Effective Physician? Evidence From Emergence Medicine
Summary (3 min read)
Introduction
- Healthcare spending is projected to rise to 19.9% of the GDP by 2025 (Keehan et al. 2017), spurring interest in finding new ways to increase both the efficiency and effectiveness of care delivery.
- Specifically, the authors consider how frequently a physician admits patients who are subsequently discharged after a brief period, which suggests that the physician potentially overcalled the patients’ illness severity (i.e., patients are admitted when they could have potentially been discharged).
- In particular, the authors consider peer physician characteristics such as relative effectiveness, efficiency, experience, gender, and type of medical degree (MD vs. DO), and examine how they affect another physician’s efficiency and effectiveness.
- Contrary to the conventional wisdom that efficiency may come at the price of effectiveness (and vice-versa), their findings demonstrate a statistically significant positive association between efficiency and effectiveness (P = 0.0209).
Data
- The authors data consist of detailed care delivery information in a leading U.S. hospital with 32 ED physicians.
- The patients are algorithmically assigned to physicians upon arrival through an automated rotational patient assignment process (Traub et al. 2016).
- The authors included all patient visits from July 12, 2012, to July 31, 2016 who were identified in the electronic health record system as having been seen by an ED physician.
- Encounter-level data included laboratory tests, chief complaint, Emergency Severity Index (ESI), day of the ED visit, time of day, etc., totaling over 70 variables.
- To avoid distortion of the results by outliers, 6 physicians with relatively low patient volumes (fewer than 250 visits over the 4-year period) were excluded from the analysis.
DEA Models
- DEA, first introduced by Charnes, Cooper, and Rhodes in (Charnes et al. 1978), is a methodology useful in evaluating the relative performance of a set of decision making units (DMUs) in a multiple input, multiple output setting.
- The original DEA model was based on a constant returns to scale (CRS) methodology.
- The efficiency DEA model considers the relative use of hospital resources by a physician to his/her throughput.
- The authors have chosen herein to define the models’ input and output variables in terms of parameters that (a) best reflect a physician’s performance, and (b) for which there is at least face validity and some level of agreement among physicians.
- These variables are then used along with the optimization program (1) to create efficiency and effectiveness scores.
Effectiveness DEA Model
- Outputs: Rate of discharged patients who do not return within 72-hours :.
- Since this would be considered an undesirable output, the authors use the 6-hour non-upgrade patient admission rate as an output variable.
- The choice of threshold numbers (72, 17, and 6) is made based on observations made in the literature (see, e.g., Keith et al.
- Average number of the physician’s lab orders per patient visit; Plain Radiograph Order Count : 8 Electronic copy available at: https://ssrn.com/abstract=3227873.
Efficiency DEA Model
- Average number of patients seen by the physician per shift; Inputs: Lab Order Count :.
- Average number of the physician’s lab orders per patient visit; Ultrasound Order Count :.
- It is important to note that the authors compared the physicians’ average number of hours worked per shift using the paired-observation t-test method and after removing two physicians from their analysis, they found no significant differences between the remaining physicians’ average hours worked per shift.
- The authors did not include the plain radiograph order count as an input variable in the efficiency model because of its negative correlation with the model’s output variable.
- The authors choice of variables for both models was validated using the stepwise variable selection method mentioned earlier.
Peer-Effect DEA Model
- In order to also examine the effects of peer presence on a physician’s effectiveness and efficiency, the authors use a variation of the proposed DEA models in which each DMU, denoted by 9 Electronic copy available at: https://ssrn.com/abstract=3227873 jk, comprise a physician j who has worked alongside his/her peer physician k for at least 5 hours.
- The authors choose the 5-hour criterion to be able to capture any meaningful peer physician influence.
- This assumption was tested by Simar and Wilson’s (Simar et al. 2002, Simar et al. 2011) returns-to-scale tests for input-oriented DEA models.
- For this reason, an input-oriented approach was used to test whether a DMU under evaluation can reduce its inputs while keeping the outputs at their current levels.
Statistical Methodology
- With the goal of learning about the practice of physicians who have a better performance than the rest, the authors regress the generated DEA scores of physician i (θi), on a set of explanatory variables related to patient, physician, and peer physician j characteristics which they denote by Ui, Wi, and Zij, respectively.
- The regression model takes the following general form θi = a+.
- This is because this standard regression technique assumes a normal and homoscedastic distribution of the noise.
- Following the normalization approach of Greene (1993) which assumes a censoring point at zero, the authors transform the DEA scores to: yi = (1/θi)− 1, where θi is the DEA measure of physician i’s performance.
Discussion and Results
- The authors begin their analysis by examining the relationship between physicians’ scores on measures related to effectiveness and efficiency.
- This is a counterintuitive result, which questions the traditional belief that experience enables physicians to use hospital resources more frugally.
- The regression results of peer physician efficiency analysis (displayed in Table 4) show that the presence of a more effective peer is associated with (on average) an increase in physician efficiency.
- Nevertheless, the natural experiment setting discussed earlier removes potential biases, and hence, many of the associations the authors find may be causal.
Conclusions
- Using evidence from emergency medicine, the authors develop and analyze metrics for physicians’ effectiveness and efficiency.
- Unlike what the conventional wisdom suggests, their findings show that a physician’s effectiveness is positively (and not negatively) associated with his/her efficiency.
- The authors also find that highly efficient physicians have on average lower MRI orders per patient visit.
- The authors believe that their analysis serves as an early step to explore issues of physician effectiveness and efficiency.
- Importantly, the authors do not believe that the scores they develop are the 19 Electronic copy available at: https://ssrn.com/abstract=3227873 only ways to measure effectiveness or efficiency.
Did you find this useful? Give us your feedback
Citations
4 citations
References
150 citations
149 citations
"Who Is an Efficient and Effective P..." refers background in this paper
...…to his/her cognitive abilities, his/her 1 In EDs, the service is provided by a specific physician who is in charge of the patient, and the ED service is very rarely composed of a teamwork among physicians (see, e.g., Saghafian et al. 2012, Saghafian et al. 2019, and the references therein)....
[...]
140 citations
"Who Is an Efficient and Effective P..." refers background in this paper
...Specifically, given the large body of literature examining the effects of high workloads on physician performance (KC and Terwiesch 2009, Powell et al. 2012, Berry Jaeker and Tucker 2017, Batt and Terwiesch 2017), we study whether and how physician effectiveness and efficiency are affected by high ED volume....
[...]
...…given the large body of literature examining the effects of high workloads on physician performance (KC and Terwiesch 2009, Powell et al. 2012, Berry Jaeker and Tucker 2017, Batt and Terwiesch 2017), we study whether and how physician effectiveness and efficiency are affected by high ED volume....
[...]
138 citations
"Who Is an Efficient and Effective P..." refers background in this paper
...…given the large body of literature examining the effects of high workloads on physician performance (KC and Terwiesch 2009, Powell et al. 2012, Berry Jaeker and Tucker 2017, Batt and Terwiesch 2017), we study whether and how physician effectiveness and efficiency are affected by high ED volume....
[...]
123 citations
"Who Is an Efficient and Effective P..." refers background in this paper
...Low-ESI patients and those older than 65 are known to be patients that have a relatively higher contact-to-disposition time than other patients (Latham and Ackroyd-Stolarz 2014)....
[...]