Who Is an Efficient and Effective Physician? Evidence From Emergence Medicine
Summary (3 min read)
Introduction
- Healthcare spending is projected to rise to 19.9% of the GDP by 2025 (Keehan et al. 2017), spurring interest in finding new ways to increase both the efficiency and effectiveness of care delivery.
- Specifically, the authors consider how frequently a physician admits patients who are subsequently discharged after a brief period, which suggests that the physician potentially overcalled the patients’ illness severity (i.e., patients are admitted when they could have potentially been discharged).
- In particular, the authors consider peer physician characteristics such as relative effectiveness, efficiency, experience, gender, and type of medical degree (MD vs. DO), and examine how they affect another physician’s efficiency and effectiveness.
- Contrary to the conventional wisdom that efficiency may come at the price of effectiveness (and vice-versa), their findings demonstrate a statistically significant positive association between efficiency and effectiveness (P = 0.0209).
Data
- The authors data consist of detailed care delivery information in a leading U.S. hospital with 32 ED physicians.
- The patients are algorithmically assigned to physicians upon arrival through an automated rotational patient assignment process (Traub et al. 2016).
- The authors included all patient visits from July 12, 2012, to July 31, 2016 who were identified in the electronic health record system as having been seen by an ED physician.
- Encounter-level data included laboratory tests, chief complaint, Emergency Severity Index (ESI), day of the ED visit, time of day, etc., totaling over 70 variables.
- To avoid distortion of the results by outliers, 6 physicians with relatively low patient volumes (fewer than 250 visits over the 4-year period) were excluded from the analysis.
DEA Models
- DEA, first introduced by Charnes, Cooper, and Rhodes in (Charnes et al. 1978), is a methodology useful in evaluating the relative performance of a set of decision making units (DMUs) in a multiple input, multiple output setting.
- The original DEA model was based on a constant returns to scale (CRS) methodology.
- The efficiency DEA model considers the relative use of hospital resources by a physician to his/her throughput.
- The authors have chosen herein to define the models’ input and output variables in terms of parameters that (a) best reflect a physician’s performance, and (b) for which there is at least face validity and some level of agreement among physicians.
- These variables are then used along with the optimization program (1) to create efficiency and effectiveness scores.
Effectiveness DEA Model
- Outputs: Rate of discharged patients who do not return within 72-hours :.
- Since this would be considered an undesirable output, the authors use the 6-hour non-upgrade patient admission rate as an output variable.
- The choice of threshold numbers (72, 17, and 6) is made based on observations made in the literature (see, e.g., Keith et al.
- Average number of the physician’s lab orders per patient visit; Plain Radiograph Order Count : 8 Electronic copy available at: https://ssrn.com/abstract=3227873.
Efficiency DEA Model
- Average number of patients seen by the physician per shift; Inputs: Lab Order Count :.
- Average number of the physician’s lab orders per patient visit; Ultrasound Order Count :.
- It is important to note that the authors compared the physicians’ average number of hours worked per shift using the paired-observation t-test method and after removing two physicians from their analysis, they found no significant differences between the remaining physicians’ average hours worked per shift.
- The authors did not include the plain radiograph order count as an input variable in the efficiency model because of its negative correlation with the model’s output variable.
- The authors choice of variables for both models was validated using the stepwise variable selection method mentioned earlier.
Peer-Effect DEA Model
- In order to also examine the effects of peer presence on a physician’s effectiveness and efficiency, the authors use a variation of the proposed DEA models in which each DMU, denoted by 9 Electronic copy available at: https://ssrn.com/abstract=3227873 jk, comprise a physician j who has worked alongside his/her peer physician k for at least 5 hours.
- The authors choose the 5-hour criterion to be able to capture any meaningful peer physician influence.
- This assumption was tested by Simar and Wilson’s (Simar et al. 2002, Simar et al. 2011) returns-to-scale tests for input-oriented DEA models.
- For this reason, an input-oriented approach was used to test whether a DMU under evaluation can reduce its inputs while keeping the outputs at their current levels.
Statistical Methodology
- With the goal of learning about the practice of physicians who have a better performance than the rest, the authors regress the generated DEA scores of physician i (θi), on a set of explanatory variables related to patient, physician, and peer physician j characteristics which they denote by Ui, Wi, and Zij, respectively.
- The regression model takes the following general form θi = a+.
- This is because this standard regression technique assumes a normal and homoscedastic distribution of the noise.
- Following the normalization approach of Greene (1993) which assumes a censoring point at zero, the authors transform the DEA scores to: yi = (1/θi)− 1, where θi is the DEA measure of physician i’s performance.
Discussion and Results
- The authors begin their analysis by examining the relationship between physicians’ scores on measures related to effectiveness and efficiency.
- This is a counterintuitive result, which questions the traditional belief that experience enables physicians to use hospital resources more frugally.
- The regression results of peer physician efficiency analysis (displayed in Table 4) show that the presence of a more effective peer is associated with (on average) an increase in physician efficiency.
- Nevertheless, the natural experiment setting discussed earlier removes potential biases, and hence, many of the associations the authors find may be causal.
Conclusions
- Using evidence from emergency medicine, the authors develop and analyze metrics for physicians’ effectiveness and efficiency.
- Unlike what the conventional wisdom suggests, their findings show that a physician’s effectiveness is positively (and not negatively) associated with his/her efficiency.
- The authors also find that highly efficient physicians have on average lower MRI orders per patient visit.
- The authors believe that their analysis serves as an early step to explore issues of physician effectiveness and efficiency.
- Importantly, the authors do not believe that the scores they develop are the 19 Electronic copy available at: https://ssrn.com/abstract=3227873 only ways to measure effectiveness or efficiency.
Did you find this useful? Give us your feedback
Citations
4 citations
References
25 citations
"Who Is an Efficient and Effective P..." refers methods in this paper
...While performance evaluation of hospitals has been explored in prior literature (Zheng et al. 2018, Castelli et al. 2015, Varabyova and Schreyogg 2013, Hollingsworth 2008), the performance of physicians has proven to be more difficult to assess because of diversity in patient mix and treatments,…...
[...]
25 citations
"Who Is an Efficient and Effective P..." refers methods in this paper
...…is considered to be more effective if the chance of returning to the ED (e.g., due to an unresolved issue) is minimized per hour spent in the ED.1 Both the LOS and 72-hour rate of return metrics have been used in the literature as valid measures (see, e.g., Chilingerian 1995, Fiallos et al. 2017)....
[...]
24 citations
"Who Is an Efficient and Effective P..." refers background in this paper
...Saghafian et al. (2018) study the speed-quality tradeoffs in a telemedical physician triage system in the context of an ED setting....
[...]
23 citations
"Who Is an Efficient and Effective P..." refers background or methods in this paper
...…a high 72-hour return rate is an undesirable indicator of care delivery effectiveness in the ED (see, e.g., Abualenain et al. 2013, Pham et al. 2011, Klasco et al. 2015), we use the proportion of patients discharged by a physician who did not return to the ED within 72 hours of their original…...
[...]
...The 72-hour rate of return has also been proposed as a measure of quality in the Emergency Medicine literature (see, e.g., Abualenain et al. 2013, Pham et al. 2011, Klasco et al. 2015) although using it for measuring quality (which is different than effectiveness) of care is controversial....
[...]
12 citations
"Who Is an Efficient and Effective P..." refers methods in this paper
...Collier et al. (2006) use the total billable charges attributed to physicians as one of the outputs of their proposed model....
[...]