Is More Information Better? The Effects of 'Report Cards' on Health Care Providers

TLDR: In this article, the authors used national data on Medicare patients at risk for cardiac surgery and found that cardiac surgery report cards in New York and Pennsylvania led both to selection behavior by providers and to improved matching of patients with hospitals.

Abstract: Health care report cards—public disclosure of patient health outcomes at the level of the individual physician or hospital or both—may address important informational asymmetries in markets for health care, but they may also give doctors and hospitals incentives to decline to treat more difficult, severely ill patients. Whether report cards are good for patients and for society depends on whether their financial and health benefits outweigh their costs in terms of the quantity, quality, and appropriateness of medical treatment that they induce. Using national data on Medicare patients at risk for cardiac surgery, we find that cardiac surgery report cards in New York and Pennsylvania led both to selection behavior by providers and to improved matching of patients with hospitals. On net, this led to higher levels of resource use and to worse health outcomes, particularly for sicker patients. We conclude that, at least in the short run, these report cards decreased patient and social welfare.

Figures

Table 1: Mean Expenditures in Year Prior to Admission for AMI or for CABG Surgery, Elderly Medicare Beneficiaries, 1990 and 1994

Table 2: Effects of Report Cards on the Within-Hospital Coefficient of Variation (CV) and Mean of Patients’ Health Status Before Treatment, Medicare Beneficiaries with AMI and Medicare Beneficiaries Receiving CABG, 1987-94

Table 6: Alternative Models of Effects of Report Cards on CABG Surgery Rates,

Table 5: Effects of Report Cards on Hospital Expenditures and Health Outcomes Medicare Beneficiaries with AMI, 1987-94

Table 4: Effects of Report Cards on CABG, PTCA, and Catheterization Rates Medicare Beneficiaries with AMI, 1987-94

Table 3: Effects of Report Cards for Teaching and All Other Hospitals on the Mean of Patients’ Health Status Before Treatment,

Full text

NBER WORKING PAPER SERIES
IS MORE INFORMATION BETTER?
THE EFFECTS OF ‘REPORT CARDS’ ON HEALTH CARE PROVIDERS
David Dranove
Daniel Kessler
Mark McClellan
Mark Satterthwaite
Working Paper 8697
http://www.nber.org/papers/w8697
NATIONAL BUREAU OF ECONOMIC RESEARCH
1050 Massachusetts Avenue
Cambridge, MA 02138
January 2002
The views expressed herein are those of the authors and not necessarily those of the National Bureau of
Economic Research or the US Government.
© 2002 by David Dranove, Daniel Kessler, Mark McClellan and Mark Satterthwaite. All rights reserved.
Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided
that full credit, including © notice, is given to the source.

Is More Information Better? The Effects of ‘Report Cards’ on Health Care Providers
David Dranove, Daniel Kessler, Mark McClellan and Mark Satterthwaite
NBER Working Paper No. 8697
January 2002
JEL No. I1, L5
ABSTRACT
Health care report cards - public disclosure of patient health outcomes at the level of the
individual physician and/or hospital - may address important informational asymmetries in markets for
health care, but they may also give doctors and hospitals incentives to decline to treat more difficult,
severely ill patients. Whether report cards are good for patients and for society depends on whether their
financial and health benefits outweigh their costs in terms of the quantity, quality, and appropriateness
of medical treatment that they induce. Using national data on Medicare patients at risk for cardiac
surgery, we find that cardiac surgery report cards in New York and Pennsylvania led both to selection
behavior by providers and to improved matching of patients with hospitals. On net, this led to higher
levels of resource use and to worse health outcomes, particularly for sicker patients. We conclude that,
at least in the short run, these report cards decreased patient and social welfare.
David Dranove Daniel Kessler
Kellogg School of Management Graduate School of Business
Northwestern University Stanford University
2001 Sheridan Road Stanford, CA 94305
Evanston, IL 60208 and NBER
fkessler@stanford.edu
Mark McClellan Mark Satterthwaite
Council of Economic Advisers Kellogg School of Management
17th and Pennsylvania Avenue, NW Northwestern University
Washington, DC 20502 2001 Sheridan Road
and NBER Evanston, IL 60208
markmc@stanford.edu

Is More In formation Better? The Effects of
’Report Cards’ on Health Care Providers
David Dranove,
*
Daniel Kessler,
**
Mark McClellan,
***
and Mark Satterthwaite
∗
Abstract
Health care report cards–public disclosure of patient health outcomes
at the level of the individual physician and/or hospital–may address im-
portant informational asymmetries in markets for health care, but they
may also giv e doctors and hospitals incentives to decline to treat more
difficult, severely ill patients. Whether report cards are good for patients
and for society depends on whether their Þnancial and health beneÞts out-
weigh their costs in terms of the quantity, quality, and appropriateness of
medical treatment that they induce. Using national data on Medicare
patients at risk for cardiac surgery, we Þnd that cardiac surgery report
cards in New York and Pennsylvania led both to selection behavior by
providers and to improved matching of patients with hospitals. On net,
this led to higher levels of resourc e use and to worse health outcomes,
particularly for sicker patients. We conclude that, at least in the short
run, these report cards decreased patient and social welfare.
1 Introduction
In the past few years, policy makers and researchers alike have giv en consid-
erable attent ion to quality “report cards” in sectors such as health care and
education. These report cards provide information about the performance of
hospitals, physicians, and schools where performance depends both on the skill
and effort of the producer and the characteristics of their patien ts/students.
Perhaps the best known health care report card is New York State’s publica-
tion of phy sician and hospital coronary artery bypass graft (CABG) surgery
∗
Northwestern University; **Stanford University and NBER; ***Council of Economic Ad-
visers, on leave from Stanford Universit y and NBER. We w ould like to thank David Becker
for exceptional research assistance, Paul Gertler, Paul Oy er, and Patrick Romano for valuable
comments, and seminar participan ts at the Boston University/Veterans Administration Bi-
ennial Health Economics Conference, NBER IO Workshop, Northw estern/Toulouse Joint IO
Seminar, Stanford Univ ersity, UCLA, University of Chicago, University of Illinois, University
of Texas, University of Toronto, and Yale Univ ersity for helpful suggestions. Funding from
the US National Institutes on Aging and the Agency for Health Care Research and Qual-
ity through the NBER is gratefully appreciated. The views expressed in this paper do not
represent those of the US Government or any other of the authors’ institutions.
1

mortality rates. Other states and private consulting Þrms also publish ho spital
mortality rates. Many private insurers and consortia of large employers use this
information when forming physician and hospital networks and as a means of
quality assurance.
The health policy community disagrees on the merits of report cards. Sup-
porters arg ue that they enable patients to iden tify the best physicians and hospi-
tals, while simultan eously giving providers powerful incentives to improve qual-
ity.
1
Skeptics counter that there are at least three reasons why report cards
ma y encourage providers to ”game” the system by av oiding sick and/or seek-
ing healthy patients. First, it is essential for the analysts who create report
cards to adjust health outcomes for differences in patient characteristics (”risk
adjustment”), for otherwise providers who treat the most serious cases necessar-
ily appear to have low qualit y. But analysts can only adjust for characteristics
that they can observ e. Unfortunately, because of the complexity of patient care,
providers are lik ely to ha ve better information on patients’ conditions than even
the most clinically detailed data base. For this reason, providers may be able to
improve their ranking by selecting patients on the basis of c haracteristics that
are unobserva ble to the analysts but predictive of good outcomes.
2
Even if providers do not have superior information on patients’ condition,
they ma y still have two other reasons to engage in selection. Suppose that the
difference in outcomes achie ved by low and high quality providers is greater for
sick patients. Considerable circumstantial evidence supports this assumption.
For example, Capps et al. (2001) Þnd that sick patients are more willing to
incur Þnancial and travel costs to obtain treatment from high quality providers,
suggesting that sick patients hav e more to gain from doing so. In this case, low
quality providers have strong incentives to avoid the sick and seek the healthy.
By shifting their practice toward healthier patients, inferior providers make it
difficult for report cards to conÞdently distinguish them from their high-quality
counterparts, because on relatively healthy patients they ha ve almost as good
outcomes. In other words, low-quality providers pool with their high-quality
counterparts.
Lastly, even if risk-adjustment were correct in expectation terms but incom-
plete — that is, risk-adjustment produces noisy estimates of true quality — it
may not compensate risk-averse pro viders sufficiently for the do w n side of treat-
ing sic k patients. The cost in utility terms to a risk-averse provider of accepting
a sick patient would be greater than the cost of accepting a healthy patient, as
long as the variance in the unexplained portion of outcomes is greater for the
sick than for the healthy. In practical terms, the utility loss from a few bad
1
Dranove and Satterthw aite (1992), which examines price and quality determination in
markets where consumers have noisy information about each, iden tiÞes sufficient conditions
for report cards on quality to lead to long run improvements in welfare. While we do not
study long run changes in this paper, there is anecdotal evidence that pro viders did take steps
to boost quality after the publication of report cards in New York.
2
For example, even if such comorbid diseases as diabetes or heart failure are measured
accurately for purposes of adjusting report cards, physicians who treat patients with more
severe or complex cases of diabetes or heart failure are still likely to hav e worse measured
performance.
2

(risk-adjusted) outcomes that dro ve a provider to the bottom of the rankings,
generated bad publicity, and catastrophically harmed his or her reputation ex-
ceeds the utility gain from a corresponding random positive shock.
3
The fact
that report cards are often based on small samples further aggravates both of
these incentive problems.
In this paper, we develop a comprehensive empirical framework for assessing
the competing claims about report cards. We apply this framework to the
adoption of mandatory CABG surgery report cards in New York (NY) and
Pennsylvania (PA) in the early 1990s. We begin by testing for three poten tial
effects of report cards on the treatment of cardiac illness:
• Thematchingofpatientstoproviders. If sick patients have more to
gain b y receiving treatment from high quality providers, then report cards
can improv e welfare through improved matching of patients to providers.
Sick patients disproportionately have an incentiv e to seek out the best
providers. In addition, the best providers have less incen tive to sh un the
sickest patient s.
• The incidence and quantity of CABG surgeries. Provider selection can
shift the incidence of CABG surgery from sic ker to healthier patien ts. At
the same time, the total number of surgeries may go up or do wn. As
clinicians ha ve pointed out, incidence effects can be socially harmful if
sicker patients derive the greatest beneÞt from bypass surgery (e.g., Topol
and Califf1994, note 21). On the other hand, they ma y be socially
constructiv e, if the equilibrium distribution of intensive treatment in the
absence of report cards is too hea vily weighted toward sicker patients.
• The incidenc e and quantity of complementary and substitute intensive c a r-
diac procedures. For example, a report-card induced decrease in CABG
surgeries for sick patients could lead to a shift towards other substitute
revascularization procedures, such as angioplast y (PTCA). However, if
doctors and hospitals institute processes to avoid mortality from inva-
sive cardiac procedures generally, then a report-card induced decrease
in CABG could be accompanied by a decrease in PTCA. In this case,
report-card induced decreases in therapeutic procedures suc h as CABG
and PTCA would be accompanied by decreases in complem entary diag-
nostic procedures such as cardiac catheterization (CATH). This too could
be welfare-improving or reducing, depending on the consequences of the
changing mix of treatment for health care costs and patient health out-
comes.
Then, we measure the net consequences of report cards for health care expen-
ditures and patien ts’ health outcomes.
We use a difference-in-difference (DD) approach to estimate the short-run
effects of report cards in the population of all U.S. elderly heart attack (AMI)
3
Dziuban et al. (1994) present a case study focusing on physicians’ concerns about the
incentives for selection generated by prediction errors in the New York CABG report card.
3

Frequently asked questions

Q1. What are the contributions in "Is more information better? the effects of ’report cards’ on health care providers" ?

Using national data on Medicare patients at risk for cardiac surgery, the authors find that cardiac surgery report cards in New York and Pennsylvania led both to selection behavior by providers and to improved matching of patients with hospitals. The authors conclude that, at least in the short run, these report cards decreased patient and social welfare. The Effects of ’Report Cards’ on Health Care Providers David Dranove, * Daniel Kessler, * * Mark McClellan, * * * and Mark Satterthwaite∗ 

Q2. What are the future works in "Is more information better? the effects of ’report cards’ on health care providers" ?

Future empirical work should analyze recent state initiatives that use detailed clinical data to report on populations of patients with speciÞc illnesses, in order to investigate if such design changes can address the shortcomings of procedure-based report cards. Future work should also measure if report cards in the long run cause providers to take steps to improve quality, a behavioral response that may dominate the short-run harm that the selection response caused during the period the authors examine here. 

Q3. What is the effect of discharge-abstract based report cards?

Since the national HCFA report card preceded state-level report cards and since discharge-abstract based report cards are more likely to suffer from noise and bias problems (e.g., Romano et al. 1999, Romano and Chan 2000), the discharge-abstract based report cards states produced are unlikely to have had noticeable effects on patient and provider behavior during their study period. 

Q4. Why do providers have better information on patients conditions than even the clinically detailed data base?

because of the complexity of patient care, providers are likely to have better information on patients conditions than even the most clinically detailed data base. 

Q5. How did the DD estimate the effect of report cards on the quantity of CABG surgery?

In particular, report cards increase the probability that the average AMI patient will undergo CABG surgery within 1 year of admission for AMI by 0.60 or 0.91 percentage points, depending on the assumed effective date of report cards. 

Q6. What did the authors find to be the effect of report cards on the quantity of CABG?

report cards led to increased delays in the execution of all three intensive treatments, signiÞcantly reducing the probability that an AMI patient would receive CABG, PTCA, or CATH within one day of admission. 

Q7. What is the effect of report cards on the rate of PTCA and CATH?

Although report cards did not affect the one-year CATH rate, they led, for both sick and healthy patients, to statistically signiÞcant declines in the one-day CATH rate, a measure of the rate at which patients are on a rapid track for subsequent intensive therapeutic treatment. 

Q8. What is the effect of report cards on the overall health of CABG patients?

The overall health status of CABG patients appears to improve as a result of report cards, with signiÞcantly lower rates of AMI and mortality. 

Q9. Why do the authors exclude psychiatric and rehabilitation hospitals from the analysis?

Because their analysis involves Medicare beneÞciaries with serious cardiac illness, the authors examine only nonfederal hospitals that ever reported providing general medical or surgical services (for example, the authors exclude psychiatric and rehabilitation hospitals from analysis). 

Q10. Why do providers have to use report cards to improve their ranking?

For this reason, providers may be able to improve their ranking by selecting patients on the basis of characteristics that are unobservable to the analysts but predictive of good outcomes.