scispace - formally typeset
Search or ask a question
Journal ArticleDOI

The impact of research grant funding on scientific productivity

01 Oct 2011-Journal of Public Economics (NIH Public Access)-Vol. 95, Iss: 9, pp 1168-1177
TL;DR: In this article, the authors estimate the impact of receiving an NIH grant on subsequent publications and citations and show that the loss of a grant simply causes researchers to shift to another source of funding, consistent with a model in which the market for research funding is competitive.
About: This article is published in Journal of Public Economics.The article was published on 2011-10-01 and is currently open access. It has received 414 citations till now. The article focuses on the topics: Receipt.

Summary (3 min read)

Introduction

  • The authors sample consists of all applications (unsuccessful as well as successful) to the NIH from 1980 to 2000 for postdoctoral training grants (F32s) and standard research grants (R01s).
  • The estimates represent about 20 and 7 percent increases in research productivity for F32 and R01 recipients respectively.

II. Prior Literature

  • As alluded to above, there is little persuasive evidence regarding the impact of resources on the production of scientific output.
  • A few earlier studies have examined the effect of research funding provided by NIH in particular.
  • Applications (R grants) are assigned a percentile score, which ranks the proposal in relationship to other recent applications from the review group.
  • For this reason, the authors conduct the discontinuity analysis for research grants within units underneath the institute level, such as branches, programs or divisions.
  • Because a disproportionate share of funding is saved for the last cycle of the fiscal year, most applications are funded in the same fiscal year they are received or not at all.

IV. Methodology

  • Having discussed the context within which NIH grants operate, the authors will now outline the methodological issues relevant to the paper.
  • The authors hope is that by controlling adequately for the priority score of the grant application, researcher characteristics, and prior productivity the authors can identify the approximate causal effect of grant receipt on subsequent research productivity.
  • Thus their non-experimental estimates of the productivity impacts of NIH grants may suffer from little bias.
  • To the extent that marginal, unfunded applicants are likely to receive funding if they reapply, missing the cutoff in any particular round (i.e., year) will be less predictive of eventual grant receipt than it is of immediate grant receipt.
  • Measurement error in the dependent variable does complicate their use of nonlinear estimation methods, as is the case when estimating models to determine whether receiving an NIH grant increases the probability that a researcher surpasses a particular threshold of productivity.

V. Data

  • This study relies on several data sources.
  • 12 Regardless of the nature or extent of the measurement error, it is important to keep in mind that it will not affect the consistency of their estimates as long as the measurement error is not correlated with priority scores in the same nonlinear fashion as the funding cutoff.
  • The majority of their outcomes reflect the degree of productivity, including the number of publications, citations, and subsequent grant funding, in different time periods following the NIH grant application.
  • When examining NIH funding for the five years subsequent to the application date, the lack of recent data causes problems for grant applications submitted after 1998.
  • To minimize measurement error, the authors focus on the 44 percent of F32 and the 45 percent of R01 applications in which the applicants have uncommon names, defined as those whose last name was associated with 10 or fewer unique NIH applicants during their time period.

VI. Findings

  • The results are presented in three parts.
  • First, the authors present results from baseline OLS estimates to gain a basic understanding of associations in the data and to explore how much selection on observables appears to exist in the NIH funding process.
  • This section also explains several of their outcome measures in greater detail, and discusses several important issues regarding interpretation.
  • The authors will present the main results, followed by a series of sensitivity analyses and extensions.
  • The third section covers the results for research program grants.

A. Baseline OLS Estimates

  • The IV estimate for research direction of 0.08 is not statistically different from the OLS estimate of 0.12, but also not statistically different from zero.
  • 15 The IV estimates on the number of publications was large (1.2 publications relative to a baseline of 5.8) but statistically significant at only the ten percent level.

D. Sensitivity Analysis and Other Extensions

  • If the authors focus specifically on the measures of pre-treatment productivity, they see that researchers who were awarded a grant because they scored just below the cutoff had lower productivity prior to the grant application.
  • Row 2 presents IV estimates with no covariates other than institute and year fixed effects and a quadratic in the priority score.
  • While the number of comparisons suggests that one should be cautious in interpreting any differences, several interesting patterns emerge.
  • Receipt of an NIH grant has a significantly larger impact for researchers in the biological sciences than those in the physical or social sciences.

VII. Discussion

  • The results above suggest that NIH postdoctoral fellowships result in significant relative increases in research productivity.
  • An equilibrium in this market is defined by an allocation of money to researchers such that the marginal productivity of money is the same for all funded researchers and the total supply of funding available for grants equals the total amount of money received by all researchers.
  • In other words, one would expect that the causal impact of receiving a particular grant on subsequent research to be quite small, even if the marginal impact of funding were positive.
  • In order to explore how receipt of NIH funding impacts other sources of funding, the authors collected more complete funding information for a sub-sample of researchers in their data.
  • Of the articles the authors were able to locate, 83 percent reported some funding and the average (standard deviation) number of different funding sources among this group is 2.9 (1.6).

VIII. Conclusion

  • The authors utilize a regression discontinuity design to estimate the causal impact of NIH funding on scientific output.
  • This may be less than one for a number of reasons.
  • Second, the published work may have been performed prior to grant receipt.
  • Note that if researchers do not use NIH funding for all research projects they are working on, the authors may not expect receipt of one particular grant to be critical for an individual’s entire research effort.
  • This analysis presents the first step toward examining the effectiveness of government expenditures in R&D.

Did you find this useful? Give us your feedback

Citations
More filters
Journal ArticleDOI
TL;DR: In this article, the authors evaluate the impact of an R&D subsidy program implemented in a region of northern Italy on innovation by beneficiary firms and find that the program had a significant impact on the number of patents, more markedly in the case of smaller firms.
Abstract: This paper evaluates the impact of an R&D subsidy program implemented in a region of northern Italy on innovation by beneficiary firms. In order to verify whether the subsidies enabled firms to increase patenting activity, we exploit the mechanism used to allot the funds. Since only projects that scored above a certain threshold received the subsidy, we use a sharp regression discontinuity design to compare the number of patent applications, and the probability of submitting one, of subsidized firms with those of unsubsidized firms close to the cut-off. We find that the program had a significant impact on the number of patents, more markedly in the case of smaller firms. Our results show that the program was also successful in increasing the probability of applying for a patent, but only in the case of smaller firms.

452 citations

Journal ArticleDOI
TL;DR: In this article, the authors evaluate the impact of an R&D subsidy program implemented in a region of northern Italy in the early 2000s on innovation by beneficiary firms using a regression discontinuity design strategy to assess the effect of the grants on the number of patent applications and the likelihood of submissions by subsidized firms.

316 citations

Journal ArticleDOI
TL;DR: The results show that winners just above the funding threshold accumulate more than twice as much funding during the subsequent eight years as nonwinners with near-identical review scores that fall just below the threshold, suggesting that early funding itself is an asset for acquiring later funding.
Abstract: A classic thesis is that scientific achievement exhibits a “Matthew effect”: Scientists who have previously been successful are more likely to succeed again, producing increasing distinction. We investigate to what extent the Matthew effect drives the allocation of research funds. To this end, we assembled a dataset containing all review scores and funding decisions of grant proposals submitted by recent PhDs in a €2 billion granting program. Analyses of review scores reveal that early funding success introduces a growing rift, with winners just above the funding threshold accumulating more than twice as much research funding (€180,000) during the following eight years as nonwinners just below it. We find no evidence that winners’ improved funding chances in subsequent competitions are due to achievements enabled by the preceding grant, which suggests that early funding itself is an asset for acquiring later funding. Surprisingly, however, the emergent funding gap is partly created by applicants, who, after failing to win one grant, apply for another grant less often.

296 citations

Journal ArticleDOI
TL;DR: In this article, an Italian program of subsidies for the applied development of innovations, exploiting a discontinuity in programme financing due to an unexpected shortage of public money, is investigated. But the results indicate that the programme was not effective in stimulating innovative investment.
Abstract: To evaluate the effect of an R&D subsidy one needs to know what the subsidized firms would have done without the incentive. This paper studies an Italian programme of subsidies for the applied development of innovations, exploiting a discontinuity in programme financing due to an unexpected shortage of public money. To identify the effect of the programme, the study implements a regression discontinuity design and compares firms that applied for funding before and after the shortage occurred. The results indicate that the programme was not effective in stimulating innovative investment.

224 citations

References
More filters
Journal ArticleDOI
TL;DR: In this paper, the authors compare the geographic location of patent citations to those of cited patents, as evidence of the extent to which knowledge spillovers are geographically localized, and find that citations to U.S. patents are more likely to come from the U. S., and more likely than coming from the same state and SMSA as cited patents than one would expect based only on the preexisting concentration of related research activity.
Abstract: We compare the geographic location of patent citations to those of the cited patents, as evidence of the extent to which knowledge spillovers are geographically localized. We find that citations to U.S. patents are more likely to come from the U.S., and more likely to come from the same state and SMSA as the cited patents than one would expect based only on the preexisting concentration of related research activity. These effects are particularly significant at the local (SMSA) level, and are particularly apparent in early citations.

5,937 citations

Journal ArticleDOI
05 Jan 1968-Science
TL;DR: The psychosocial conditions and mechanisms underlying the Matthew effect are examined and a correlation between the redundancy function of multiple discoveries and the focalizing function of eminent men of science is found—a function which is reinforced by the great value these men place upon finding basic problems and by their self-assurance.
Abstract: This account of the Matthew effect is another small exercise in the psychosociological analysis of the workings of science as a social institution. The initial problem is transformed by a shift in theoretical perspective. As originally identified, the Matthew effect was construed in terms of enhancement of the position of already eminent scientists who are given disproportionate credit in cases of collaboration or of independent multiple discoveries. Its significance was thus confined to its implications for the reward system of science. By shifting the angle of vision, we note other possible kinds of consequences, this time for the communication system of science. The Matthew effect may serve to heighten the visibility of contributions to science by scientists of acknowledged standing and to reduce the visibility of contributions by authors who are less well known. We examine the psychosocial conditions and mechanisms underlying this effect and find a correlation between the redundancy function of multiple discoveries and the focalizing function of eminent men of science—a function which is reinforced by the great value these men place upon finding basic problems and by their self-assurance. This self-assurance, which is partly inherent, partly the result of experiences and associations in creative scientific environments, and partly a result of later social validation of their position, encourages them to search out risky but important problems and to highlight the results of their inquiry. A macrosocial version of the Matthew principle is apparently involved in those processes of social selection that currently lead to the concentration of scientific resources and talent ( 50 ).

5,689 citations

Posted Content
TL;DR: In this article, the existence of geographically mediated "spillovers" from university research to commercial innovation is explored using state-level time-series data on corporate patents, corporate R&D, and university research.
Abstract: The existence of geographically mediated "spillovers" from university research to commercial innovation is explored using state-level time-series data on corporate patents, corporate R&D, and university research. A significant effect of university research on corporate patents is found, particularly in the areas of drugs and medical technology, and electronics, optics, and nuclear technology. In addition, university research appears to have an indirect effect on local innovation by inducing industrial R&D spending. Copyright 1989 by American Economic Association.

3,236 citations

Journal ArticleDOI
TL;DR: In this article, the authors show that identifying conditions invoked in previous applications of regression discontinuity methods are often overly strong and that treatment effects can be nonparametrically identified under an RD design by a weak functional form restriction.
Abstract: Ž. THE REGRESSION DISCONTINUITY RD data design is a quasi-experimental design with the defining characteristic that the probability of receiving treatment changes discontinuously as a function of one or more underlying variables. This data design arises frequently in economic and other applications but is only infrequently exploited as a source of identifying information in evaluating effects of a treatment. In the first application and discussion of the RD method, Thistlethwaite and Campbell Ž. 1960 study the effect of student scholarships on career aspirations, using the fact that awards are only made if a test score exceeds a threshold. More recently, Van der Klaauw Ž. 1997 estimates the effect of financial aid offers on students’ decisions to attend a particular college, taking into account administrative rules that set the aid amount partly on the basis of a discontinuous function of the students’ grade point average and SAT Ž. score. Angrist and Lavy 1999 estimate the effect of class size on student test scores, taking advantage of a rule stipulating that another classroom be added when the average Ž. class size exceeds a threshold level. Finally, Black 1999 uses an RD approach to estimate parents’ willingness to pay for higher quality schools by comparing housing prices near geographic school attendance boundaries. Regression discontinuity methods have potentially broad applicability in economic research, because geographic boundaries or rules governing programs often create discontinuities in the treatment assignment mechanism that can be exploited under the method. Although there have been several discussions and applications of RD methods in the literature, important questions still remain concerning sources of identification and ways of estimating treatment effects under minimal parametric restrictions. Here, we show that identifying conditions invoked in previous applications of RD methods are often overly strong and that treatment effects can be nonparametrically identified under an RD design by a weak functional form restriction. The restriction is unusual in that it requires imposing continuity assumptions in order to take advantage of the known discontinuity in the treatment assignment mechanism. We also propose a way of nonparametrically estimating treatment effects and offer an interpretation of the Wald estimator as an RD estimator.

2,577 citations

BookDOI
01 Jan 2005
TL;DR: In this article, Nunez et al. present the CECE model, a new theory of success among Racially Diverse College Student Populations (CECE) model, and the Completion Agenda, the Unintended Consequences for Equity in Community Colleges.
Abstract: 1. The Complexity of Higher Education: a Career in Academics and Activism Philip G. Altbach.- 2. Advancing an Intersectionality Framework in Higher Education: Power and Latino Postsecondary Opportunity Anne-Marie Nunez.- 3. Student Veterans in Higher Education David T. Vacchi and Joseph B. Berger.- 4. The Changing Nature of Cultural Capital Jenna R. Sablan and William G. Tierney.- 5. The Culturally Engaging Campus Environments (CECE) Model: A New Theory of Success among Racially Diverse College Student Populations Samuel D. Museus.- 6. Organizational Identity in Higher Education: Conceptual and Empirical Perspectives David J. Weerts, Gwendolyn H. Freed and Christopher C. Morphew.- 7. Student Ratings of Instruction in College and University Courses Stephen L. Benton and William E. Cashin.- 8. College Enrollment: an Economic Analysis Leslie S. Stratton.- 9. The Welding of Opposite Views: Land-Grant Historiography at 150 Years Nathan M. Sorber and Roger L. Geiger.- 10. The Completion Agenda: The Unintended Consequences for Equity in Community Colleges Jaime Lester.- 11. Using IPEDS for Panel Analyses: Core Concepts, Data Challenges, and Empirical Applications Ozan Jaquette and Edna E. Parra.- 12. Toward a Better Understanding of Equity in Higher Education Finance and Policy Luciana Dar.

2,366 citations

Frequently Asked Questions (13)
Q1. What are the contributions in this paper?

In this paper, the authors estimate the impact of receiving an NIH grant on subsequent publications and citations. 

Given the importance of technological innovation for economic growth and the considerable public resources devoted to R & D, further research is clearly warranted. In future work, the authors plan to explore the impact of NIH funding on patents, which may be a more useful measure of societal value. 

While the existence of out-of-order funding, rejected awards, and reapplication makesa sharp RD design infeasible, it is still possible to leverage the nonlinear relationship between normalized priority score and the probability of eventual grant receipt to identify the causal impact of research funding. 

3Because funding decisions are made within institutes (in contrast to research grantproposals, which are evaluated by review groups examining applications from different institutes), the NIH normalizes scores within review groups. 

On average, the sampled articles listed 2.45 sources of funding, with about 30 percent of articles listing at least three different sources of funding. 

In the United States, for example, the National Insittutes of Health (NIH) and the National Science Foundation (NSF) allocate over $30 billion annually for basic and applied research in the sciences. 

The authors also drop 5,089 R01 applications from institute-years in which grants did not appear to be allocated strictly on the basis of the observed priority score cutoff. 

Postdoctoral fellowships have a significantly greater impact on researchers in the social sciences than those in either the biological or physical sciences in terms of publications and citations. 

Since name frequency is unlikely to be correlated with whether an individual is just above or below the funding cutoff (conditional on flexible controls for her priority score), this restriction will not influence the consistency of their estimates. 

There are several ways in which unsuccessful researchers might obtain funding to continue their research: (1) they might obtain funding from another source, such as the NSF, a private foundation or their home institution; (2) they might collaborate with another researcher who was successful at obtaining NIH funding; or (3) they might collaborate with another researcher who was successful at obtaining non-NIH funding. 

Their second approach relies upon the fact that NIH funding is awarded on the basis ofobservable priority scores, and that there is a highly nonlinear relationship between this score and the probability of funding. 

Because of this, the local average treatment effect (LATE) implicitly compares the productivity of applicants who received a grant because of a low application score to that of applicants who were rejected due to a higher score (controlling for a smooth function of the normalized application score). 

One possibility is that NIH funding could displace funding from other public agencies or private entities, either because the researcher is less inclined to apply for such funding if she has already received an NIH award or because other funding agencies correctly perceive the marginal utility of an additional dollar to a funded researcher isless valuable than an additional dollar to an unfunded researcher.