An academic scientist’s professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, disciplinary incentives encourage design, analysis, and reporting decisions that elicit positive results and ignore negative results. Prior reports demonstrate how these incentives inflate the rate of false effects in published science. When incentives favor novelty over replication, false results persist in the literature unchallenged, reducing efficiency in knowledge accumulation. Previous suggestions to address this problem are unlikely to be effective. For example, a journal of negative results publishes otherwise unpublishable reports. This enshrines the low status of the journal and its content. The persistence of false findings can be meliorated with strategies that make the fundamental but abstract accuracy motive—getting it right—competitive with the more tangible and concrete incentive—getting it published. This article develops strategies for improving scientific practices and knowledge accumulation that account for ordinary human motivations and biases.

/pdf/scientific-utopia-ii-restructuring-incentives-and-practices-4ll5s2mnqk.pdf

Scientific Utopia II. Restructuring Incentives and Practices to Promote Truth Over Publishability

An academic scientist’s professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, disciplinary incentives encourage design, analysis, and reporting decisions that elicit positive results and ignore negative results. Prior reports demonstrate how these incentives inflate the rate of false effects in published science. When incentives favor novelty over replication, false results persist in the literature unchallenged, reducing efficiency in knowledge accumulation. Previous suggestions to address this problem are unlikely to be effective. For example, a journal of negative results publishes otherwise unpublishable reports. This enshrines the low status of the journal and its content. The persistence of false findings can be meliorated with strategies that make the fundamental but abstract accuracy motive – getting it right – competitive with the more tangible and concrete incentive – getting it published. We develop strategies for improving scientific practices and knowledge accumulation that account for ordinary human motivations and self-serving biases.

Scientific Utopia: II - Restructuring Incentives and Practices to Promote Truth Over Publishability

Data sharing

https://repository.arizona.edu/bitstream/10150/105807/1/RobbinLossOfPersonalPrivacyJGI2001.pdf

The loss of personal privacy and its consequences for social research

The sharing of research data to permit secondary analyses is rooted in the collegial tradition of science. Recently there has been a gradual shift from viewing data sharing as a voluntary activity to considering sharing as an obligation of researchers. This paper identifies two important dimensions of data sharing: the degree to which the primary investigators may determine whether they will share their data and the reason for which the data set is being requested. In addition to discussing the dimensions of data sharing, the negative aspects of viewing data sharing as an obligation are discussed. These negative factors are increased burden on the primary investigator; lack of incentive to share data; and loss of control over the use of data and negative effects on scientific progress. Recommendations regarding data sharing policies are made in light of these negative effects.

Data sharing: The primary researcher's perspective.

This paper is based on a programmatic effort to elicit and reanalyze data stemming from the evaluation of social programs. The stress is on identifying problems and negotiating for data and analyzing it. This includes: researchers' reluctance to disclose data for documentation and data control systems; vague or nonexistent policy for assuring access. The possible solutions presented here are based on governmental regulation and policy for publicly supported evaluation.

Terry E. Hedrick

Papers

On ensuring the availability of evaluative data for secondary analysis