Assessing the Impact and Quality of Research Data Using Altmetrics and Other Indicators
Stacy Konkiel
- Vol. 2, Iss: 1
Reads0
Chats0
TLDR
The research into research data metrics, these metrics’ strengths and limitations with regard to formal evaluation practices, and the possible meanings of such indicators are discussed, and heuristics for policymakers and evaluators interested in doing so are suggested.Abstract:
Research data in all its diversity—instrument readouts, observations, images, texts, video and audio files, and so on—is the basis for most advancement in the sciences. Yet the assessment of most research programmes happens at the publication level, and data has yet to be treated like a first-class research object. How can and should the research community use indicators to understand the quality and many potential impacts of research data? In this article, we discuss the research into research data metrics, these metrics’ strengths and limitations with regard to formal evaluation practices, and the possible meanings of such indicators. We acknowledge the dearth of guidance for using altmetrics and other indicators when assessing the impact and quality of research data, and suggest heuristics for policymakers and evaluators interested in doing so, in the absence of formal governmental or disciplinary policies. Policy highlights Research data is an important building block of scientific production, but efforts to develop a framework for assessing data’s impacts have had limited success to date. Indicators like citations, altmetrics, usage statistics, and reuse metrics highlight the influence of research data upon other researchers and the public, to varying degrees. In the absence of a shared definition of “quality”, varying metrics may be used to measure a dataset’s accuracy, currency, completeness, and consistency. Policymakers interested in setting standards for assessing research data using indicators should take into account indicator availability and disciplinary variations in the data when creating guidelines for explaining and interpreting research data’s impact. Quality metrics are context dependent: they may vary based upon discipline, data structure, and repository. For this reason, there is no agreed upon set of indicators that can be used to measure quality. Citations are well-suited to showcase research impact and are the most widely understood indicator. However, efforts to standardize and promote data citation practices have seen limited success, leading to varying rates of citation data availability across disciplines. Altmetrics can help illustrate public interest in research, but availability of altmetrics for research data is very limited. Usage statistics are typically understood to showcase interest in research data, but infrastructure to standardize these measures have only recently been introduced, and not all repositories report their usage metrics to centralized data brokers like DataCite. Reuse metrics vary widely in terms of what kinds of reuse they measure (e.g. educational, scholarly, etc). This category of indicator has the fewest heuristics for collection and use associated with it; think about explaining and interpreting reuse with qualitative data, wherever possible. All research data impact indicators should be interpreted in line with the Leiden Manifesto’s principles, including accounting for disciplinary variation and data availability. Assessing research data impact and quality using numeric indicators is not yet widely practiced, though there is generally support for the practice amongst researchers.read more
Citations
More filters
Journal Article
The State of OA: A large-scale analysis of the prevalence and impact of Open Access articles
Journal ArticleDOI
Are data repositories fettered? A survey of current practices, challenges, and future technologies
TL;DR: This study identifies the current challenges and needs for improving data repository functionalities and user experiences, and makes main recommendations for future repository systems.
Journal ArticleDOI
The data-index: An author-level metric that values impactful data and incentivizes data sharing
TL;DR: Data-index as mentioned in this paper is a new author-level metric, which values both dataset output (number of datasets) and impact(number of data-index citations), so promotes generating and sharing data as a result.
Journal ArticleDOI
Evaluación de la investigación científica: mejorando las políticas científicas en Latinoamérica
TL;DR: A revisión bibliográfica, basada en artículos sobre políticas de evaluación de la investigación científica and agendas internacionales implementadas en los últimos años (principalmente en el Reino Unido, Estados Unidos, Australia, China and Latinoamérica) as mentioned in this paper , indicate que no existe un solo método de evalueación, and that ningún indicador es absoluto.
Journal ArticleDOI
Istraživački podatci hrvatskih autora na platformi Web of Science
TL;DR: Istraživački podatci (engl. research data) as discussed by the authors, i.e., istraživanjem, prikupljaju and bilježe tijekom eksperimenta, promatranja, modeliranja, intervjua, and sl., are the most common types of prikopljaje.
References
More filters
Journal ArticleDOI
Anatomy of open access publishing: a study of longitudinal development and internal structure
Mikael Laakso,Bo-Christer Björk +1 more
TL;DR: OA journal publishing is disrupting the dominant subscription-based model of scientific publishing, having rapidly grown in relative annual share of published journal articles during the last decade.
Journal ArticleDOI
Evaluation practices and effects of indicator use : a literature review
TL;DR: A review of the international literature on evaluation systems, evaluation practices, and metrics (mis)uses was written as part of a larger review commissioned by the Higher Education Funding Co....
Journal ArticleDOI
Australia's continental-scale acoustic tracking database and its automated quality control process.
Xavier Hoenner,Charlie Huveneers,Andre Steckenreuter,Colin A. Simpfendorfer,Katherine Tattersall,Fabrice R. A. Jaine,Natalia Atkins,Russell C. Babcock,Stephanie Brodie,Jonathan Burgess,Hamish A. Campbell,Michelle R. Heupel,Bénédicte Pasquer,Roger Proctor,Matthew D. Taylor,Matthew D. Taylor,Vinay Udyawer,Robert Harcourt +17 more
TL;DR: The database and quality control procedures developed to collate 49.6 million valid detections from 1891 receiving stations are presented, which constitutes a valuable resource facilitating meta-analysis of animal movement, distributions, and habitat use and is important for relating species distribution shifts with environmental covariates.
Journal ArticleDOI
The notion of data and its quality dimensions
TL;DR: The most important dimensions of data quality: accuracy, completeness, consistency and currentness are defined and several related dimensions are discussed and defined and discussed in detail.
Journal ArticleDOI
Author Correction: Imaging and clinical data archive for head and neck squamous cell carcinoma patients treated with radiotherapy
Aaron J. Grossberg,Aaron J. Grossberg,Abdallah S.R. Mohamed,Abdallah S.R. Mohamed,Hesham Elhalawani,William C. Bennett,Kirk E. Smith,Tracy S. Nolan,Bowman Williams,Sasikarn Chamchod,J. Heukelom,J. Heukelom,Michael E. Kantor,Theodora Browne,Katherine A. Hutcheson,G. Brandon Gunn,Adam S. Garden,William H. Morrison,Steven J. Frank,David I. Rosenthal,John Freymann,Clifton D. Fuller +21 more
TL;DR: In the original version of the Data Descriptor the surname of author Hesham Elhalawani was misspelled and this has now been corrected in the HTML and PDF versions.