Quantitative research assessment: using metrics against gamed metrics
Quantitative research assessment: using metrics against gamed metrics
Quantitative bibliometric indicators are widely used and widely misused for research assessments. Some metrics have acquired major importance in shaping and rewarding the careers of millions of scientists. Given their perceived prestige, they may be widely gamed in the current “publish or perish” or “get cited or perish” environment. This review examines several gaming practices, including authorship-based, citation-based, editorial-based, and journal-based gaming as well as gaming with outright fabrication. Different patterns are discussed, including massive authorship of papers without meriting credit (gift authorship), team work with over-attribution of authorship to too many people (salami slicing of credit), massive self-citations, citation farms, H-index gaming, journalistic (editorial) nepotism, journal impact factor gaming, paper mills and spurious content papers, and spurious massive publications for studies with demanding designs. For all of those gaming practices, quantitative metrics and analyses may be able to help in their detection and in placing them into perspective. A portfolio of quantitative metrics may also include indicators of best research practices (e.g. data sharing, code sharing, protocol registration, and replications) and poor research practices (e.g. signs of image manipulation). Rigorous, reproducible, transparent quantitative metrics that also inform about gaming may strengthen the legacy and practices of quantitative appraisals of scientific work.
Bibliometrics, Citations, Fraud, Gaming, Gift authorship, Impact factor, Research assessment, Self-citations
Ioannidis, John P.A.
d7938c59-470f-4b51-be62-34b986953d07
Maniadis, Zacharias
70ffa309-94c9-487c-982f-778294ea2a13
3 November 2023
Ioannidis, John P.A.
d7938c59-470f-4b51-be62-34b986953d07
Maniadis, Zacharias
70ffa309-94c9-487c-982f-778294ea2a13
Ioannidis, John P.A. and Maniadis, Zacharias
(2023)
Quantitative research assessment: using metrics against gamed metrics.
Internal and Emergency Medicine.
(doi:10.1007/s11739-023-03447-w).
Abstract
Quantitative bibliometric indicators are widely used and widely misused for research assessments. Some metrics have acquired major importance in shaping and rewarding the careers of millions of scientists. Given their perceived prestige, they may be widely gamed in the current “publish or perish” or “get cited or perish” environment. This review examines several gaming practices, including authorship-based, citation-based, editorial-based, and journal-based gaming as well as gaming with outright fabrication. Different patterns are discussed, including massive authorship of papers without meriting credit (gift authorship), team work with over-attribution of authorship to too many people (salami slicing of credit), massive self-citations, citation farms, H-index gaming, journalistic (editorial) nepotism, journal impact factor gaming, paper mills and spurious content papers, and spurious massive publications for studies with demanding designs. For all of those gaming practices, quantitative metrics and analyses may be able to help in their detection and in placing them into perspective. A portfolio of quantitative metrics may also include indicators of best research practices (e.g. data sharing, code sharing, protocol registration, and replications) and poor research practices (e.g. signs of image manipulation). Rigorous, reproducible, transparent quantitative metrics that also inform about gaming may strengthen the legacy and practices of quantitative appraisals of scientific work.
Text
Metrics Paper
- Accepted Manuscript
Text
s11739-023-03447-w
- Version of Record
More information
Accepted/In Press date: 26 September 2023
Published date: 3 November 2023
Additional Information:
Funding Information:
The work of John Ioannidis is supported by an unrestricted gift from Sue and Bob O’Donnell to Stanford. Zacharias Maniadis is supported by the project SInnoPSis, funded by Horizon 2020 under grant agreement ID: 857636.
Publisher Copyright:
© 2023, The Author(s).
Keywords:
Bibliometrics, Citations, Fraud, Gaming, Gift authorship, Impact factor, Research assessment, Self-citations
Identifiers
Local EPrints ID: 484746
URI: http://eprints.soton.ac.uk/id/eprint/484746
ISSN: 1828-0447
PURE UUID: a9c75501-dcc7-4cc6-beae-d728a9db4b11
Catalogue record
Date deposited: 21 Nov 2023 17:32
Last modified: 18 Mar 2024 03:22
Export record
Altmetrics
Contributors
Author:
John P.A. Ioannidis
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics
