Highlights

1. The peer-review process is critical in academic publishing.
2. Referees’ reports should help authors improve their manuscripts before publication.
3. There are three primary types of peer review: open, single-blind, and double-blind.

Abstract

Peer review is a method for establishing quality or scholarly legitimacy for research. In research, peer review is used to determine whether an academic paper is worthy of publication. In the case of scholarly articles, peer reviewers evaluate if an article meets the standards of a learned journal. The process has a long history in academia and is now widely recognized as one of the most important ways to foster quality research and ensure that findings have external validity. One of the most necessary aspects of a review is that it needs to be written in a helpful and understandable way. In this article, the historical paradigm and new trends in academic peer-review have been explained, including various benefits and disadvantages of the whole peer review system, possibilities to make peer reviews open access, and more. We all use peer review in different ways but rarely stop to think about how it’s supposed to work.

Keywords

Peer Review | Reviewers | Author | Research | Editor

Introduction

Peer review is the assessment of the work done by authors within the same discipline. It serves as a gatekeeper for academic publishing in which experts assess the quality of research before it is published in academic journals. Peer review is intended to ensure that articles meet certain standards of quality, rigour, relevance and significance, and originality. The process is critical for ensuring quality in academic publishing. It provides editors with expert advice on handling manuscripts and holds authors accountable for their research through rigorous evaluation.

The process is also slow and labour-intensive for editors, however, who must handle submissions in a timely manner while ensuring that all authors and reviewers are treated fairly. To begin the review process, authors submit their manuscript to an editor who will decide whether or not it is ready for consideration for publication via peer review. The editor will then select reviewers in relevant subjects to review the submission, who usually commit in this process to reading and giving detailed feedback on a manuscript multiple times before confirming that it is of sufficient quality to be published.

Peer review has been a base of the scholarly communication system since its inception. Yet, despite its successes, it also draws a fair share of criticism.

One strand of such criticism is that peer review is undemocratic and works against equality and diversity. But, the idea of democratizing peer review does raise some interesting questions. How can we ensure that peer review is more inclusive and representative?

“Perhaps unsurprisingly, peer review has become one of the most mysterious and contentious academic practices, causing anguish for many academics—both reviewers and those whose work is reviewed—and sometimes more distress than is necessary.”

— Is peer review in academic publishing still working? By Liz Jackson, et al.

Workflow of the Bustling Process of Peer-review

The Instrumental Side of Peer-review

Peer reviews are an indispensable part of the scholarly publishing process, but very little is known by the junior scholar about what happens during peer reviews. Peer reviews are generally treated as confidential documents. Nevertheless, they have always been a major part of communication in the academic community. The referees’ reports ideally provide vital information on new developments in the field, and other feedback about how communication and expression in the piece obstruct its readability, which helps authors improve their manuscripts before publication. However, in traditional peer review processes, these reports are only seen by the authors and editors of the journal, even though they might contain insights, comments, suggestions, and questions that could be useful for other researchers.

There are three primary types of peer review: open, single-blind, and double-blind. Each method has its own advantages and disadvantages.

• An open review (a.k.a. transparent review) is a type of peer review that allows the authors and reviewers to know the other’s identity. Open peer review can help to prevent the reviewer from leaving careless or harmful derogatory comments or unnecessarily procrastinating on the completion of the review. It promotes reviewers to be open and honest without sounding disrespectful.
• A single-blind review is a type of peer review in which the identity of the author is revealed, but the reviewer remains anonymous. Because the author isn’t aware of who the reviewer is, this gives the reviewer greater flexibility and freedom to provide a more impartial critique, which in theory gives authors an improved chance at receiving honest and rigorous feedback.
• A double-blind review is a type of peer review in which the identities of both authors and reviewers are unknown to each other. As there is no known relationship between the reviewer and the author, the author’s identity or reputation will not be a source of bias for the reviewer. Even authors with magnificent publication histories are judged in this case more based on the perceived merit of the work itself.

The Emerging Lamentable State of Peer-review

Peer review is the main method used traditionally and today for vetting academic literature prior to publication. However, it has faced severe criticism in recent years for various reasons. In some cases, reviewers have failed to spot serious errors in an author's methodology. Some reasons for such 'failures' in the peer review process include reviewers' heavy workloads, as more papers are published while the time demands on academics are often quite high, as well as poor selections of reviewers by editors, who may have difficulty finding appropriate and expert peer reviewers willing to conduct reviews.

Peer reviewers are usually experienced scholars who read the article submitted by an author, assess its merit, and comment on specific concerns they may have about it. They help the editor decide on whether to accept or reject the article for publication. They also help authors improve their work and possibly get their work accepted for publication in other journals in the case that the match is not right between an article and a journal.

However, major drawbacks and concerns about peer-review include the following:

• Outdated

The number of papers published has increased fiercely over the past few decades, while the pool of willing peer reviewers has not kept pace with this growth.

• Bias

A controversial topic but worth talking about: The review process is one of many areas in academic work and research where women face a lot of difficulties. A study shows that,

“Women are underrepresented in the peer-review process and that editors of both genders operate with substantial same-gender preference (homophily) when appointing reviewers.”

Likewise, bias can occur against non-native English speakers and ethnic minorities within the context of a scholarly journal and its mainstream or elite editorial board and reviewer community.

• Slow

The peer-review process is one of the most important but also one of the longest and most tedious steps in academic publishing. It is now sometimes also considered to be outdated. Long delays are seen as a weakness in the system as quality work may not be published or seen for years after the original author submission.

• Dishonesty

No matter how professional editors as scholars are, they are human. They are likely to make mistakes sometimes. But sometimes editors can choose reviewers in an inappropriate manner. This can happen with editors invite as reviewers their personal friends (who might be tempted to say nice things to get work published), or known critics (who might be known to give damning reviews on a regular basis).

“How often editors elect to use ‘good’ or critical reviewers without exhausting or overworking them, and the potential consequences this might have on professional or personal relationships between the different parties and their respective reputations.”

— The limitations to our understanding of peer review

The Plan Of Action

The Peer Reviewers' Openness Initiative believes that “openness and transparency are core values of science” and supports the spread of open research practices. Central to this idea is the belief that reviewers can engage authors on issues of scientific openness during the review process. Meanwhile, journals like nature have adopted policies to make the process healthier for authors as well as reviewers.

“We strive to innovate in our peer-review process to improve its efficacy, quality, and transparency. We are constantly monitoring and reviewing our efforts to address more closely the needs of the research community.”

— Nature’s Initiatives

Some alternatives are also proposed to tackle the underlying problems of the process, such as:

• Displaying reviewers’ suggestions along with the published paper
• Allowing authors to move reviews from one journal to another
• Running a public review system
• Training reviewers

Peer review is familiar. It seems to work. It feels solid, and reliable, even though it is flawed.

Open and Published Peer-Review

Ideally, peer review is educational for both the reviewer and the author. Both contribute to shaping an article that opens up the field to the conversation while also playing crucial roles in assessing the quality of data and claims. To consider peer review as pedagogical does, however, raise a question: What value does anonymity add? One of the concerns of anonymous peer reviewing is that a reviewer can lose sight of the duty to act in the best interests of an author and the field. Likewise, an author may be affronted by a reviewer’s comments and vent frustrations to the editor. To be a competent peer reviewer demands expert knowledge. An expert reviewer couples knowledge with the wisdom of humility, and a desire to serve.

Another possibility is a review by known reviewers who do not know the identity of authors. Interestingly, some reviewers have taken up this idea so enthusiastically that they now self-publish their review reports on their ORCID profiles. Among the advantages of reviewers being identified (either to the authors only or to the readers of the published article) is to prevent ‘author bashing’ by reviewers who write behind the safety of the double-anonymous review. Open review could also prevent property theft: reviewers stealing ideas from the article they review. Furthermore, published reviews can enhance an early researcher’s profile.

Yet published reviews also run some risks. First, open reviews can be too pallid and consensual. Other risks to non-anonymous reviewing come with the rise of social media.

Conclusion

The future of peer review is hard to predict within the dynamic domain of publishing and information sharing. The age of digital reason has brought about a new logic of co-creation and co-production of knowledge, along with various novel models of peer review. For instance, the Video Journal of Education and Pedagogy is unable to conduct anonymous peer review because video articles, as a rule of thumb, cannot be anonymised (Peters, Besley, et al., 2016). By and large, new models and challenges of peer review have failed to make a significant impact in the cloistered world of mainstream academic publishing thus far. However, some of the possibilities discussed here may dramatically alter the research landscape of the future. Contemporary academic publishing contains a mix of diverse approaches to peer review. While the agglomeration of academic publishers might have been thought to bring about the unification of peer review, the ecology of new journals and publishers gives rise to new understandings.

While peer review can be educational for authors and reviewers, this is not always the case. Training new scholars in handling and interpreting peer-review feedback and engaging in peer review, are important in this context. In regard to inequities in the peer-review process, academic outlets should gather information at each phase of peer review to assist in determining obstacles to the publication that some groups of academics might confront. Author identity facts such as gender could be included in information on manuscripts submitted to a journal, sent for review, sent for revision, etc.

Finally,

“It is valuable for scholars to be responsive to the dynamic world of academic publishing, including continually thinking through their interactions and changing relationships with each other and with publishing companies, as they shape the way knowledge is produced and disseminated collaboratively and academic lives are experienced.“

— Is peer review in academic publishing still working? By Liz Jackson, et al.

Here, we have aimed to provide a foundation to think more about these important issues.

In case you want to learn more about dedicated research solutions, head over to the SciSpace product page. You will be able to learn more about how we can simplify your research workflows, allowing you to focus on advancing science.

Trusted by researchers, universities, and publishers worldwide, SciSpace comes with everything a researcher needs. An exhaustive repository with more than 200 million research papers from across disciplines with SEO-optimized summaries, public profiles to showcase your experience and expertise, a specifically-built collaborative text editor, 20,000+ journal templates that can be inserted in a click, and so much more.

Co-Authorship

I would like to express my gratitude to Dr. Liz Jackson, Professor, Department of International Education, who invested her time in co-authoring this article and penned down her valuable insights into the crucial role of peer-review in scholarly publishing.

She works in the field of philosophy of education, focusing particularly on the diversity of human experience and has published more than 150 articles, chapters, books, and special journal issues. Professor Jackson also serves as President of the Philosophy of Education Society of Australasia and Deputy Editor for the leading journal in the field, Educational Philosophy and Theory.

Suggested Reading:

1. Publishing A Paper? Here are 7 Types Of Peer Review Processes You Should Know
2. Thoughts on the future of academic publishing
3. Enabling the libraries of the future: Tools and systems for scholarly communications
4. The Role and Impact of Preprints on Open Access Publishing