I’m saying goodbye to peer reviewing. Of course, you can’t just tap out of the system in which you work, but as much as possible, I’m saying goodbye to welcoming exploitation into my inbox.
A young academic’s CV or grant application invariably touts their service to the scientific community as a reviewer for a suite of prestigious journals. It’s a convenient way to re-assure people that you are part of a community – you put your hand up to serve and you are part of the space where decisions get made. Ideally, it shows that in your little part of the scientific world, you are a somebody.
In the past I’ve written a few blog posts that have focused on peer review. I’ve discussed my own frustrations as an author, based on firsthand experiences. I’ve also discussed peer review more broadly as a means to legitimise scientific knowledge. But in spite of several blog posts and twitter conversation, I still can’t figure out what the devil we expect to get out of peer review and what role we expect editors, reviewers and authors to play in the process.
Sometime ago I reviewed a paper for a “decent”* journal that I was surprised went out to review. I try to be a generous reviewer, but the paper was riddled with errors ranging from typographical to methodological. It was highly descriptive, the analysis and discussion were scant and the conclusions unoriginal. It wasn’t ready to be submitted.
The other two reviewers’ opinions aligned with my own report. The editor decided major revisions were in order and eventually a revised manuscript was returned to me. Only superficial changes had been made and many reviewer comments went largely ignored. I recommended a rejection, and was surprised sometime later to be thanked and notified that the paper had been accepted for publication with only cursory improvements.
More recently, I reviewed a paper for an “excellent”* journal. I was asked to review for a particular purpose and given some guiding questions to help in helping the editor. In my opinion, the paper was good – well written, nice figures, punchy and of interest. I had some qualms about a few minor analytical points and felt that there were a few unjustified bombastic statements. The other two reviewers had differing areas of expertise but had formed similar impressions of the quality of the paper.
It was later returned to me with a request for a quick turnaround review. Such requests naturally align with a week jam-packed with meetings, workshops, dentist appointment and trips to drop off/pick a relative up from the airport. I made a decision and obliged with the request. I had some very minor comments but thought it was good to go. Reviewer 3 was more critical but agreed it was nearly ready.
A few weeks later, I received an email that the paper had been rejected due to an editorial decision. I was stumped – the paper had been assessed prior to peer review, which was positive.
A strong editor with a fair and consultative approach can cut through differing opinions (/egos) and direct an author to problem areas. A strong editor can make a paper. But editors who disregard the advice of reviewers that they have personally assigned, is this common?
In one instance, the editor was a scientist within the discipline serving a term on a journal’s editorial board in an unpaid role or possibly minimally compensated for time. In the other, the editor was in a paid position, dedicate to that role, with a strong scientific background but was not a scientist and certainly not an active expert in the field.
What’s the point of reviewers? At times, it seems like we are gratefully exploited rubber stamps. In these two instances, I voluntarily dedicated time to lucrative businesses. In the first case, the results are now considered to have met some minimum standard of quality, having undergone peer review. In the second case, a decision was made about the quality and novelty of a study that was vastly different from opinions sourced from within the scientific community.
At the moment, my capacity to review papers is diminished due to an increasing amount of time spend in service as an editor of a journal*. In this sense my frustrations are convenient, as I already have a solid basis for rejecting review assignments.
Regardless, I would have insurmountable qualms about accepting further assignments from either of these journals, or others within the same publishing houses. While not all experiences of reviewing or being reviewed reflect these two particular instances, messy review situations are commonplace.
At my most cynical, I could view a list of review assignments on my CV as an acknowledgement that I am willing to squander my taxpayer salary in service to a company with a record of thinking my opinion should be disregard.
Less cynically, reviewing is an ambiguous contract between profit making, editors, authors and the community. Reviewers are not provided with clear expectations from the editor, or with information about how their recommendations will be used. There is no way for the authors or readers to cut through this process and understand how publishing decisions are made.
Until such processes are brought into the open*, I’m saying goodbye to the weekly requests for enslavement to the review process.
* Insofar as impact factor can measure the quality of a journal
* Which does not have page or subscription charges and is published with no budget through the community’s sense of heart feels, rainbows, puppies and skittles
* Or until I decide otherwise out of guilt for expecting others to review my work