Academic work is peer reviewed and published in scholarly articles. Right? Well, not quite.
Scientists, in particular, tend to view peer review as a necessary step in science. Peer review has been described as the ‘heart’ of science – ‘It is the method by which grants are allocated, papers published, academics promoted, and Nobel prizes won.’
Peer review helps us select the best manuscripts for publication and improves the quality of published papers. It is also a critical process in conferring reliability and credibility in scientific knowledge. For example, John Cook investigated the views of climate scientists on anthropogenic climate change and quantified consensus on the issue based on peer reviewed findings.
This is considered the key mechanism of quality control in scientific discourse. So perhaps analyses and statements, such as climate skeptic blogs, which haven’t been peer reviewed can be disregarded as unreliable and unscientific?
Scientists typically distinguish between peer-reviewed and non-peer-reviewed work, but these distinctions aren’t always easy or useful. In a new essay accepted into the Bulletin of the American Meteorological Society, I explore details of the quickly evolving field of rapid extreme event attribution.
The attribution of extreme weather and climate events to a particular cause is a quickly expanding field. After an extreme weather or climate event, it is natural to ask questions about its cause and links to climate change. A key aspect of addressing these questions, and fuelling this expansion, has been a focus on near real-time attribution.
International projects such as the World Weather Attribution (WWA) project can now attribute events in near real-time. Results are communicated first through the ‘grey’ literature in online reports and articles on websites, which are produced outside of traditional commercial and academic publication and distribution channels.
But the timely publication and widespread communication of scientific results aren’t traditional priorities in science or in peer review. So how do these rapid attribution approaches fit into a standard understanding of science as peer-reviewed?
It turns out the peer review isn’t necessarily as central to science as it might first seem. For a start, while peer review seems to underpin all scholarly activity, this is a relatively modern understanding. It wasn’t really until mid-way through the 20th century that peer review became linked to credibility and reliability, and then peer review only took off after the 1970s as a foundation of scientific practice.
In addition, there are important precedents for science occurring in the grey literature. Releasing the results of scientific analyses prior to review is not that unusual. It’s common for unpublished analyses to be presented at conferences, papers to be circulated amongst colleagues for feedback and drafts to be published on researcher’s websites.
Operational systems can also provide useful analogues for rapid attribution. Meteorology and climatology operational systems provide information on seasonal outlooks for Pacific Ocean conditions seasonal forecasts, and river flow forecasts. Operational systems rely on methodologies and analytical tools that have been peer-reviewed, but the specific results intended for stakeholders and policymakers haven’t. This is a similar approach to event attribution.
It’s clear that the grey literature can be a useful location for conducting science and for communicating scientific results broadly. In terms of rapid event attribution approaches, using the grey zone for the best effect for both the discipline and for society requires some careful additional thought.
Attribution has expanded quickly. While the first studies focused on the causes of a specific weather event (e.g. a heatwave or flood in a particular location), the field of attribution now investigates complex impacts on human and natural systems. Recent studies have targeted coral reef bleaching and human health outcomes.
Attribution is also occurring more and more rapidly. When the first attribution studies were done, the period between an extreme event occurring and the publication of results was vast. Nowadays, some events can be analysed as they are occurring.
Discussion in the scientific community about how science is being done haven’t quite kept up with theses technical advances and capabilities. Does science have to be peer reviewed to be ‘true science’? Is the application of peer-reviewed methodologies sufficient to instill confidence in analytical results, or must interpretations and conclusions also be peer-reviewed?
Without discussing these details about scientific practice, the grey zone might not be used to full effect, or might even pose risks to the credibility of the discipline. What if an analysis published in the grey literature is then rejected? What if multiple studies come up with different answers? How then can attribution help society be prepared for climate change?
In my essay I focus on event attribution and make recommendations both for optimising the grey zone and minimising the risks to science from using this space, just as climate and weather forecasts and operational systems have done.
While peer review seems to be the immutable foundation of science, it is a modern approach that is popular at the moment. It is really just one way to evaluate knowledge, but sometimes different publication models for different outcomes and different types of science are required. When used carefully, the grey zone provides a venue to communicate scholarly scientific information rapidly widely.