Nature Adopts an Experiment on Open Peer Review
14 Sep 2006 in Regulatory Science, Information Quality, Peer Review
Nature, one of the world's most distinguished science journals, recently supplemented its existing anonymous peer review with an open peer review process. A short story about it appeared in a UPI report, a longer one in Thursday's Wall Street Journal (subscription required). Scholarly peer review traditionally has been anonymous and confidential. It's purpose has been to ascertain whether scientific research reports meet the minimum quality standard for the journal in question, and secondarily, to allocate scarce pages to the best science that is compatible with the journal's mission.
The effectiveness of this model has come under frequent criticism for failing to detect fraud or error -- both of which are different and much more demanding objectives. In public policy, the Office of Management and Budget's 2002 Information Quality Guidelines confer a rebuttable presumption of objectivity on science that has survived scholarly peer review. This is an odd space for scholarly journals to occupy because objectivity, as that term is defined in the OMB guidelines, rarely is the defined purpose of scholarly publication.
Nature's open peer review system uses blogging technology, which permits a vibrant exchange of views. The Journal's Nicholas Zamiska writes:
Lay readers can see the submitted articles as well, but the site says postings are only for scientists in the discipline, who must list their names and institutional email addresses. Nature says its editors screen out those they find irrelevant, intemperate or otherwise inappropriate.
Meanwhile, the papers also make their way through the journal's traditional peer-review gauntlet. Nature says it's taking both sets of comments into account when deciding whether to publish.
Unlike conventional peer review, open peer review is very public (comments are posted so long as they meet the editors' standards for taste ("You can be as critical or controversial as you like, but please don't get personal or offensive.") Commenters also are supposed to identify themselves ("We strongly encourage you to use your real, full name."). Failure to reveal one's identity may cause the editors to decline to publish a comment. In short, Nature's open peer review system permits virtually unrestricted scientific criticism of research papers in the hopper but it mostly prevents critics from hiding in the weeds.
The open system supplements, but does not supplant, traditional peer review:
Entries posted here are manuscripts that have been submitted for publication in the journal Nature, and which have been sent by the journal editors to peer reviewers for confidential assessment. While this process is continuing, postings here are open for comments by scientists in the discipline. If you make a comment, your identity will be published with it.
After receiving peer reviewers' reports on a manuscript, editors will close the commenting facility, and may take into account any comments received when making their decision about publication in Nature.
Manuscripts posted on this site are not "in press" with Nature and are not endorsed by the journal.
This caveat is important, and it parallels the Peer Review Policy established in 2004 by OMB. Under that policy, information that is distributed for peer review and given an appropriate disclaimer is exempt from the definition of "dissemination," the trigger for applicability of the Information Quality Guidelines.
Nature's experiment with open peer review began in June 2006 and was projected to last three months. A useful FAQ sheet about the new process is here.
