How Intensive Was the Peer Review for the Commentary by Ginsberg and Rice in Environmental Health Perspectives?

30 Mar 2006 in Regulatory Science, Peer Review

Peer review is often viewed as the "gold standard" for scientific quality. But like everything else, the quality of peer review varies.

The commentary by Ginsberg and Rice criticizing the review of National Academy of Sciences perchlorate review panel and the main study on which it relied was published by Environmental Health Perspectives, the journal of the US National Environmental Health Sciences Institute.

EHP was a logical publication outlet for such a commentary: the main study that NAS relied on (and which Ginsberg and Rice are critical) also was published by EHP in 2002. While the NAS is certainly not infallible, it’s nevertheless unusual for scientists to criticize its work products. For that reason, it’s reasonable to characterize the Ginsberg and Rice commentary as controversial.

Did EHP’s editors subject the Ginsberg and Rice commentary to an intense peer review befitting its controversial nature? Alternatively, did EHP’s editors apply the same intensity of peer review as it does to other commentaries it publishes? There is no way for an outsider to know for sure.

Peer review can take a long time for many reasons. For example, some peer reviewers are dilatory and simply do not fulfill their responsibilities promptly. Alternatively, authors may be dilatory in making changes in their manuscripts sufficient to respond to peer reviewers’ concerns. Neither phenomenon says anything about the intensity of journal peer review. Still, the more controversial the commentary, the more carefully one would expect journal editors to be in conducting peer review and the longer, on average, it would take to complete.

On the other hand, there are only a few reasons why journal peer review will be unusually short. It is possible, for example, that a submitted manuscript is so high in observable quality that peer review is not difficult or time-consuming. It is also possible that the peer reviewers selected by the editors were unusually proficient at task. There are also more worrisome explanations. For example, manuscripts submitted by distinguished or well-connected authors may benefit from lax peer review to the extent that editors and peer reviewers behave deferentially. Alternatively, a manuscript may encounter a short review because the views expressed resonate with those of the editors. There are no public data that provide insight into these competing explanations.

Regardless of its underlying explanation, we hypothesize that the shorter the period between a manuscript’s submission and acceptance, the less intense the peer review will have been.

We compared peer review times for all 22 peer reviewed commentaries published by EHP in 2005. The choice of a single calendar year was arbitrary. We excluded research and review articles on the ground that they were sufficiently dissimilar in character that their inclusion could be misleading. In one commentary, the dates of submission and acceptance were not reported; we dropped that commentary from our analysis. Our data are provided below.

For the remaining 21 commentaries, the following summary statistics were obtained:

Nevertheless, these data are consistent with the hypothesis that the Ginsberg & Rice commentary encountered less stringent peer review than EHP commentaries in general.

[add a comment]

Add a Comment