The BMJ's experiment with online "peer review"
November 9, 1998: “We all know,” states a British Medical Journal (BMJ) editorial (1) “that it [peer review] is expensive, slow, prone to bias, open to abuse, possibly anti-innovatory, and unable to detect fraud. We also know that the published papers that emerge from the process are often grossly deficient. Perhaps,” the editorial continues, “because scientific publishing without peer review seems unimaginable, nobody has ever done what might be called a placebo controlled trial of peer review. It has not been tested against, for instance, editors publishing what they want with revision, and letting the correspondence columns sort out the good from the bad and point out the strengths and weaknesses of studies.”
In reality, publishing without peer review is entirely imaginable. "Editors publishing what they want" and allowing qualified readers to comment in the correspondence columns is the norm in magazine publishing beyond the confines of academia. It was in accordance with this editorial model that naturalSCIENCE was established as a medium for articles by scientists who wish to communicate with a wide audience (2). This is not, however, the model currently adopted by the BMJ in an exercise it describes as "dipping our toes in the water of online peer review" (3). Specifically, BMJ states, that it is "posting an article before we've peer reviewed it, let alone decided whether to publish it in the paper journal."
"We are," the Journal goes on to state, "soliciting your comments about the article (4), which we will post as we receive them" (5). In addition, says the Journal, "The article is also being peer reviewed in our usual way, and our intention is to post the reviewers' comments, and those of the journal's editorial department, as they become available."
By undertaking this experiment, the BMJ seems to have abandoned its original and sensible idea, adopting instead a process of questionable propriety. Rather than making an editorial judgment about the work and publishing it (on the ground of its presumed merit), with or without revision, the BMJ has published an article without so much as running it through a spell-checker, let alone performing the editorial functions proper to a publisher; namely, to search diligently for and to correct errors of spelling, grammar, construction, logic and fact that detract from what is of value in an author's work. If publishers fail to fulfil their editorial role, who needs them? Authors might as well deal directly with a printer or the Internet equivalent.
The truth, however, is that almost every author needs a publisher, or the editorial equivalent. Samuel Johnson remarked that William Shakespeare never wrote six lines together without a mistake. Lesser authors may rarely manage six words together without a mistake. A publisher's role is to add value to an author's original work by applying editorial resources to eliminate as many errors as possible. The end result should be readable, concise and logically coherent. This the BMJ has failed to ensure in their current experiment. As a consequence, the authors of the article that is the subject of the BMJ's experiment have been ill-served.
The BMJ's process of so-called "online peer review" is as questionable as its decision to place an unedited article on the Web while disclaiming all responsibility for its deficiencies. No effort appears to have been made in the current round of open comment to obtain contributions from individuals with a special knowledge of the subject. Furthermore, the comments are as unedited as the article to which they refer. The result is a collection of more or less interesting, brutal, or uncritically enthusiastic remarks, which, as of this date, provide nothing amounting to a detailed or comprehensive critique.
Although the BMJ's experiment may be useful in drawing attention to the proper role of a publisher, it is not an exercise to be emulated. This, however, is not to detract from the BMJ's earlier proposal for editors "publishing what they want ... and letting the correspondence columns sort out the good from the bad," provided, that is, that editors are selective about what they publish, edit what they select before publishing it, and solicit comment for publication from those able to speak with authority.
(1) Editorial. 1997.
Peer review: reform or revolution? Br. Med. J. 315 (September 27).
(2) Editorial. 1997. The aim of this magazine. naturalSCIENCE http://naturalscience.com/ns/articles/edit/plett_01.html.
(3) 1998. Help us peer review an article on line. Br. Med. J. http://www.bmj.com/misc/peer/index.shtml.
(4) Sekikawa, A., D. Aaron, R. Nishimura, B. Acosta and R. LaPorte. 1998. The metamorphosis of biomedical journals. Br. Med. J. http://www.bmj.com/cgi/content/full/317/7162/0/z
(5) 1998. On line review comments on "The metamorphosis of biomedical journals." Br. Med. J. http://www.bmj.com/cgi/content/full/317/7162/0/z#responses.
Your comment on this item is invited and should be addressed to: firstname.lastname@example.org. For further information on submitting a contribution to naturalSCIENCE, please see the Author Guide