Why the BEST papers failed to pass peer review

Whoa, this is heavy.  Ross McKitrick, who was a peer review referee for the BEST papers with the Journal of Geophysical Research got fed up with Muller’s media blitzing and  tells his story:

excerpts:

In October 2011, despite the papers not being accepted, Richard Muller launched a major international publicity blitz announcing the results of the “BEST” project. I wrote to him and his coauthor Judy Curry objecting to the promotional initiative since the critical comments of people like me were locked up under confidentiality rules, and the papers had not been accepted for publication. Richard stated that he felt there was no alternative since the studies would be picked up by the press anyway. Later, when the journal turned the paper down and asked for major revisions, I sought permission from Richard to release my review. He requested that I post it without indicating I was a reviewer for JGR. Since that was not feasible I simply kept it confidential. 

On July 29 2012 Richard Muller launched another publicity blitz (e.g. here and here) claiming, among other things, that “In our papers we demonstrate that none of these potentially troublesome effects [including those related to urbanization and land surface changes] unduly biased our conclusions.” Their failure to provide a proper demonstration of this point had led me to recommend against publishing their paper. This places me in an awkward position since I made an undertaking to JGR to respect the confidentiality of the peer review process, but I have reason to believe Muller et al.’s analysis does not support the conclusions he is now asserting in the press.

…

I take the journal peer review process seriously and I dislike being placed in the position of having to break a commitment I made to JGR, but the “BEST” team’s decision to launch another publicity blitz effectively nullifies any right they might have had to confidentiality in this matter. So I am herewith releasing my referee reports.

Read it all here

Some backstory via Andrew Revkin from Elizabeth Muller. Revkin asked:

1) What’s the status of the four papers that were submitted last fall (accepted, in review…etc?)

2) There can be perils when publicity precedes peer review. Are you all confident that the time was right to post the papers, including the new one, ahead of review? Presumably this has to do with Tuesday deadline for IPCC eligibility?

Here’s her reply:

All of the articles have been submitted to journals, and we have received substantial journal peer reviews. None of the reviews have indicated any mistakes in the papers; they have instead been primarily suggestions for additions, further citations of the literature. One review had no complaints about the content of the paper, but suggested delaying the publication until the long background paper, describing our methods in detail, was actually published.

In addition to this journal peer review, we have had extensive comments from other scientists based on the more traditional method of peer review: circulation of preprints to other scientists. It is worthwhile remembering that the tradition in science, going back pre World War II, has been to circulate “preprints” of articles that had not yet been accepted by a journal for publication. This was truly “peer” review, and it was very helpful in uncovering errors and assumptions. We have engaged extensively in such peer review. Of course, rather than sending the preprints to all the major science libraries (as was done in the past), we now post them online. Others make use of arXiv. This has proven so effective that in some fields (e.g. string theory) the journalistic review process is avoided altogether, and papers are not submitted to journals. We are not going to that extreme, but rather are taking advantage of the traditional method.

We note that others in the climate community have used this traditional approach with great effectiveness. Jim Hansen, for example, frequently puts his papers online even before they are submitted to journals. Jim has found this method to be very useful and effective, as have we. As Jim is one of the most prominent members of the climate community, and has been doing this for so long, we are surprised that some journalists and scientists think we are departing from the current tradition.

The journal publication process takes time. This fact is especially true when new methods of analysis are introduced. We will be posting revised versions of 3 of the 4 papers previously posted later today (the 4th paper has not changed significantly). The core content of the papers is still the same, though the organization and detail has changed a fair amount.

The new paper, which we informally call the “Results” paper, has also undergone journal peer review (and none of the review required changing our results). We are posting it online today as a preprint, because we also want to invite comments and suggestions from the larger scientific community.

I believe the findings in our papers are too important to wait for the year or longer that it could take to complete the journal review process. We believe in traditional peer review; we welcome feedback the public and any scientists who are interested in taking the time to make thoughtful comments. Indeed, with the first 4 papers submitted, many of the best comments came from the broader scientific community. Our papers have received scrutiny by dozens of top scientists, not just the two or three that typically are called upon by journalists.