BEST: What I agree with and what I disagree with – plus a call for additional transparency to prevent "pal" review

There’s lots of hay being made by the usual romminesque flaming bloggers, some news outlets and the like, over my disagreement with the way data was handled in one of the Berkeley Earth Surface Temperature (BEST) papers, the only one I got to review before yesterday’s media blitz. Apparently I’m not allowed to point out errors, and BEST isn’t allowed to correct any before release, such as the six incorrectly spelled citations of the Fall et al 2011 paper I pointed out to BEST a week earlier, which they couldn’t be bothered to fix.

And then there’s the issue of doing a 60 year study on siting, when we only guaranteed 30. Even NOAA’s Menne et al paper knew not to make such a stupid mistake. Making up data where there isn’t any is what got Steig et al into trouble in Antarctica and they got called on it by Jeff Id, Steve McIntyre, and Ryan O’Donnell in a follow on peer-reviewed paper.

But I think it’s useful to note here (since I know some other bloggers will just say “denier” and be done with it) what I do in fact agree with and accept, and what I don’t. They wanted an instant answer, before I had a chance even to read the other three papers. Media outlets were asking for my opinion even before the release of these papers, and I stated clearly that I had only seen one and I couldn’t yet comment on the others. That didn’t matter, they lumped that opinion on one I had seen into an opinion on all four.

What I agree with:

1. The Earth is warmer than it was 100-150 years ago. But that was never in contention –  it is a straw man argument. The magnitude and causes are what skeptics question.
2. From the BEST press release “Global Warming is real”  …see point one. Notably, “man-made global warming” was not mentioned by BEST, and in their findings they point out explicitly they didn’t address this issue as they state in this screencap from the press release:
3. As David Whitehouse wrote: “The researchers find a strong correlation between North Atlantic temperature cycles lasting decades, and the global land surface temperature. They admit that the influence in recent decades of oceanic temperature cycles has been unappreciated and may explain most, if not all, of the global warming that has taken place, stating the possibility that the “human component of global warming may be somewhat overstated.”. Here’s a screencap from that paper:
4. The unique BEST methodology has promise. The scalpel method used to deal with station discontinuity was a good idea and I’ve said so before.
5. The findings of the BEST global surface analysis match the finding of other global temperature metrics. This isn’t surprising, as much of the same base raw data was used. There’s a myth that NASA GISS, HadCRUT, NOAA’s, and now Berkeley’s source data are independent of one another. That’s not completely true. They share a lot of common data from GHCN, administered by NOAA’s National Climatic Data. So it isn’t surprising at all they would match.

What I disagree with:

1. The way they dealt with my surfacestation data in analysis was flat-out wrong, and I told them so days ahead of this release. They offered no correction, nor even an acknowledgement of the issue. The issue has to do with the 60 year period they used. Both peer-reviewed papers on the subject, Menne et al 2010, and Fall et al 2011 used 30 year periods. This is a key point because nobody knows (not me, not NOAA, not BEST) what the siting quality of weather stations was 30-60 years ago. Basically they did an analysis on a time period for which metadata doesn’t exist. I’ve asked simply for them to do it on 30 years as the two peer reviewed papers did, an apples-to-apples comparison. If they do that and the result is the same, I’m satisfied. OTOH, they may find something new when done correctly, we all deserve that opportunity.

Willis Eschenbach points out this quote from the paper:

We evaluate the effect of very-rural station siting on the global average by applying the Berkeley Earth Surface Temperature averaging procedure to the very-rural stations. By comparing the resulting average to that obtained by using all the stations we can quantify the impact of selecting sites not subject to urbanization on the estimated average land temperature.

He adds: That seems crazy to me. Why compare the worst stations to all stations? Why not compare them to the best stations?

2. The UHI study seems a bit strange in its approach. They write in their press release that:

They didn’t adequately deal with that 1% in my opinion, by doing a proper area weighting. And what percentage of weather stations were in that 1%? While they do have some evidence of the use of a “kriging” technique, I’m not certain is has been done properly. The fact that 33% of the sites show a cooling is certainly cause for a much harder look at this. That’s not something you can easily dismiss, though they attempt to. This will hopefully get sorted out in peer review.

3. The release method they chose, of having a media blitzkrieg of press release and writers at major MSM outlets lined up beforehand is beyond the pale. While I agree with Dr. Muller’s contention that circulating papers among colleagues for wider peer review is an excellent idea, what they did with the planned and coordinated (and make no mistake it was coordinated for October 20th, Liz Muller told me this herself) is not only self-serving grandiosity, but quite risky if peer review comes up with a different answer.

The rush to judgment they fomented before science had a chance to speak is worse than anything I’ve ever seen, and from my early dealings with them, I can say that I had no idea they would do this, otherwise I would not have embraced them so openly. A lie of omission is still a lie, and I feel that I was not given the true intentions of the BEST group when I met with them.

So there you have it, I accept their papers, and many of their findings, but disagree with some methods and results as is my right. It will be interesting to see if these survive peer review significantly unchanged.

One thing we can count on that WON’T normally be transparent is the peer review process, and if that process includes members of the “team” who are well versed enough to but already embracing the results such as Phil Jones has done, then the peer review will turn into “pal review”.

The solution is to make the names of the reviewers known. Since Dr. Muller and BEST wish to upset the apple cart of scientific procedure, putting public review before peer review, and because they make this self-assured and most extraordinary claim in their press release:

That’s some claim. Four papers that have not been peer-reviewed yet, and they KNOW they’ll pass peer review and will be in the next IPCC report? Is it just me or does that sound rigged? Or, is it just the product of an overactive ego on the part of the BEST group?

I say, if BEST and Dr. Muller truly believes in a transparent approach, as they state on the front page of their website…

…let’s make the peer review process transparent so that there is no possibility of “pal review” to ramrod this through without proper science being done.

Since Dr. Muller claims this is “one of the most important questions ever”, let’s deal with it in an open a manner as possible. Ensuring that these four papers get a thorough and non-partisan peer review is the best way to get the question answered.

Had they not made the claim I highlighted above of it passing peer review and being in the next IPCC report before any of that even is decided, I would never think to ask for this. That overconfident claim is a real cause for concern, especially when the media blitzkrieg they launched makes it difficult for any potential review scientists to not notice and read these studies and news stories ahead of time, thus becoming biased by media coverage.

We can’t just move the “jury pool” of scientists to the next county to ensure a fair trial now that is been blathered worldwide can we?

Vote on it: