Revkin on the Gergis et al 'on hold' affair

I promised Andrew Revkin yesterday that I’d give his post on Gergis et all some attention, because he’s done a good job of summarizing it all, plus getting some other angles, such as that of Retractionwatch. I was especially pleased to note that he reports that the blogosphere is becoming increasingly important as a tool of peer review. Unlike The Team, David Karoly had the good sense to at least acknowledge McIntyre’s contributions. Below is an excerpt of Revkin’s article. – Anthony

Australian Warming, Hockey Sticks and Open Review

A much-cited study (paper here) concluded last month that the extent of warming in Australia in recent decades was so great compared to climate variations in the last millennium that it had to be mainly the result of warming from the human-driven buildup of greenhouse gases in the atmosphere. (Here’s a video interview from May with the lead author, Joëlle Gergis from the University of Melbourne.)

It’s the latest research in more than a decade of work producing a climate “hockey stick” — graphs of global or regional temperatures showing relatively little variation over a millennium or more and then a sharp uptick since the middle of the twentieth century (the blade at the end of the stick).

Now the paper, at the request of the authors, has been “put on hold” by the Journal of Climate after questions were raised publicly about one of the researchers’ methods, starting with a comment on Steve McIntyre’s Climate Audit blog. This field of study uses sophisticated statistical methods to derive meaning from scattered and variegated indirect indicators of past temperature — with tree rings being the most familiar example.

It is unclear whether the problem will affect the study’s conclusions. Depending on the result, readers of the initial burst of news could end up with a familiar sense of whiplash.

To see how quickly the research results made the rounds, check the headlines here. My unfavorite would be “IT’S OFFICIAL: Australia is warming and it is your fault,” in the Herald Sun. This is a classic case of what I’ve been calling “single-study syndrome,” the bias in the news process toward the “front-page thought” and tendency to forget that science is a herky-jerky process.

Over the weekend, I got in touch with David Karoly, one of the paper’s authors … who wrote:

As I said in my e-mail to Stephen, “This is a normal part of science. The testing of scientific studies through independent analysis of data and methods strengthens the conclusions. In this study, an issue has been identified and the results are being re-checked.”

Indeed, this is an increasingly normal part of science these days. While the blogosphere comes with lots of noise, it also is providing a second level of review — after the initial round of closed peer review during the publication process — that in the end is making tough, emerging fields of science better than they would otherwise be.

Read the full article here, well worth your time for the additional comments Revkin included from others.

============================================================

Steve McIntyre also has some additional thoughts in a new post yesterday: More on Screening in Gergis et al 2012. The first section reads:

First, let’s give Gergis, Karoly and coauthors some props for conceding that there was a problem with their article and trying to fix it. Think of the things that they didn’t do. They didn’t arrange for a realclimate hit piece, sneering at the critics and saying Nyah, nyah,

what about the hockey stick that Oerlemans derived from glacier retreat since 1600?… How about Osborn and Briffa’s results which were robust even when you removed any three of the records?

Karoly recognized that the invocation of other Hockey Sticks was irrelevant to the specific criticism of his paper and did not bother with the realclimate juvenilia that has done so much to erode the public reputation of climate scientists. Good for him.

Nor did he simply deny the obvious, as Mann, Gavin Schmidt and so many others have done with something as simple as Mann’s use of the contaminated portion of Tiljander sediments according to “objective criteria”. The upside-down Tiljander controversy lingers on, tarnishing the reputation of the community that seems unequal to the challenge of a point that a high school student can understand.

Nor did they assert the errors didn’t “matter” and challenge the critics to produce their own results (while simultaneously withholding data.) Karoly properly recognized that the re-calculation obligations rested with the proponents, not the critics.

I do not believe that they “independently” discovered their error or that they properly acknowledged Climate Audit in their public statements or even in Karoly’s email. But even though Karoly’s email was half-hearted, he was courteous enough to notify me of events. Good for him. I suspect that some people on the Team would have opposed even this.

McIntyre goes on to explain the “screening fallacy” (or the cherry pick if you will) in detail.