Why the BEST papers failed to pass peer review

Red Fail

Red Fail (Photo credit: griffithchris)

Whoa, this is heavy.  Ross McKitrick, who was a peer review referee for the BEST papers with the Journal of Geophysical Research got fed up with Muller’s media blitzing and  tells his story:

excerpts:

In October 2011, despite the papers not being accepted, Richard Muller launched a major international publicity blitz announcing the results of the “BEST” project. I wrote to him and his coauthor Judy Curry objecting to the promotional initiative since the critical comments of people like me were locked up under confidentiality rules, and the papers had not been accepted for publication. Richard stated that he felt there was no alternative since the studies would be picked up by the press anyway. Later, when the journal turned the paper down and asked for major revisions, I sought permission from Richard to release my review. He requested that I post it without indicating I was a reviewer for JGR. Since that was not feasible I simply kept it confidential. 

On July 29 2012 Richard Muller launched another publicity blitz (e.g. here and here) claiming, among other things, that “In our papers we demonstrate that none of these potentially troublesome effects [including those related to urbanization and land surface changes] unduly biased our conclusions.” Their failure to provide a proper demonstration of this point had led me to recommend against publishing their paper. This places me in an awkward position since I made an undertaking to JGR to respect the confidentiality of the peer review process, but I have reason to believe Muller et al.’s analysis does not support the conclusions he is now asserting in the press.

I take the journal peer review process seriously and I dislike being placed in the position of having to break a commitment I made to JGR, but the “BEST” team’s decision to launch another publicity blitz effectively nullifies any right they might have had to confidentiality in this matter. So I am herewith releasing my referee reports.

Read it all here

Some backstory via Andrew Revkin from Elizabeth Muller. Revkin asked:

1) What’s the status of the four papers that were submitted last fall (accepted, in review…etc?)

2) There can be perils when publicity precedes peer review. Are you all confident that the time was right to post the papers, including the new one, ahead of review? Presumably this has to do with Tuesday deadline for IPCC eligibility?

Here’s her reply:

All of the articles have been submitted to journals, and we have received substantial journal peer reviews. None of the reviews have indicated any mistakes in the papers; they have instead been primarily suggestions for additions, further citations of the literature. One review had no complaints about the content of the paper, but suggested delaying the publication until the long background paper, describing our methods in detail, was actually published.

In addition to this journal peer review, we have had extensive comments from other scientists based on the more traditional method of peer review: circulation of preprints to other scientists. It is worthwhile remembering that the tradition in science, going back pre World War II, has been to circulate “preprints” of articles that had not yet been accepted by a journal for publication. This was truly “peer” review, and it was very helpful in uncovering errors and assumptions. We have engaged extensively in such peer review. Of course, rather than sending the preprints to all the major science libraries (as was done in the past), we now post them online. Others make use of arXiv. This has proven so effective that in some fields (e.g. string theory) the journalistic review process is avoided altogether, and papers are not submitted to journals. We are not going to that extreme, but rather are taking advantage of the traditional method.

We note that others in the climate community have used this traditional approach with great effectiveness. Jim Hansen, for example, frequently puts his papers online even before they are submitted to journals. Jim has found this method to be very useful and effective, as have we. As Jim is one of the most prominent members of the climate community, and has been doing this for so long, we are surprised that some journalists and scientists think we are departing from the current tradition.

The journal publication process takes time. This fact is especially true when new methods of analysis are introduced. We will be posting revised versions of 3 of the 4 papers previously posted later today (the 4th paper has not changed significantly). The core content of the papers is still the same, though the organization and detail has changed a fair amount.

The new paper, which we informally call the “Results” paper, has also undergone journal peer review (and none of the review required changing our results). We are posting it online today as a preprint, because we also want to invite comments and suggestions from the larger scientific community.

I believe the findings in our papers are too important to wait for the year or longer that it could take to complete the journal review process. We believe in traditional peer review; we welcome feedback the public and any scientists who are interested in taking the time to make thoughtful comments. Indeed, with the first 4 papers submitted, many of the best comments came from the broader scientific community. Our papers have received scrutiny by dozens of top scientists, not just the two or three that typically are called upon by journalists.

Advertisements

  Subscribe  
newest oldest most voted
Notify of
DavidA

He’s made his headlines but isn’t making too many friends.
Mike Mann posting on Facebook:

My view is that Muller’s efforts to promote himself by belittling the collective efforts of the entire atmospheric/climate research community over several decades, though, really does the scientific community a disservice. Its great that he’s reaffirmed what we already knew. But for him to pretend that we couldn’t trust this entire scientific field until Richard Muller put his personal stamp of approval on their conclusions is, in my view, a very dangerously misguided philosophical take on how science works. It seems, in the end–quite sadly–that this is all really about Richard Muller’s self-aggrandizement 🙁

The self-aggrandizement comment seems to be spot on, though of course much disagreement exists elsewhere. Really, has the type of simplistic analysis performed by Muller not been done to death already?

The believer side is treating this as a fait accompli. If you ask me this is highly irregular and almost narcissistic in its scope. Astounding. It’s like a Gleick, only it’s attempting to be legitimate SCIENCE.

u.k.(us)

It is getting late, i’ll revisit in the morning.
[rep] warned it might get bloody.

Its great that he’s reaffirmed what we already knew. But for him to pretend that we couldn’t trust this entire scientific field until Richard Muller put his personal stamp of approval on their conclusions is, in my view, a very dangerously misguided philosophical take on how science works.
Let the internecine sniping begin…

“Our results show that the average temperature of the earth’s land has risen by 2½ degrees Fahrenheit over the past 250 years” Professor Muller wrote in an opinion piece for The New York Times.
Exactly, the warming has been going on since pre-industrialisation (which really kicked off in terms of anthropogenic, CO production post-WW2). In Prof. Muller’s own words, the current climate warming can not be CO2 driven.

Justthinkin

Self-aggrandizement? Good grief. How many hundreds of examples would it take to point out Mr.Mann’s self-aggrandizement?

“It is worthwhile remembering that the tradition in science, going back pre World War II, has been to circulate “preprints” of articles that had not yet been accepted by a journal for publication.”
That’s not really true. This is a time before laser printers or even photocopiers. Manuscripts were submitted as typescripts and carbon copies, and stayed that way until they had been typeset by a journal (after acceptance). Then you could get preprints to circulate.

Linked this at PJM too: http://pjmedia.com/tatler/2012/07/29/even-more-excitement-in-the-climate-change-world/
My original PJM post is linked by Instapundit now and on the PJM front page.

Konrad.

Well, well, Richard Muller seems to be taking flack from all sides today.
Note to Dr. Muller – tacking flack does not always mean you are over the target. Mooning an “88” crew can also achieve a similar response.

Jarryd Beck

Meanwhile, the Sydney Morning Herald is claiming that they had all these wonderful results and that Muller even used to be a sceptic and now isn’t. I bet we don’t see anything about Anthony’s press release in any newspaper in Australia.
http://www.smh.com.au/environment/climate-change/climate-results-convert-sceptic-let-the-evidence-change-our-minds-20120730-23769.html

A. Scott

BEST and Muller being savaged by many – including warmists – here are just a few:
David Appell:

Attributing climate is more like figuring out the structure of DNA than it is like figuring out the laws of quantum mechanics — simple curve-fitting (“exponentials, polynomials”) doesn’t cut it. In fact, it makes you look kind of foolish.

And former BEST author Judith Curry (by way of Revkin):

Their latest paper on the 250-year record concludes that the best explanation for the observed warming is greenhouse gas emissions. Their analysis is way oversimplistic and not at all convincing in my opinion. There is broad agreement that greenhouse gas emissions have contributed to the warming in the latter half of the 20th century; the big question is how much of this warming can we attribute to greenhouse gas emissions. I don’t think this question can be answered by the simple curve fitting used in this paper, and I don’t see that their paper adds anything to our understanding of the causes of the recent warming.

Warmist William Connolley – Muller still Rubbish:

‘And what they say…appears absurdly naive’

Michael Mann (h/t Tom Nelson):

My view is that Muller’s efforts to promote himself by belittling the collective efforts of the entire atmospheric/climate research community over several decades, though, really does the scientific community a disservice …. It seems, in the end–quite sadly–that this is all really about Richard Muller’s self-aggrandizement 🙁

‎”My overall impression is that the authors have not actually read all the papers they cite, and have not come to terms with the technical issues involved in the current debate.”

Oh, it burns!
From McKitrick’s first referee statements.

AntonyIndia

Look here at the bottom of JGR – Journal of Geophysical Atmospheres Research’s list of papers in press today (my bold):
Cornes, R. C., and P. D. Jones
An Examination of Storm Activity in the Northeast Atlantic Region over the 1851-2003 period using the EMULATE gridded MSLP data series. J. Geophys. Res., doi:10.1029/2010JD014865, in press. [Abstract] [PDF] (accepted 7 December 2010)
Withdrawn
Preissler, J., F. Wagner, S. Pereira, and J. L. Guerrero-Rascado
Correction to “Multi-instrumental observation of an exceptionally strong Saharan dust outbreak over Portugal” J. Geophys. Res., doi:10.1029/2012JD017645, in press. [Abstract] [PDF] (accepted 16 February 2012; withdrawn 29 February 2012)
The famous peer review suffers from more maladies.

Hari Seldon

Mullers activity is simply a grab for market share by dissing all the compettitors. No wonder ‘the crew’ are so antagonistic.
Muller is after all making money from his climate consultancy business.

MangoChutney

Should be interesting to see how the msm and IPCC deal with this.
Reporting BEST, but not reporting Watts, would clearly show bias. Claiming not to have seen the press release is untenable. Ignoring BEST is untenable given the previous media coverage. Ignoring Watts is untenable.
If the IPCC use BEST and ignore Watts, we have clear bias. Ignoring both is untenable given previous history and even the use of “grey literature”
Clever move by Watts, although we all need to stay calm until the paper has passed peer review.

Geez Muller is making a fool of himself. Even Michael Mann attacked him.

Marion

Many thanks to Ross for publising his reviews, a difficult decision but very necessary to counter the major propaganda campaign that Muller has indulged in.
I have no doubt that Revkin’s observation on the Muller team papers being posted amidst a huge publicity campaign was accurate ie “Presumably this has to do with Tuesday deadline for IPCC eligibility?”
Nor surely can the Mullers object in any way to the publication of his reviews of their work. After all, Elizabeth Muller is on record as saying –
“We also believe in full transparency, which is why we are posting our data and programs – even before our results have been formally published in a journal. We would love for other people to get into the data and analysis – the sooner the better. Again, we think the results are important, and they need to be looked at sooner rather than later.”
And Ross’s reviews are certainly very helpful in this respect.

And Dr. Judith Curry says, in part:

Their latest paper on the 250 year record concludes that the best explanation for the observed warming is greenhouse gas emissions. In my opinion, their analysis is way over simplistic and not at all convincing . There is broad agreement that greenhouse gas emissions have contributed to the warming in the latter half of the 20th century; the big question is how much of this warming can be attributed to greenhouse gas emissions. I dont think this question can be answered by the simple curve fitting used in this paper, and I don’t see that their paper adds anything to our understanding of the causes of the recent warming.

http://judithcurry.com/2012/07/29/a-new-release-from-berkeley-earth-surface-temperature/

Jarryd, Muller was never a skeptic,
http://www.populartechnology.net/2012/06/truth-about-richard-muller.html
“I was never a skeptic” – Richard Muller, 2011
“If Al Gore reaches more people and convinces the world that global warming is real, even if he does it through exaggeration and distortion – which he does, but he’s very effective at it – then let him fly any plane he wants.” – Richard Muller, 2008
“There is a consensus that global warming is real. …it’s going to get much, much worse.” – Richard Muller, 2008
“Let me be clear. My own reading of the literature and study of paleoclimate suggests strongly that carbon dioxide from burning of fossil fuels will prove to be the greatest pollutant of human history. It is likely to have severe and detrimental effects on global climate.” – Richard Muller, 2003

Wow, well done Ross McKitrick, that has to be the right decision at this juncture.
Michael Mann’s accusations of self-aggrandizement – that’s a bonus indeed. Muller has never taken back his criticisms of the hockey stick, as far as I know.
This is now as messy as climate science deserves to be shown to be – the find of internecine mess that could lead to final implosion and a rebuild of true science on the ruins. We can only hope.

Elizabeth Muller: “None of the reviews have indicated any mistakes in the papers”.
Anthony’s reviews: [Actually, Ross’s reviews … -w.]
“I will recommend that the paper be rejected in its current form.”
“In the absence of so much elementary material it is difficult even to review this paper.”
“Meanwhile I cannot recommend this draft for publication.”
“…their method is, in principle, unable to support the conclusions they draw”
“…my original recommendation against publication is also unchanged.”
So either Elizabeth Muller’s statement is deliberately being economical with the truth, or some reviews were not made available to her by the journal. Do we know which?

johanna

Re Elizabeth Muller’s comments, e.g.
“All of the articles have been submitted to journals, and we have received substantial journal peer reviews. None of the reviews have indicated any mistakes in the papers; they have instead been primarily suggestions for additions, further citations of the literature.”
It seems that not only did they not read (or perhaps comprehend) all the papers they cited, as per Ross M.’s review, it seems they did not read (or perhaps comprehend) his review either.

Rick Bradford

Mann thinks Muller is into “self-aggrandizement”?
Well, Mann should know, he’s the current Olympic champion and record holder in that event.

Marion

Re: DavidA says:
July 29, 2012 at 11:03 pm
“He’s made his headlines but isn’t making too many friends.
Mike Mann posting on Facebook:
My view is that Muller’s efforts to promote himself by belittling the collective efforts of the entire atmospheric/climate research community over several decades, though, really does the scientific community a disservice. Its great that he’s reaffirmed what we already knew. But for him to pretend that we couldn’t trust this entire scientific field until Richard Muller put his personal stamp of approval on their conclusions is, in my view, a very dangerously misguided philosophical take on how science works. It seems, in the end–quite sadly–that this is all really about Richard Muller’s self-aggrandizement 🙁
The self-aggrandizement comment seems to be spot on, though of course much disagreement exists elsewhere. Really, has the type of simplistic analysis performed by Muller not been done to death already? ”
—————————————————————————————————–
Interesting how this gets touted by Leo Hickman in the UK’s newspaper “The Guardian”
“Prof Michael Mann, the Penn State palaeoclimatologist who has faced hostility from climate sceptics for his famous “hockey stick” graph showing a rapid rise in temperatures during the 20th century, said he welcomed the Best results as they “demonstrated once again what scientists have known with some degree of certainty for nearly two decades”. He added: “I applaud Muller and his colleagues for acting as any good scientists would, following where their analyses led them, without regard for the possible political repercussions. They are certain to be attacked by the professional climate change denial crowd for their findings.” ”
http://www.guardian.co.uk/science/2012/jul/29/climate-change-sceptics-change-mind

A. Scott

Ross M. – thanks for what must have been a difficult decision … the technical and other insight your comments offer is very worthwhile.

Well done Ross McKitrick, you did right.
I had a feeling you would be up to something at this point.

Ian H

I find the claim that he is a “Koch funded” skeptic to be be particularly aggravating.
Technically he can claim this since the Koch foundation did provide a small amount of funding for BEST. But the story is told in a way that leaves the strong impression that he was once one of those mythical big-oil paid sceptics for hire and is now turning on his oil industry bosses because his conscience won’t allow him to continue his evil ways any more.
Effectively this is a disgusting lie constructed by assembling technical truths in a misleading fashion. I cannot respect anyone who engages in such deceptive tactics.

Brian Johnson uk

You can lead the Media horse to the water of Natural Climate Variability but it will not drink, as that would wash away the manipulated poison that is Green Garbage Hysteria.

Poptech: **I** know he wasn’t, but somehow the world media is convinced that he was.

Poptech: **I** know he wasn’t, but somehow the world media has convinced themselves that he was.

Mike Ozanne

“DavidA says:
July 29, 2012 at 11:03 pm
He’s made his headlines but isn’t making too many friends.
Mike Mann posting on Facebook:”
If we’re talking self aggrandisment, I think you’ve found an unimpeachable source…..

You do have to enjoy the irony and pure inevitability of it. Mann and Muller; two little men with egos the size of planets, both competing for glory, both on the same alarmist side and both determined “there can only be one.” With personalities like that, it was always going to be a fight to the death.
http://thepointman.wordpress.com/2012/06/22/mullering-the-data/
Who says science is ego free?
Pointman

Harold Ambler

From a blog piece I did yesterday:
The logic Muller employs is devastatingly weak. He says the recent warming matches the recent rise in carbon dioxide. But he knows that there have been past warmings, and he knows that they by definition were not caused by human emissions. How he can determine that the past physics that drove past warmings have ceased to be operative? In other words, if the Sun rises 39 billion, 999 million, 999 thousand, 999 times, and on the 40 billionth time the Sun rises I deem that it has done so for a new reason, most sane folks would declare me mad. And they would be right.
There is no compelling reason to look fearfully at the climate system until it begins to exhibit new behavior, just as there is no compelling reason to ascribe the Sun’s rising to anything new.

More here

KnR

That Mann has a go at Muller is not a big surprise, Muller’s heretical views on the scared hockey stick , which is Mann favourite ‘child ‘ , and Mann’s never forget and never forget approach means there is no love lost between them.
But there may well come a point in which the needs of ‘the cause ‘ overrule the bitching between them , if Muller is willing to bow down before the ‘one true god and his icons’ I am sure all will be forgiven and ‘the Team’ mobilised in support .
But if were lucky it will turn into a bitching slapping contest to the benefit and amusement of all.

The Guardian ones again proves it is run by evangelists who will lie to the British public (the diminishing number who buy their lies anyway) to push their religion.

John Doe

The circus is back in town. Pass the popcorn!

Caleb

The irony of Mann sniping at Muller approaches being absurdity.
I mean, if you can’t even handle support, how the h— can you handle pointed criticism? Yet it is exactly that pointed (and sometimes barbed) criticism that is most condusive to growth.
Perhaps living in an echo-chamber causes deafness?

NMR

Has the endothermic element of the androgenic communities’ repartee been factored into any of the formulations? CO2-laden hot air is certainly contributory, irrespective of its origins.

MangoChutney

KnR says: July 30, 2012 at 4:28 am
That Mann has a go at Muller is not a big surprise, Muller’s heretical views on the scared hockey stick ,

I’d say the Hockey Stick was terrified, not just scared 😉
(Apologies in advance)

And how do you explain that an economist would function as a peer reviewer of a publication for Geophysicists (never mind that he also just so happens to run a “climate skeptic” think tank)?
This story smells…

Muller deserved to be thrown under the bus, as he was, but he still seems to think he’s on his pedestal.

ferdberple

Muller’s method of comparing temperature change to static population levels, rather than the change in population levels is fundamentally flawed at such a basic level that it cannot be accidental.
For example, if you compared the change in the speed of your car (on level ground) with how far down the gas pedal was pressed, you would see no correlation between the gas pedal and your car’s speed.
This is what Muller has done and from this he concludes there is no correlation between temperature and urbanization.
However, if you compared the change in the car’s speed to the change in the gas pedal, you would find very close correlation. This indicates a possible cause and effect relationship between the gas pedal and the car’s speed.
Muller has not done this. He did not look at the change in urbanization. He looked at static urbanization and compared this to the change in temperature.
This is such an obvious flaw that one can only conclude that this was not accidental. Rather that a flawed methodology was knowingly used to try and disprove a connection between population change and climate change.
Thus leading to Muller’s conclusion that in the absence of any other explanation, the cause must be CO2. The logically fallacy of ignorance. If we can’t find the cause, then whatever we did find must be the cause. This was the same argument that led to the burning of witches. If we can’t find the cause, the cause must be what we can find, our neighbors.

JC

“I believe the findings in our papers are too important to wait for the year or longer that it could take to complete the journal review process.”
Too important for peer review!!! Priceless. Absolutely priceless.

Cardin Drake

More of the same: Conclusions first, data later.

Brian H

That was unkind. Quoting Elizabeth Muller after McKitrick is like putting a turd on a steak.

Willis Eschenbach

Ross, first I want to thank you for your difficult decision to make the reviews public, and for your ethical actions in both contacting the authors and in waiting until there was no other alternative.
Second, I want to commend you on the clarity and insight of your review. I can see why the editors wanted you as a reviewer.
Finally, to everyone that has been jumping on my head because for a year or so I have been repeatedly claiming that Richard Muller is a lying, two-faced weasel, saying that such language was unacceptable, the elegance and gentility of Ross’s language has made me see your point.
As a result, in the future I will restrict myself to calling Muller a prevaricating, Janus-faced mustelid …
w.

thisisnotgoodtogo

The Mullers have barely tapped the immense reservoir of contempt they’ve earned.

Willis Eschenbach says:
July 30, 2012 at 8:53 am
OK you owe me a new monitor the old one being soaked with a large amount of soda that sprayed out when I laughed “prevaricating, Janus-faced mustelid” I will have to find an occasion to use this at work just so I can watch the confused looks from the higher up educated idiots. Ross well done and well said.

David A. Evans

ponfi says:
July 30, 2012 at 6:40 am

And how do you explain that an economist would function as a peer reviewer of a publication for Geophysicists (never mind that he also just so happens to run a “climate skeptic” think tank)?
This story smells…

Perhaps because Geophysicists doubt their own competence in statistics.
DaveE.