Why the BEST papers failed to pass peer review

Red Fail
Red Fail (Photo credit: griffithchris)

Whoa, this is heavy.  Ross McKitrick, who was a peer review referee for the BEST papers with the Journal of Geophysical Research got fed up with Muller’s media blitzing and  tells his story:

excerpts:

In October 2011, despite the papers not being accepted, Richard Muller launched a major international publicity blitz announcing the results of the “BEST” project. I wrote to him and his coauthor Judy Curry objecting to the promotional initiative since the critical comments of people like me were locked up under confidentiality rules, and the papers had not been accepted for publication. Richard stated that he felt there was no alternative since the studies would be picked up by the press anyway. Later, when the journal turned the paper down and asked for major revisions, I sought permission from Richard to release my review. He requested that I post it without indicating I was a reviewer for JGR. Since that was not feasible I simply kept it confidential. 

On July 29 2012 Richard Muller launched another publicity blitz (e.g. here and here) claiming, among other things, that “In our papers we demonstrate that none of these potentially troublesome effects [including those related to urbanization and land surface changes] unduly biased our conclusions.” Their failure to provide a proper demonstration of this point had led me to recommend against publishing their paper. This places me in an awkward position since I made an undertaking to JGR to respect the confidentiality of the peer review process, but I have reason to believe Muller et al.’s analysis does not support the conclusions he is now asserting in the press.

I take the journal peer review process seriously and I dislike being placed in the position of having to break a commitment I made to JGR, but the “BEST” team’s decision to launch another publicity blitz effectively nullifies any right they might have had to confidentiality in this matter. So I am herewith releasing my referee reports.

Read it all here

Some backstory via Andrew Revkin from Elizabeth Muller. Revkin asked:

1) What’s the status of the four papers that were submitted last fall (accepted, in review…etc?)

2) There can be perils when publicity precedes peer review. Are you all confident that the time was right to post the papers, including the new one, ahead of review? Presumably this has to do with Tuesday deadline for IPCC eligibility?

Here’s her reply:

All of the articles have been submitted to journals, and we have received substantial journal peer reviews. None of the reviews have indicated any mistakes in the papers; they have instead been primarily suggestions for additions, further citations of the literature. One review had no complaints about the content of the paper, but suggested delaying the publication until the long background paper, describing our methods in detail, was actually published.

In addition to this journal peer review, we have had extensive comments from other scientists based on the more traditional method of peer review: circulation of preprints to other scientists. It is worthwhile remembering that the tradition in science, going back pre World War II, has been to circulate “preprints” of articles that had not yet been accepted by a journal for publication. This was truly “peer” review, and it was very helpful in uncovering errors and assumptions. We have engaged extensively in such peer review. Of course, rather than sending the preprints to all the major science libraries (as was done in the past), we now post them online. Others make use of arXiv. This has proven so effective that in some fields (e.g. string theory) the journalistic review process is avoided altogether, and papers are not submitted to journals. We are not going to that extreme, but rather are taking advantage of the traditional method.

We note that others in the climate community have used this traditional approach with great effectiveness. Jim Hansen, for example, frequently puts his papers online even before they are submitted to journals. Jim has found this method to be very useful and effective, as have we. As Jim is one of the most prominent members of the climate community, and has been doing this for so long, we are surprised that some journalists and scientists think we are departing from the current tradition.

The journal publication process takes time. This fact is especially true when new methods of analysis are introduced. We will be posting revised versions of 3 of the 4 papers previously posted later today (the 4th paper has not changed significantly). The core content of the papers is still the same, though the organization and detail has changed a fair amount.

The new paper, which we informally call the “Results” paper, has also undergone journal peer review (and none of the review required changing our results). We are posting it online today as a preprint, because we also want to invite comments and suggestions from the larger scientific community.

I believe the findings in our papers are too important to wait for the year or longer that it could take to complete the journal review process. We believe in traditional peer review; we welcome feedback the public and any scientists who are interested in taking the time to make thoughtful comments. Indeed, with the first 4 papers submitted, many of the best comments came from the broader scientific community. Our papers have received scrutiny by dozens of top scientists, not just the two or three that typically are called upon by journalists.

The climate data they don't want you to find — free, to your inbox.
Join readers who get 5–8 new articles daily — no algorithms, no shadow bans.
5 1 vote
Article Rating
107 Comments
Inline Feedbacks
View all comments
Robert of Texas
July 30, 2012 7:42 pm

Muller’s press release – same old…stuff
McKitrick’s analysis – Priceless – even I could follow his points.

rogerknights
July 30, 2012 7:47 pm

ponfi says:
July 30, 2012 at 6:40 am
And how do you explain that an economist would function as a peer reviewer of a publication for Geophysicists (never mind that he also just so happens to run a “climate skeptic” think tank)?
This story smells…

Under the “here” link, McKitrick says, in the 2nd item, “our conclusion is that a valid explanatory model of the pattern of climate change over land requires use of both socioeconomic indicators and GCM processes. The failure to include the socioeconomic factors in empirical work may be biasing analysis….” He was an expert on the subject of the impact of non-CO2 changes on the climate, which he said were major, and Muller was claiming they were negligible, so he got sent the paper to review.

JMW
July 31, 2012 4:55 am

Seems a perfectly sound strategy.
Take some data.
Sift trough for the data that supports the view (or can be made to seem to do so) and junk the rest.
Write up a paper under the university heading, make sure you spell “professor” right and send for peer review late. Include some creative charts with it.
Then publish anywhere.
Now look at what you have:
The paper is by a scientist.
It says what they want it to say and it has been published and it has been been peer reviewed – That the publishers didn’t peer review it and those that peer reviewed it refused to publish is a minor quibble, it is now IPCC gold standard source material.
.

July 31, 2012 9:29 am

RE: ponfi: July 30, 2012 at 6:40 am
And how do you explain that an economist would function as a peer reviewer of a publication for Geophysicists
The primary explanation is that Muller didn’t stick to geophysics, made ill-supported statements regarding attribution and advocated a policy in his conclusion, which encroaches into the field of economics. A reason J. Curry declined to be a co-author is Muller made attribution statements Curry felt were unjustified.
Had he stuck to just attempting to improve the resolution of the past 100 year temperature record, you might have a point. Statisticians and mathematicians still need to be reviewers.
My gut feel is that the BerkleyE process will be an epic failure when finally peer reviewed by people who do Fourier Analysis for a living. Not only does the scalpel and suture not remove UHI, but it can multiply the UHI effect by turning a saw-tooth wave into a straight climbing line. Low frequencies must be retained in the temperature records and the scalpel destroys low frequencies.
Stephen Rasey,
B.Sc Geophysical Engineering
Ph.D. Mineral Economics

July 31, 2012 11:03 am

Journal peer review is so paper age 20th century .
Nick Stokes is right . Those preprints in the old days were almost always the typeset proofs from the journal . And the entire motivation for peer review before publication was the substantial cost of preparing a technical paper for paper publication . The Web has disintermediated that into history and the only value of journal publication is branding . And the old brands like Science and Nature have lost much of their cachet due to the corrupt “boys club” character of their review process .
It’s continually annoying to see the “climate scientology” apologists act as if it were some exalted field rather than just a branch of applied physics which anyone with a strong quantitative education has all the expertise required to appropriately , and severely , evaluate .

Steven Sullivan
July 31, 2012 11:19 am

I seriously doubt scientists are less eager to get papers in Science or Nature due to some AGW ‘skeptics’ deciding it’s old hat and/or corrupt. That idea bespeaks either unfamiliarity with science and scientists, or wishful thinking. Probably both.

July 31, 2012 12:09 pm

SS
At this point , I’d rather have the WUWT or Heartland imprimatur on these topics than either Science or Nature , or Scientific American , all of which I respected in my youth .

Steven Sullivan
July 31, 2012 12:45 pm

Er….SciAm isn’t a journal original research. It’s a popular science magazine. You know that, right? Are you at all involved in science professionally?
Anyway. A more interesting question: Why does the headline of this post refer to , ‘BEST papers’, plural, when McKitrick’s whinging is solely about Wickham et al.’s (2011/revised 2012) “Influence of Urban Heating on the Global Temperature Land Average Using Rural Sites Identified from MODIS Classifications”, and the recent press for BEST has been about a different paper entirely: Rhode et al. (2012) “A New Estimate of the Average Earth Surface Land Temperature Spanning 1753 to 2011”. Is WUWT or RM claiming that *this* paper has been rejected?
And as for AW’s plaintive query #2, “What’s the status of the 4 papers submitted in 2011′, the answer for at least one of them is posted right there on the BEST website:
“The final paper has been provisionally accepted (pending the acceptance of the paper on the Averaging process) by JGR Atmospheres, and has not changed significantly since October 2011. It is posted again here for convenience:
“Decadal Variations in the Global Atmospheric Land Temperatures””

Christoph Dollis
July 31, 2012 3:16 pm

This Richard Muller is a might skeevy.

Hilary Ostrov (aka hro001)
July 31, 2012 8:46 pm

Sean says: July 30, 2012 at 11:33 am

Re Elizabeth Muller’s comments, e.g.
“All of the articles have been submitted to journals, and we have received substantial journal peer reviews. None of the reviews have indicated any mistakes in the papers; they have instead been primarily suggestions for additions, further citations of the literature.”
Either she is a liar or she is illiterate and can not read the reviews. Neither possibilities reflect well on her.[…]

Alternatively, perhaps she took a few leaves out of David Karoly’s book of “revisionist” scholarship … which also does not reflect well on her … or on dear old Dad, whom she seems to have very tightly wrapped around her little green finger!
But the credibility of les Mullers is certainly not enhanced by Ross McKitrick’s:

[Update July 30: JGR told me “This paper was rejected and the editor recommended that the author resubmit it as a new paper.”]

July 31, 2012 9:58 pm

Bunny Rabbit disorder boy, let me know when Mosher “takes apart” a paper by actually publishing something not simply commenting on the fake skeptic site of Judith Curry,
http://www.populartechnology.net/2012/05/truth-about-judith-curry.html

Hilary Ostrov (aka hro001)
July 31, 2012 10:37 pm

Poptech, with all due respect for a lot of good work that you’ve done, I’m afraid your “portrait” of Judith Curry does us all a disservice. You wrote this in May 2012 and rather than acknowledging her accomplishments, integrity and openness to those of us of the skeptic persuasion – not to mention her frequent criticisms of the IPCC/UNFCC and her willingness to openly challenge her colleagues when she believes they are wrong – and all you could focus on was dredging up quotes from the very distant past?!
Shame on you!
I’ve no idea what your beef is, Poptech. But perhaps its time to get over it!

August 1, 2012 1:21 am

hro001, How does the truth do anyone a disservice? I am sorry if the truth hurts but her record does not change no matter how much you want to white wash it. Every claim I made is fully cited and sourced.
Her recent past is directly relevant to her recent attempts at being an “arbiter” of scientific integrity and trust. Especially when this behavior continued,
Her comment on McIntyre only finding “relatively minor errors” in Mann’s Hockey Stick papers was post-climategate (2009). So was her comment about Climategate scientists, “I don’t think anybody’s come at this with bad motives”. She also continued to use the word “denier” right here at WUWT,
http://wattsupwiththat.com/2010/02/24/on-the-credibility-of-climate-research-part-ii-towards-rebuilding-trust/
http://wattsupwiththat.com/2009/11/28/the-curry-letter-a-word-about-deniers/
I think her alleged epiphany is a big joke and never trusted her motives anymore than I trusted Muller – and I was right about Muller. The fact that her site has so many people who do not know her real background is disturbing. It is like a bunch of sheep to the slaughter. The ploy works well as she attracts the types that delusionally believe they are more “rational” and “moderate” then everyone else but are still intellectually honest about ridiculous alarmist claims. Why do you think Mosher spends so much time there? This only works so well because so many came into the debate post-Climategate.
What exact “accomplishments” and “integrity” should I acknowledge? That she was calling skeptics like me “deniers” in front of congress? I will never have any respect for nor trust any scientist who was knee-jerk calling skeptics – “deniers” because they would not accept the scientist’s alarmist positions.
People can read it and make up their own minds.

Paul Vaughan
August 1, 2012 7:27 am

Peer review’s INTRACTABLY corrupt.
I LOUDLY applaud ALL circumvention of it.
The ONLY thing that matters about Muller & “BEST”:
100% ignorance &/or deception on natural variability.
Unacceptable time & resources (including Ross McKitrick’s) are being drained into this unworthy sideshow. The level of interest in this non-story emphasizes that the climate discussion is suffering from a FATAL leadership deficit. We need an INTENSE shake-up at the highest-level ranks of the non-alarmist movement as the current “leadership” is taking us nowhere fast. I suspect high-level infiltration has driven the sabotage.
Volunteer-research update: I have isolated a non-stationary ~13.44 year pattern in LOD that beats with the nonstationary solar-terrestrial-climate weave pattern [ http://i49.tinypic.com/2jg5tvr.png ] to generate a nonstationary ~62.5 year climate wave. This formerly missing link resolves a framework connecting semi-annual, annual, QBO, & ~1470 year variations. There’s an observational twist on this story that some will find unbearably unpalatable. I suspect that some will be so angered by the observation that they will instantly prescribe hanging without trial. We need to eliminate (the influence of) ignorant &/or deceptive judges without further delay no matter the fallout.

Steven Sullivan
August 1, 2012 10:05 am

It appears Poptech is the arbiter and guardian of the True Skeptics list. He flits around the blogs — just in the last few days I’ve seen him on Eli’s and Revkin’s — insisting that Muller was NEVER a skeptic, SO THERE. Now apparently neither was Judith Curry, superstar, either.
Regarding Paul Vaughn’s suspicions of ‘high-level infiltration’ driving the ‘sabotage’, and his volunteer research uncovering a climate science ‘missing link’: I *suspect* PV is a crank.
But it’s all most amusing, do continue. And let me know when Anthony has submitted his paradigm -changing new work to E&E, as I *suspect* that’s the only place that will publish it without serious revision. [you might wish to expand on your reasoning here, so we all can add to what we know , thanks . . kbmod]

August 1, 2012 11:35 am

JMW says:
July 31, 2012 at 4:55 am

It says what they want it to say and it has been published and it has been been peer reviewed – That the publishers didn’t peer review it and those that peer reviewed it refused to publish is a minor quibble, it is now IPCC gold standard source material.

My new term for it is “peer rejected”. SM and JC, having been released from the confidentiality closet, are making it quite clear why.

August 1, 2012 11:37 am

Sorry, got my initials crossed above. That should be RM and JC, of course.

August 1, 2012 5:01 pm

Steven Sullivan, I am not “insisting anything”, I actually provide quotes and sources to support my arguments. The quotes speak for themselves,
“The influence of global warming deniers, consisting of a small group of scientists plus others that are motivated to deny global warming…” – Judith Curry, 2006
“Gore’s statement in the movie is that we can expect more storms like Katrina in a greenhouse-warmed world. I would agree with this” – Judith Curry, 2006
“We’re looking at a much worse [Hurricane] risk than people were thinking about a year ago …some places are going to become uninsurable.” – Judith Curry, 2006
“If Al Gore reaches more people and convinces the world that global warming is real, even if he does it through exaggeration and distortion – which he does, but he’s very effective at it – then let him fly any plane he wants.” – Richard Muller, 2008
“There is a consensus that global warming is real. …it’s going to get much, much worse.” – Richard Muller, 2008
“Let me be clear. My own reading of the literature and study of paleoclimate suggests strongly that carbon dioxide from burning of fossil fuels will prove to be the greatest pollutant of human history. It is likely to have severe and detrimental effects on global climate.” – Richard Muller, 2003
Everyone is of course free to make up their own minds, all I can do is provide evidence to support my arguments. I certainly don’t believe either are skeptics, then or now.

Paul Vaughan
August 1, 2012 8:12 pm

Steven Sullivan (August 1, 2012 at 10:05 am) wrote
“volunteer research […] I *suspect* PV is a crank.”

Ignorance &/or deception.
“eyeballs deep in muddy water”Tool – The Pot
By dozens of methods the solar-terrestrial-climate weave [ http://i49.tinypic.com/2jg5tvr.png ] is robustly observed with crystal clarity via:
A. Law of Conservation of Angular Momentum.
B. Central Limit Theorem.
C. Thermal Wind Relation.
The data are here:
ftp://ftp.iers.org/products/eop/long-term/c04_08/iau2000/eopc04_08_IAU2000.62-now
You would sound equally sensible suggesting 1 + 1 does not equal 2.
You can find both patterns (in a more-difficult-to-recognize visual format) in a 1997 NASA JPL publication (that does not address the patterns in writing).

Steven Sullivan
August 1, 2012 9:16 pm

Apparently JC Superstar’s blog-famous testimonials from a couple of years back, about how she USED TO accept the IPCC consensus, but no longer does, have not impressed. Therefore she is NOT of the body! It is the Will of Poptech.

Hilary Ostrov (aka hro001)
August 1, 2012 9:22 pm

Poptech says August 1, 2012 at 1:21 am

People can read it and make up their own minds.

Not when you provide a distinctly cherry-picked half the story. Seems to me that if you really wanted people to make up their own minds (rather than blindly following yours!) then you would have provided a more complete picture.
But speaking of making up one’s own mind … I don’t know if you’ve seen my “portrait” of Muller (which preceded yours by eight months, give or take a day or so), but you might be interested in it:
Will the real Richard Muller please stand up
Then again – since you seem bound and determined to ludicrously equate Muller and Curry – perhaps not.
Oh, well … never mind!
P.S. There’s an occasional denizen at Judth Curry’s blog whose only “contributions” consist of rude off-topic derogatory rants against the hostess. Goes by the nym of “cwon”; s/he also refers to her as a “fake skeptic”. Quite incorrectly, I might add – particularly since Dr. Curry has never claimed to be a “skeptic”, and in fact has often made the point that labelling from either side adds nothing to the discussion.
Would “cwon” be a friend of yours, by any chance?!

August 1, 2012 10:16 pm

hro001, Nothing in my article is cherry picked. Those quotes cannot be interpreted any other way, they are all in the explicity derogatory context she used then in.
Why are you so scared people will learn about the real Judith Curry who said that world renowned hurricane expert Dr. Gray had “brain fossilization” not the white-wash facade you would like to present?
http://blog.chron.com/sciguy/2006/02/speaking-of-hurricanes-off-season-fireworks/

Dr. Curry, in an interview at her Georgia Tech office, said Dr. Gray has “brain fossilization.” She added: “Nobody except a few groupies wants to hear what he has to say.”

Her congressional testimony is a matter of public record,
http://curry.eas.gatech.edu/climate/pdf/testimony-curry.pdf
The influence of global warming deniers, consisting of a small group of scientists plus others that are motivated to deny global warming…” – Judith Curry, 2006
I have no idea who any of her blog groupies are as I only comment on reputable sites like WUWT.
I am well aware that she is trying desperately to cover up her true motivations and the suckers are taking the bait.

Hilary Ostrov (aka hro001)
August 2, 2012 12:15 am

Poptech, what is it about “half the story” that you are having such difficulty understanding?!
And exactly what do you think you are accomplishing with your one-sided one-man smear campaign?
Contrary to what you presume, no one is “white-washing” Judith Curry – or her blog. We all have our blind-spots but some of us are capable of changing our views in the light of new information.

I am well aware that she is trying desperately to cover up her true motivations and the suckers are taking the bait.

And with this ludicrous assertion you have proven yourself to be the mirror-image of those who presume to know the “true motivations” of any and all who oppose them.
Have you been taking lessons from Bain, Mooney, Corner & Lewandowsky and their ilk?!

August 2, 2012 3:18 am

hro001, There is no “other half” to the derogatory statements she made towards skeptics nor the alarmist positions she held and I believe still does.
Smear? Oh, please. What did I state that was not true? If all of my information was not so convincing you would not be so upset. If any of it could be refuted you would have done so.
Anyone is free to change their views but I will never trust anyone who behaved like she did towards skeptics. I find people who do naive. Whether you share my position on her or not is irrelevant to my purpose. I simply want to provide this information to people so they can get the other side of the story about her that they likely have not been told. Especially in light of so many getting off guard with Muller twice. All of this information is completely factual, fully cited and sourced.
I stand by my assertion as I held the same thing with Muller.

Myrrh
August 2, 2012 3:24 am

Blade says:
July 30, 2012 at 1:19 pm
22acaciaavenue [July 30, 2012 at 12:34 pm]
Well that last one should be Blade. I would again like to thank WordPress for changing things like forcing logging in, having randomly expiring cookies, etc. Real professional shop you got there.
=============
Tested this, it still stole my email forcing me to log into wordpress.
And I’m not going to get another email account because you can’t sort this out.