People send me stuff. In this case I have received an embargoed paper and press release from Nature from another member of the news media who wanted me to look at it.
The new paper is scheduled to be published in Nature and is embargoed until 10AM PDT Sunday morning, July 20th. That said, Bob Tisdale and I have been examining the paper, which oddly includes co-authors Dr. Stephan Lewandowsky and Dr. Naomi Oreskes and is on the topic of ENSO and “the pause” in global warming. I say oddly because neither Lewandowsky or Oreskes concentrates on physical science, but direct their work towards psychology and science history respectively.
Tisdale found a potentially fatal glaring oversight, which I verified, and as a professional courtesy I have notified two people who are listed as authors on the paper. It has been 24 hours, and I have no response from either. Since it is possible that they have not received these emails, I thought it would be useful to post my emails to them here.
It is also possible they are simply ignoring the email. I just don’t know. As we’ve seen previously in attempts at communication with Dr. Lewandowsky, he often turns valid criticisms into puzzles and taunts, so anything could be happening behind the scenes here if they have read my email. It would seem to me that they’d be monitoring their emails ahead of publication to field questions from the many journalists who have been given this press release, so I find it puzzling there has been no response.
Note: for those that would criticize my action as “breaking the embargo” I have not even named the paper title, its DOI, or used any language from the paper itself. If I were an author, and somebody spotted what could be a fatal blunder that made it past peer review, I’d certainly want to know about it before the paper press release occurs. It is about 24 hours to publication, so they still have time to respond, and hopefully this message on WUWT will make it to them.
Here is what I sent (email addresses have been link disabled to prevent them from being spambot harvested):
===============================================================
From: Anthony
Sent: Friday, July 18, 2014 9:01 AM
To: james.risbey at csiro.au
Subject: Fw: Questions on Risbey et al. (2014)
Hello Dr. Risbey,
At first I had trouble finding your email, which is why I sent it to Ms.Oreskes first. I dare not send it to professor Lewandowsky, since as we have seen by example, all he does is taunt people who have legitimate questions.
Can you answer the question below?
Thank you for your consideration.
Anthony Watts
—–Original Message—–
From: Anthony
Sent: Friday, July 18, 2014 8:48 AM
To: oreskes at fas.harvard.edu
Subject: Questions on Risbey et al. (2014)
Dear Dr. Oreskes,
As a climate journalist running the most viewed blog on climate, I have been graciously provided an advance copy of the press release and paper Risbey et al. (2014) that is being held under embargo until Sunday, July 20th. I am in the process of helping to co-author a rebuttal to Risbey et al. (2014) I think we’ve spotted a major blunder, but I want to check with a team member first.
One of the key points of Risbey et al. is the claim that the selected 4 “best” climate models could simulate the spatial patterns of the warming and cooling trends in sea surface temperatures during the hiatus period.
But reading and re-reading the paper we cannot determine where it actually identifies the models selected as the “best” 4 and “worst” 4 climate models.
Risbey et al. identifies the 18 originals, but not the other 8 that are “best” or “worst”.
Risbey et al. presented histograms of the modeled and observed trends for the 15-year warming period (1984-1998) before the 15-year hiatus period in cell b of their Figure 1. So, obviously, that period was important. Yet Risbey et al. did not present how well or poorly the 4 “best” models simulated the spatial trends in sea surface temperatures for the important period of 1984-1998.
Is there some identification of the “best” and “worst” referenced in the paper that we have overlooked, or is there a reason for this oversight?
Thank you for your consideration.
Anthony Watts
WUWT
============================================================
UPDATE: as of 10:15AM PDT July 20th, the paper has been published online here:
http://www.nature.com/nclimate/journal/vaop/ncurrent/full/nclimate2310.html
Well-estimated global surface warming in climate projections selected for ENSO phase
Abstract
The question of how climate model projections have tracked the actual evolution of global mean surface air temperature is important in establishing the credibility of their projections. Some studies and the IPCC Fifth Assessment Report suggest that the recent 15-year period (1998–2012) provides evidence that models are overestimating current temperature evolution. Such comparisons are not evidence against model trends because they represent only one realization where the decadal natural variability component of the model climate is generally not in phase with observations. We present a more appropriate test of models where only those models with natural variability (represented by El Niño/Southern Oscillation) largely in phase with observations are selected from multi-model ensembles for comparison with observations. These tests show that climate models have provided good estimates of 15-year trends, including for recent periods and for Pacific spatial trend patterns.
of interest is this:
UPDATE2: rebuttal has been posted
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Excellent questions that the reviewers should have caught.
Is this the same James Risbey who wrote this paper in 2011: http://www.marine.csiro.au/~ris009/pubfiles/cc_know_ign_clires.pdf ? If so, it is hard for me to square what seems to be the thrust of the current paper with “The ability of CGCMs to simulate changes in the 3d flow in the atmosphere is
severely hampered by the lack of resolution in the ocean component of current
CGCMs. The ocean models in CGCMs used for climate projections do not resolve
mesoscale eddies. This means that they don’t resolve the main source of dynamic
instability of the flow in these models and only very crudely parameterize some of
the components of that instability (Section 3.2).” If it is the same author, did he make a breakthrough or do CGCMs at the global level not suffer from these same limitations?
the selected 4 “best” climate models
============
the obvious mechanism is that they checked all the models and cherry picked the 4 that accidentally happened to have the best fit with observations.
as has been shown repeatedly, when you cherry pick a sample from a larger population because they happen to match observations, this does not demonstrate the sample has any skill at predicting the observations. the laws of probability tell us that some members of a population will match the observations simply by chance.
thus, for example, the hockey stick, and similar results. selection on the dependent variable leads to spurious correlations.
What a discouraging start to a lovely summer weekend; an invitation to review the latest weeping excrescence from the anti-science Púca worshippers. Needless to say, given the authors, it’s a very dubious trap.
Now because I commented that, I must be a conspiracy nut. Not that Oreskes and Lewnydowsky are capable of truly being conspiracists; because they’re blind oblivious parasitic fleas (Xenopsylla cheopis)chock full of enterobacteria (Yersinia pestis) infesting the most disgusting rats. A proper conspiracist must be capable of maintaining multiple layers of deceit; whereas the CAGW believers tend to stick with outbursts of opprobrium and weird application of statistics to poorly kept data.
Speaking of poorly kept data. Anyone else suspect that the tango twins mentioned above are actually waiting for skeptics to thresh the models looking for a so-called best four?
What factors truly make any model best? Just because one accidentally seems to replicate a chosen period? Running what entry positions? Does the model return the same results every time?
Will all data be posted?
Will all model code be published?
Anthony: Your response is well made and spoken. You are treating them as proper scientists. As you’ve demonstrated so often, you are responding as a gentleman would respond to gentlefolk.
Be careful with any next steps. Remember a previous Lewnydoodoo involved a deliberate deception on who sent what. The lack of a response is anomalous or perhaps intentional.
Good Luck!
Eliza says:
July 19, 2014 at 10:32 am
WUWT mistake again..Throwing your pearls to the pigs. I would have kept the info, allowed it to be published and THEN only then nailed both the authors and The Journal. Huge mistake once again by WUWT re SG
————————————————
Perhaps that’s the difference between an honest skeptic and dishonest ideologues.
You do realize the presence of non physical scientists Oreskes and Lewandowsky on the list of authors is probably so the BBC can treat them as valid “climate experts”. I guess we have to look forward to a period of their “views” being paraded by the Beeb as consensus climate science.
=================================================================
A mistake? Not if the goal is accuracy and honesty in the field.
Oreskes on climate science???
Small world, not many rats left onboard the sinking AGW-vessel.
This can only be an act of desperation.
Anyway, dont expect a more helpful response than:
“Why should I make the data available to you, when your aim is to try and find something wrong with it…”
One of the key points of Risbey et al. is the claim that the selected 4 “best” climate models could simulate the spatial patterns of the warming and cooling trends in sea surface temperatures during the hiatus period.
Well I suppose I will have to wait for the paper, but the obvious follow up question would be how well did they simulate the spatial patterns prior to the hiatus period? Further, how well did they simulate spatial patterns other than sea surface both before and after the hiatus period? Four models getting one part of the problem right for one part of the time = FAIL.
What might be equally interesting is if this provokes a couple of other possible reactions:
1. The modelling groups fingered as “the worst” defending their position and in so doing, attacking this paper’s credibility.
2. If the paper holds up, and the four worst are really that much different and that bad, then what is the excuse for continuing to use them as part of the ensemble mean? If the paper holds up, these models should be dropped from the ensemble mean for their inaccuracy, the side effect of which would be to lower the sensitivity calculation of the ensemble.
Let’s see.
We know there are 4 best and 4 worst.
It might not be an oversight to not name them.
Hint. Modelers and those who evaluate models generally don’t identify the best versus the worst.
Leads to model wars.
Mine is bigger. No mine is.. Blah blah blah
There are some authors who do name names however.
Hoffer gets at one reason for not naming names.
But one can not simply throw out the worst
The issue is the four worst on this test will be the best on
Some other test
Anthony, I think your approach was the best of all the options, and I would also agree with Mr. Mosher that omission is the better perspective, but the problem with Mr. Mosher’s comment was that you used it in regard to one of your papers and not that of Risbey. You wrote “oversight” to that affect.
” Mick says:
July 19, 2014 at 11:17 am
You do realize the presence of non physical scientists Oreskes and Lewandowsky on the list of authors is probably so the BBC can treat them as valid “climate experts”. I guess we have to look forward to a period of their “views” being paraded by the Beeb as consensus climate science.”
Now thats a brilliant observation, very good Mick!
You nailed it!
I seem to recall during the climategate thing that there was some controversy over which data were used for a particular analysis; that the selection of stations — several out of very many, if I recall correctly, was neither published nor furnished upon request. The reply was “we provided you with all the data”.
We could be seeing the early days of a similar kind of reply here.
Hmm. I’ve got an undergraduate degree in political theory, so it seems that qualifies me to co-author a paper on climate change? I can certainly pitch in lessons learned from Machiavelli. Whoops! Seems they’ve already read the Cliff Notes! Anthony, I hope they don’t pursue legal action against you regarding the embargo. But I wouldn’t be at all surprised if they did.
Steven Mosher;
Steven Mosher says:
July 19, 2014 at 11:31 am
Hoffer gets at one reason for not naming names.
But one can not simply throw out the worst
The issue is the four worst on this test will be the best on
Some other test
>>>>>>>>>>>>>>>>>>>>
Ah yes. If the model got something right, we should keep it’s results across the board and average the parts known to be wrong into the ensemble anyway. Pffft.
Steven Mosher says:
July 19, 2014 at 11:31 am
Hoffer gets at one reason for not naming names.
But one can not simply throw out the worst
The issue is the four worst on this test will be the best on
Some other test
Steven, you been eating your weetabix again. That is a really important point you make. Good on yer. Only thing I would change is “will” to might or could or maybe. 🙂
But one can not simply throw out the worst
The issue is the four worst on this test will be the best on
Some other test
================
thus demonstrating that it is chance, not skill that is determining the model results.
think of it this way. you are back in school. every day there is a math quiz. the worst 4 students one day are not going to be the 4 best students another day, no matter how many times you give a quiz. that is because the scores on the math quiz reflect skill.
It is very difficult to properly understand what is going on without sight of the paper.
it does surprise me that if reliance is being placed upon the ‘best’ 4 models full details of these models how they are tested, validated and what they say over their entire range is not set out in the paper as well as the reason for selecting those particular models. Are they superior in some way/ or is it just per chance that their outputs better reflect observational data?
Whilst I can see both the pros and the cons of contacting the authors with your enquiry prior to publication of the paper, and I can therefore see why you considered that to contact them is the best approach (although others may disagree, I myself consider it is the best approach), I am not sure why you would wish to share this with us, the readers of your blog, prior to the publication of the paper.
When the paper is published, you could have provided a copy of the paper on this blog, and at the same time set out your (and Bob’s) comments, and explain that you had contacted the authors in advance of publication but they had not responded. That might have been the most ‘saintly’ approach, since it is possible that people will not like the fact that you have referred to an embargoed paper, in advance of publication, and in future you may not be given copies of such papers in advance. Not a criticism, just a thought.
Also, I am unsure, from a tactical perspective, why in your message you would say “.. I am in the process of helping to co-author a rebuttal…” since this may cause the shutters to go up, whereas a more neutral response not mentioning this fact, but merely raising your enquiry regarding the models might be more likely to elicit a constructive response from the authors. As soon as you mention rebuttal, the authors no doubt jump into defence mode. That could explain their lack of response.
Again, not a criticism per se, just my thoughts.
“ferdberple says:
July 19, 2014 at 12:17 pm
But one can not simply throw out the worst
The issue is the four worst on this test will be the best on
Some other test
================
thus demonstrating that it is chance, not skill that is determining the model results.
################
not really. clearly you havent looked at the matter
“Steven, you been eating your weetabix again. That is a really important point you make. Good on yer. Only thing I would change is “will” to might or could or maybe. 🙂
Gavin and others have made the same point. Its a known problem cause the democracy of the models.
not really headline news
hooofer
“Ah yes. If the model got something right, we should keep it’s results across the board and average the parts known to be wrong into the ensemble anyway. Pffft.”
I wish that willis were here to tell you to quote my words
Simple fact is that the avergae of models is a better tool than any given one.
deal with it.
Same with hurricane prediction in some cases.
does it make sense to average models? probably not. But you get a better answer that way
so until someone devises a test to score models.. that is what you have,
pragmatics
I agree with others that it will be useful to know which the best are for this particular feature of climate, because they can then be tested against other features. If they aren’t the best for other features then it would seem more likely they match by chance than by skill. Surely the AGW community has somehow to narrow down the vast range of models to a handful that they consider to be most skillful? Discarding the obvious outliers would demonstrate progress.
Colluded before so here is a context for a trio. Note the ages and probable experience.
Whack javascript off first
http://www.scientificamerican.com/author/stephan-lewandowsky-james-risbey-and-naomi-oreskes/
so until someone devises a test to score models…
We have a test score for models: accurate, repeated predictions.
Reality = 1
Models = 0