People send me stuff. In this case I have received an embargoed paper and press release from Nature from another member of the news media who wanted me to look at it.
The new paper is scheduled to be published in Nature and is embargoed until 10AM PDT Sunday morning, July 20th. That said, Bob Tisdale and I have been examining the paper, which oddly includes co-authors Dr. Stephan Lewandowsky and Dr. Naomi Oreskes and is on the topic of ENSO and “the pause” in global warming. I say oddly because neither Lewandowsky or Oreskes concentrates on physical science, but direct their work towards psychology and science history respectively.
Tisdale found a potentially fatal glaring oversight, which I verified, and as a professional courtesy I have notified two people who are listed as authors on the paper. It has been 24 hours, and I have no response from either. Since it is possible that they have not received these emails, I thought it would be useful to post my emails to them here.
It is also possible they are simply ignoring the email. I just don’t know. As we’ve seen previously in attempts at communication with Dr. Lewandowsky, he often turns valid criticisms into puzzles and taunts, so anything could be happening behind the scenes here if they have read my email. It would seem to me that they’d be monitoring their emails ahead of publication to field questions from the many journalists who have been given this press release, so I find it puzzling there has been no response.
Note: for those that would criticize my action as “breaking the embargo” I have not even named the paper title, its DOI, or used any language from the paper itself. If I were an author, and somebody spotted what could be a fatal blunder that made it past peer review, I’d certainly want to know about it before the paper press release occurs. It is about 24 hours to publication, so they still have time to respond, and hopefully this message on WUWT will make it to them.
Here is what I sent (email addresses have been link disabled to prevent them from being spambot harvested):
===============================================================
From: Anthony
Sent: Friday, July 18, 2014 9:01 AM
To: james.risbey at csiro.au
Subject: Fw: Questions on Risbey et al. (2014)
Hello Dr. Risbey,
At first I had trouble finding your email, which is why I sent it to Ms.Oreskes first. I dare not send it to professor Lewandowsky, since as we have seen by example, all he does is taunt people who have legitimate questions.
Can you answer the question below?
Thank you for your consideration.
Anthony Watts
—–Original Message—–
From: Anthony
Sent: Friday, July 18, 2014 8:48 AM
To: oreskes at fas.harvard.edu
Subject: Questions on Risbey et al. (2014)
Dear Dr. Oreskes,
As a climate journalist running the most viewed blog on climate, I have been graciously provided an advance copy of the press release and paper Risbey et al. (2014) that is being held under embargo until Sunday, July 20th. I am in the process of helping to co-author a rebuttal to Risbey et al. (2014) I think we’ve spotted a major blunder, but I want to check with a team member first.
One of the key points of Risbey et al. is the claim that the selected 4 “best” climate models could simulate the spatial patterns of the warming and cooling trends in sea surface temperatures during the hiatus period.
But reading and re-reading the paper we cannot determine where it actually identifies the models selected as the “best” 4 and “worst” 4 climate models.
Risbey et al. identifies the 18 originals, but not the other 8 that are “best” or “worst”.
Risbey et al. presented histograms of the modeled and observed trends for the 15-year warming period (1984-1998) before the 15-year hiatus period in cell b of their Figure 1. So, obviously, that period was important. Yet Risbey et al. did not present how well or poorly the 4 “best” models simulated the spatial trends in sea surface temperatures for the important period of 1984-1998.
Is there some identification of the “best” and “worst” referenced in the paper that we have overlooked, or is there a reason for this oversight?
Thank you for your consideration.
Anthony Watts
WUWT
============================================================
UPDATE: as of 10:15AM PDT July 20th, the paper has been published online here:
http://www.nature.com/nclimate/journal/vaop/ncurrent/full/nclimate2310.html
Well-estimated global surface warming in climate projections selected for ENSO phase
Abstract
The question of how climate model projections have tracked the actual evolution of global mean surface air temperature is important in establishing the credibility of their projections. Some studies and the IPCC Fifth Assessment Report suggest that the recent 15-year period (1998–2012) provides evidence that models are overestimating current temperature evolution. Such comparisons are not evidence against model trends because they represent only one realization where the decadal natural variability component of the model climate is generally not in phase with observations. We present a more appropriate test of models where only those models with natural variability (represented by El Niño/Southern Oscillation) largely in phase with observations are selected from multi-model ensembles for comparison with observations. These tests show that climate models have provided good estimates of 15-year trends, including for recent periods and for Pacific spatial trend patterns.
of interest is this:
UPDATE2: rebuttal has been posted
The reason it is more likely to be a “blunder” than an “oversight” is that the authors likely did not and would not intend to tell the reader the actual 4 best and 4 worst models by thisbtest.
Thus, their position amounts to “trust us” — as we have seen so often in CliSci pseudo-science.
Only the authors can tell us whether the omission is accidental or intentional, although either way it is indefensible. How did the reviewers miss this?? oh right, the paper was given the usual lightweight pal review, it seems.
davidmhoffer says:
July 19, 2014 at 2:21 pm
Since we have no information as to what those specific models say going forward, I wouldn’t make that assumption.
Good point. However check out the following. The best so far are also more or less the lowest in the future.
http://wattsupwiththat.com/2014/02/10/95-of-climate-models-agree-the-observations-must-be-wrong/
The point being that while you may be able to find some sort of better fit (whatever that actually means) NOW, unless your estimators are all unbiased (as noted by Jordan), and they constitute an ensemble, any relationship you see NOW, cannot be guaranteed to hold TOMORROW.
This is why there is divergence in the reconstructions Mann keeps shoving down our throats. He is simply too blinded by ideology, or likely, so completely ignorant of the statistics he is employing, that he cannot come to grips with this fact. Phil Plait (another statistical ignoramus) can blather on all he wants about climate statistics and how much climate scientists know about statistics, but at the end of the day, not one of these buffoons really understands the concept of a spurious relationship. And, if they do, they are liars for not saying so.
Mark
Robustness tests for the above paper:
> How do the researchers justify selection of 4 models? Why not use only the “best” model?
> Are the conclusions (assertions) sustained as averaging rises from using only the “best” model to averaging over the top-two, top-three, etc and until all 18 are included in the averaging?
> If the conclusions are not robust by the previous test, what proportion of all possible model combinations would confirm the conclusions?
Kate Forney – great comment with excellent questions and testing of reasoning.
Mosh – “general observation about all the models”. Cannot possibly apply to a biased estimator. We absolutely must demonstrate the expected value of model error is zero as a most basic test of its value.
Mosh: ” If you take the mean of the models you get a better fit. why? dunno. just a fact.”. Declaration of faith in the GCMs. Until/unless you can demonstrate the GCMs are unbiased estimators.
a note on terms: I did not mean to imply above that the accidental/intentional distinction is mirrored precisely by the oversight/blunder distinction,
Under the category of “omission” we would often call an accidental omission an “oversight” — however, if the omission is sufficiently serious and/or significant it can also be a “blunder”…..
i.e., a blunder can be accidental or intentional. If the omission is not too serious and/or there is at least a plausible argument for the omission, then it might be termed only an “omission” or “oversight” which are less loaded terms. However, this issue above seems serious enough that it may well deserve to be termed a blunder. More definite judgment waits upon seeing any response and justification the authors may offer.
Of course, with noted non-scientist charlatans like Lewandowsky and Oreskes in the author list, nothing said by the authors can be relied upon.
Don’t trust, only verify or falsify!
Mark T: “This is also a tacit admission the models are *not* unbiased”
Yes, with one proviso. Even for unbiased estimators there could be loss of certain signals due to averaging of a set of statistically independent observations of the system.
However I do not see this as justification of the methodology used for this paper. Quite the contrary as follows …
If we understand the system to the extent that we know certain signals could be lost by averaging, we would be able to create a single model which produces those signals.
This researchers’ methodology (collecting different model results and averaging) contains a tacit admission that we do not understand the climate system sufficiently well to support their conclusions.
I think only if they are not completely capturing the true physics of the system OR if the observation/sample noise is such that it overwhelms the signals you refer to. If they were completely capturing the physics, then all that *should* be left is random error and parameter variation (since it turns into an initial conditions exercise once all the physics are captured properly). I suppose the latter could include spurious cancellations, which seems to be what you are implying…?
I did not think you were justifying the methodology, btw. Quite frankly, none of us really know what it is except that it is based on models that have not had any rigorous verification applied.
Mark
Oreskes and Lew are political additions to the paper meant to help along in case there were any “bumps” on the review process.
Allowing Steven Mosher to make his confused and confusing comments here is a good thing.
In his opinion, which echoes the opinion of most Climate ‘s’cientists, the models do not need to work, i.e. be useful for prediction, not can they be compared or ranked qualitatively. From the point of view of Warmists these are indeed quite useful attributes.
So, according to Stephen Mosher, the best way to find the bullseye on a dart board is to throw a lot of darts at it and see where the most concentrated cluster of darts are.
Most of us would just examine the dart board itself to get the answer …
Steven M. Mosher, B.A. English, Northwestern University (1981); Teaching Assistant, English Department, UCLA (1981-1985); Director of Operations Research/Foreign Military Sales & Marketing, Northrop Corporation [Grumman] (1985-1990); Vice President of Engineering [Simulation], Eidetics International (1990-1993); Director of Marketing, Kubota Graphics Corporation (1993-1994); Vice President of Sales & Marketing, Criterion Software (1994-1995); Vice President of Personal Digital Entertainment, Creative Labs (1995-2006); Vice President of Marketing, Openmoko (2007-2009); Founder and CEO, Qi Hardware Inc. (2009); Marketing Consultant (2010-2012); Vice President of Sales and Marketing, VizzEco Inc. (2010-2011); [Marketing] Advisor, RedZu Online Dating Service (2012-2013); Advisory Board, urSpin (n.d.); Team Member, Berkeley Earth 501C(3) Non-Profit Organization unaffiliated with UC Berkeley (2013-Present)
I hate embargoed papers.
And the reason I hate embargoed papers is, I can’t reply to comments or answer questions until tomorrow at 1PM Eastern (US) time.
charles nelson says:
July 19, 2014 at 4:33 pm
Steven M. Mosher, B.A. English, Northwestern University (1981); Teaching Assistant, English Department, UCLA (1981-1985); Director of Operations Research/Foreign Military Sales & Marketing, Northrop Corporation [Grumman] (1985-1990); Vice President of Engineering [Simulation], Eidetics International (1990-1993); Director of Marketing, Kubota Graphics Corporation (1993-1994); Vice President of Sales & Marketing, Criterion Software (1994-1995); Vice President of Personal Digital Entertainment, Creative Labs (1995-2006); Vice President of Marketing, Openmoko (2007-2009); Founder and CEO, Qi Hardware Inc. (2009); Marketing Consultant (2010-2012); Vice President of Sales and Marketing, VizzEco Inc. (2010-2011); [Marketing] Advisor, RedZu Online Dating Service (2012-2013); Advisory Board, urSpin (n.d.); Team Member, Berkeley Earth 501C(3) Non-Profit Organization unaffiliated with UC Berkeley (2013-Present)
==============
Yep, and the NSA and IRS didn’t glom on to that comment 🙂
So now psychologists and historians are writing climate papers on the climat.
lol.
Bob Tisdale: “And the reason I hate embargoed papers is, I can’t reply to comments or answer questions until tomorrow at 1PM Eastern (US) time.”
Well Bob, now that the World Cup is over we have all the time in the world tomorrow to read your comments and answers. 🙂
Of course, at my age I may have forgotten the darn questions by then! 🙁
Guys, this is “Climate Science” TM. You need to think inside the box.
What they have obviously done is splice the best parts of the best 4 models to create one modelled result that hides the decline of agreement with real-life observations.
By the way, the name calling on Steve Mosher is completely low class and uncalled for. Sort of a cringe worthy example of ad hom. And I do disagree with him on issues frequently.
For those posting his CV, I suggest that you re-read it very carefully between the lines for content. We have regular columnists here who are quite bright and even more self-educated. He has played in a highly technical league for a long time. Cryptic and caustic? Can be. Some internet self-declared expert who is actually a kook? No. Some of the pile on in this blog thread is unworthy and is not building skeptical critical skills or credibility.
Steve,
I do have a question on the models and averaging them:
Is it not true that error tends to multiply, and as Dr. Pielke, Sr. pointed out more than once, the models as individuals and in ensemble (I paraphrase) show no meaningful predictive skill.
If that is that is the case, why should this sort of study be done before models are constructed that are in fact useful?
There are no real climate models since there happens to be so little historical climate data to base those models on. Recent variation in the high emissions postwar era has near exact precedence in the low emissions era before it yet in the former era the warming is simply unexplained and the postwar cooling only has hand waving excuses for it such as aerosols yet as the such pollution has been reduced we have yet another end of warming, unexplained. If the several major fluctuations in temperature are basically unexplained with no continuous data going back far enough to enter into computer models then there is obviously no valid data being used, just modeled input data too!
So what caused the initial decades of warming? And exactly what data series is input into climate models to reproduce it? Given how likely chaotic ocean cycles have such a massive influence but there is no data other than sea surface temperature as a result, any model that uses the result as *input* isn’t a model at all, just a faithful mirror of already known results. Yet strongly note how fundamental criticism is ignored as the focus is put on lawyerly details by model enthusiasts including the bizarre Frankenstein mixing of models together as if there was any input data to support them. That’s a classic smoke screen meant to get you all upset about post processing details until the thread peters out in obscurity.
http://www.woodfortrees.org/plot/hadcrut4gl/from:1955/to:2013/plot/hadcrut4gl/from:1895/to:1954
WHEN THE STANDARD IS NOT PERFORMANCE.
If the average is best then the climate model nearest the average must be the best model.
So if you are betting on a horse race, averaging the times of all those horses when they last ran a similar race and betting on the horse nearest that average would make you a winner, right?
Eugene WR Gallun
Well, applying Mosher’s logic, it seems that if the climate modelers would just gin up about 500 more models to throw into the mix and average them all together, they should be able to make predictions accurate to about 4 or 5 decimal places. After all, the more models you average, the more accurate the prediction is his reasoning.
I thnk is what Mosher is really saying, is the LAST thing the climate team wants is for infighting to start amongst the modelers when some models get called junk. It would devastate the claim that the science is settled and WOW… what a field day the skeptics would have.
So if you are betting on a horse race, averaging the times of all those horses when they last ran a similar race and betting on the horse nearest that average would make you a winner, right?
Eugene WR Gallun
That works when your getting paid to bet other peoples money.
Remember too that the biggest slander of all that these model enthusiasts have very much played along with is how:
(A) All climate alarm is based on a highly speculative amplification of the old school greenhouse effect.
(B) Climate model skeptics are said to in the main deny the old school greenhouse effect.
Yet another massive smoke screen operation going on here to this day to pretend that it’s all just basic physics you see, and denial of that basic physics by the usual creationists and tobacco industry shills even though Al Gore is the tobacco farmer and Michael Mann has hired a tobacco industry lawyer and Phil Jones now uses a Saudi Arabian university as his affiliation and RealClimate.org is site registered to the same notorious PR firm that promoted both the breast implant scare and the vaccine scare.