This is something I never expected to see in print. Climate modeler Dr. Gavin Schmidt of NASA GISS comments on the failure of models to match real world observations.
Source:
[ http://twitter.com/ClimateOfGavin/status/340605947883962368 ]
While the discussion was about social models, it is also germane to climate modeling since they too don’t match real world observations. Below is an example of climate models -vs- the real world; something’s clearly not right.
Graph source: IPCC AR5 draft
Is it maths or assumptions (or both) that cause the divergence?
UPDATE: In comments, I had a discussion with reader “jfk” which I think is worth sharing. He made some good points, and it helped hone my own thinking on the issue:
jfk says: Submitted on 2013/06/01 at 8:40 am
Well, I still think it’s a bit unfair to Gavin (and I am no fan of his). But hey, it’s Anthony’s site.
For a good review of the many failures of statistical modeling in social sciences (and one or two successes) see the book “Statistical Models: Theory and Practice” by David Freedman. Whether or not climate modeling has devolved to the point where it is social science rather than physics, well, I hope it’s not quite that bad…
REPLY: And I think it is more than a bit unfair to us, that if he believes what he tweets, he should re-examine his own assumptions about climate modeling. We have economies, taxes, livelihood, etc. hinging (or perhaps failing) on the success of these models to predict the climate in the future. The models aren’t working, and Dr. Schmidt knows this. Unfortunately his job is tied to the idea that they do in fact work. I feel no regrets at making this comparison front and center. – Anthony
UPDATE2: RussR in comments, provides this graph below showing Hansen’s modeled scenarios against real world observations. He writes:
Here’s an excel spreadsheet comparing observed temperatures vs. model projection from: Hansen (1988), IPCC FAR (1990), IPCC SAR (1995) and IPCC TAR (2001), in pretty charts.
It can be updated as more observations are added.
https://dl.dropboxusercontent.com/u/78507292/Climate%20Models.xlsx
UPDATE3: Dr. Roger Pielke Sr. adds this in comments.
Climate models are engineering code with quite a few tunable parameters, and fitting functions in their parameterization of clouds, precipitation, land-atmospheric interfacial fluxes, long- and short-wave radiative flux divergences, etc. Only a part of these models are basic physics representations – the pressure gradient force, advection, the Coriolis effect.
The tunable parameters and fitting functions are developed by adjustment from real world data and a higher resolution models (which themselves are engineering code), but only for a quite small subset of real world conditions.
I discuss this issue in depth in my book
Pielke Sr, R.A., 2013: Mesoscale meteorological modeling. 3rd Edition, Academic Press, in press. http://www.amazon.com/Mesoscale-Meteorological-Modeling-International-Geophysics/dp/0123852374/ref=sr_1_2?ie=UTF8&qid=1370191013&sr=8-2&keywords=mesoscale+meteorological+modeling
The multi-decadal global climate model projections, when run in a hindcast mode for the last several decades are showing very substantial errors, as I summarize in the article
http://pielkeclimatesci.files.wordpress.com/2013/05/b-18preface.pdf
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Dr. Schmidt is stating the obvious and frankly I am aghast that Taleb would make such a obviously idiotic statement. Math is exact, but as Gavin points out, essentially, garbage in –> garbage out.
There is a crack in the armor. It will grow.
The metal is not of the self-mending type.
Kurt in Switzerland
Gavin is right in this case, in the social sciences and maybe also in climate science, most of the calculations are perfect; the assumptions behind the models just don’t remotely apply to the situation being studied. I have always thought Taleb only half knows what he’s talking about.
The Mann hockey stick which used principal components without centering is a notable exception; it is mathematically wrong.
Gavin:”Perfect maths plus bad assumptions still equals BS”
Really…you don’t say. Does that go for Climate Science as well? That is a rhetorical question….
Schmidt showing a shadow of a doubt in the Church of Climatology’s Delphic models? Out, out damn spot! It must be the harbinger of end of days for them …
Pointman
They have lost control over “reality” and can no longer change it?
Wait a minute, is Gavin talking about climate models or social science models? Let’s try not to put words in his mouth that he didn’t say…
REPLY: That is a point, and I dropped the word climate from the title to be fair. But the same issues apply to climate models, perhaps even more so than social models. You have to make assumptions, you have to set starting points. Maths generally can be debugged quickly, assumptions, not so much. – Anthony
Gavin Smith disappoints. — Gavin Smidt
“failure of models to match real world result far more likely a result of erroneous assumptions”
Like, CO2 is the significant factor of global warming?!?!?!!?
This is really irritating how things keep tying together thus necessitating comment
– An excellent book that seems to me to explain the mind of Warmist “True Believers” it’s called On Being Certain : Even If You Are Wrong by Robert Burton
..he explains how our unconsciousnes sis programmed to give us a feeling of certainty (cos that blur over the hill might be a tiger) . So when we OFTEN get the same feeling of absolute certainty when we don’t have full evidence that we get when we
do have certainty.
..AND he then references Fooled by Randomness” by Nassim Taleb
..I guess that would refer to the human habit of seeing patterns in the wallpaper “seeing Elvis’s face etc. when it’s not actually there”
– Now I wonder what kind of people would let that process spillover into their work ?
Does anyone else find it ironic that Gavin Schmidt would not debate John Christy on camera because he does not consider himself an entertainer and yet he tweets all the time to entertain his followers? Maybe ironic is the wrong word… hypocritical?
jfk,
Statistically wrong but not mathematically wrong, after all all the numbers added up the way Mann wanted them, I presume.
Perhaps adding the word “social” to the title? Wouldn’t want the flock to be confused. It appears a couple already are convinced that he was talking about some failure of climate models.
I agree that the title of the post ought to be changed.
Gavin wasn’t [consciously] talking about climate models in their exchange.
But he HAD to realize that the statement could be equally applicable in another discipline!
Here’s hoping that a non-dogmatic individual takes the Hansen-vacated spot at GISS.
Kurt in Switzerland
Steven Devijver says:
June 1, 2013 at 7:47 am
Absolutely! He’s dissing his own ‘discipline’.
I thought our Gav would be one of the last rats to leave. He’s actually jumping quite early. I suppose now that Grandad Hansen has gone … There’ll be a whole flock of lemmings jumping off now.
Jeff Norman – OK, you mean the arithmetic was correct, and maybe it was. I was using the word mathematically in a broader sense. It’s a mathematical error not to center the vectors before orthogonlization when calculating PCs. If a college freshman did this on a test he would probably not get any points.
I think this post should be removed or very clearly put in context. The twitter conversation between Schmidt and Taleb is about social sciences models. Schmidt statement, that perfectly working math models can be totally wrong if the assumptions are wrong, is unquestionable and generally valid, therefore it can’t be framed as a statement about “the state” of modeling, as it doesn’t refer to any current state of any particular modeling but to a general rule. You may think that Schmidt may be hinting at climate models but he doesn’t say it explicitly, so it should be made very clear in the post title and in the reported twitter conversation that he’s not.
REPLY: The same issues applies to climate models. They don’t match real world data, as seen in the IPCC AR5 graph above. Mind you, this isn’t long term modeling failure out a century, this is failure to model the present. Is it bad maths, bad assumptions or both? Dr. Schmidt may not think the issues apply to climate models, or he may be making a mental slip, either way his comment is germane to the current state of climate models and it is a fair question to pursue. – Anthony
Blarney is right. I suggest that this post should just be removed, Schmidt is not saying what you claim he said. Aren’t there enough things to complain about with Gavin without making stuff up?
REPLY: The issue is fair to discuss, since climate models also don’t match real world observations (see AR5 graph), and nothing was “made up”. Though, I did update the post to make it clearer for people who have issues such as yours. – Anthony
I suspect Gavin is campaigning (mostly within NASA) for the GISS Directorship and needs to at least appear Non-Hansenian. If NASA caves to him, Schmidt will be Hansen at Warp 10.
I saw a man pursuing the horizon;
Round and round they sped.
I was disturbed at this;
I accosted the man.
“It is futile,” I said,
“You can never — ”
“You lie,” he cried,
And ran on.
Stephen Crane
Yes, that’s quite refreshing from Dr. Gavin.
The bigger problem is that intuitive criticisms such as this, even when coming from experienced modellers and scientists, amount to little more than “Based on my experience, I think it’s wrong”, which doesn’t carry much currency either way.
A greater attention to resolving lack of predictive skill should have been applied much earlier in the process, ideally before exaggerated claims were made, or scientific cul-de-sacs entered.
Of course Gavin Schmidt is stating the obvious. Has he, at last, realised that the assumption of positive feedback, leading to enhanced water vapour content might be false? Pity he has not listened to those pesky deniers!
No matter how biased toward a particular conclusion one begins, if the data diverges far enough you have to recognize there is a problem. This could really happen to either side of the climate debate going forward. This appears to be a critical inflection point, and the next 20 years will be fascinating to see which way it breaks.
Well, I still think it’s a bit unfair to Gavin (and I am no fan of his). But hey, it’s Anthony’s site.
For a good review of the many failures of statistical modeling in social sciences (and one or two successes) see the book “Statistical Models: Theory and Practice” by David Freedman. Whether or not climate modeling has devolved to the point where it is social science rather than physics, well, I hope it’s not quite that bad…
REPLY: And I think it is more than a bit unfair to us, that if he believes what he tweets, he should re-examine his own assumptions about climate modeling. We have economies, taxes, livelihood, etc. hinging (or perhaps failing) on the success of these models to predict the climate in the future. The models aren’t working, and Dr. Schmidt knows this. Unfortunately his job is tied to the idea that they do in fact work. I feel no regrets at making this comparison front and center. – Anthony
It’s probably good to read a great rendition of the “History of Physical Science” . . . . as it demonstrates that for every answer comes many more questions. Wiki is brief, incomplete; but, a good start. Maybe others can recommend some good reading. I can not site the source of my first reading on “History of Physical Science” as it was long long ago and far far away. Gavin gives me pause to hope!
http://en.wikipedia.org/wiki/History_of_physics#The_physical_sciences