Guest essay by Michael R. Smith, C.C.M.
Forbes, “Absolute Return” column, April 21, 2008, page 246:
Here’s another name you should own, Freddie Mac ($29 per share)…Freddie is cheap at 1.1 times book [value].
Less than five months later, Freddie Mac’s stock was worth 25¢ per share, a loss of 99%. It has since recovered to 70¢ per share, so the loss is “only” 97.6%.
A forecast of a stock of a single company five months into the future seems easy. The company had government backing (federally sponsored corporation). What could go wrong?
Yet, the forecast published by Forbes, short of an outright bankruptcy, could not have been more inaccurate. It is worth examining how a situation that seemed rock solid (government-backed securities!) became catastrophic to see if there are any lessons that might apply to the atmospheric sciences.
The assumptions that Freddie Mac (and other financial stocks) were low risk was primarily a result of computer models. As one expert stated (using pseudonym at http://blogs.zdnet.com/Murphy/?p=1265 ),
The problem is inherently complex – imagine being asked to value a portfolio of 10,000 residential mortgages issued to a total of something like 17,652 individuals. Each mortgage balances some issue amount against some payment stream; each has had zero or more payments recorded against it, each has an initial interest rate; an interest computation method; zero or more early payment opportunities; some mention of late or missed payment penalties and conditions, and an expiry, renegotiation, or call date.
While I do not doubt that is “complex,” the level of complexity is miniscule when compared to the complexity of the earth-atmosphere-ocean system and their interactions. Yet, faith in these model valuations led to a prediction that Freddie Mac stock was “cheap” when a meltdown of the financial system, largely due to the incorrect valuations and risk estimates by computer models, was less than 180 days away.
After the meltdown occurred, a second Forbes article stated, “All existing models for calculating risk, he [Nassim Taleb] says, should be thrown out because they underestimate extreme price swings. ‘The track record of economists in predicting events is monstrously bad,’ he says.” (February 2, 2009, p. 21) Of course, we learn this after our home values and values of our 401K’s are wrecked.
Given the failure of these models to predict the implosion six months hence, would you invest the remainder of your 401K on what the same model predicts for the next six years or, if you are in your 20’s, what it predicts for sixty? I don’t know what your answer might be, but common sense would indicate applying the forecasts from these models to your portfolio with extreme caution.
June 1, 2009, we learned from The New York Times that “Models’ Projections for Flu Miss Mark by Wide Margin.” The model predicted, according to the Times, “by the end of May, there would only be 2,000 to 2,500 cases in the United States… On May 15, the Centers for Disease Control and Prevention estimated there were upwards of 100,000 cases in the country…”
Just six months earlier, the models’ predictive capability were touted because of real time input from Google (www.cidrap.umn.edu/cidrap/content/influenza/panflu/news/nov1308google-jw.html ). Now, the flu has been declared a “Pandemic” by the World Health Organization (/www.pandemicflu.gov/ ) in spite of the modest number of cases projected to be in existence by June, 2009, by the models. Another critical short-term modeling failure.
Question: If the model predicts low risk for the next six months, would you decide to forego a flu shot? Again, your answer might be different, but common sense would dictate getting the shot.
How do these examples relate to climate modeling and policy?
We currently have climate models that have missed the fact that atmospheric temperatures peaked 11 years ago and that oceanic heat content has, at best, failed to increase. See: http://climatesci.org/2009/03/04/large-uncertainty-in-the-simulation-of-the-global-average-surface-temperature-by-the-ipcc-models-a-study-reported-on-the-weblog-the-blackboard/ , http://climatesci.org/2009/02/09/update-on-a-comparison-of-upper-ocean-heat-content-changes-with-the-giss-model-predictions/ , among many others.
Given the inadequate performance of these models over the last 5 to 10 years, why do we believe we can make accurate, highly specific forecasts 50 to 100 years in the future? Is it because we are so close to the problem we are blinded to the dangers like the economists who did not see the meltdown coming?
Almost no one familiar with meteorology or climate models would disagree that they are more complex than the mortgage valuation or influenza prediction models. The basic processes of the earth-ocean-atmosphere are incompletely understood and we barely understand many of their interactions.
We also know that forecasting the weather beyond five days is dicey at best. Then why are we making 29,000-day weather forecasts? Don’t think we are doing that? Consider the following:
“By the period 2080-2099, devastating heat waves of the kind that killed more than 700 people in Chicago in 1995 will occur three times per year.” (USCCP, p. 119, citation below)
That is a weather forecast – a forecast of specific meteorological conditions at a specific time and place. The document is filled with similar predictions, along with recommendations based on those predictions.
We are sometimes told that climate forecasts can be made because the “weather” errors will be cancelled out because they are “random.” Here is what was said about the mortgage computer models,
Now, because you can predict roughly the probable range for most of these assumptions but not the actual values the variables involved will have for each of the time periods you have to consider, what you do is write a monte carlo simulation in which you try tens of thousands of value combinations and plot the results to see what, on average expectations, the portfolio might be worth.
Notice, that at this point even something as large as 0.0005% error in the outcome would be completely insignificant – so randomization error should have no effect, right? (op. sit.)
It was believed by most the mortgage instruments were safe because the errors (i.e., a higher default rate of subprime lenders) would cancel out (because the risks were spread) and because, if desired, default insurance could be purchased from institutions like AIG. Of course, AIG used similar models to determine its risk. We just learned how well that worked.
In spite of these spectacular failures of less complex computer modeling in economics and public health, the atmospheric sciences seem to be making similar miscalculations. If your common sense would lead you to disregard these models’ forecasts when planning your portfolio and whether you get a flu shot, I would suggest we adopt a much more modest approach to the use of climate models. While they are useful research tools, the numerous uncertainties (cloud feedback, particulates, volcanic ash, the current quiet sun, etc.) are so great we cannot claim to have forecast skill decades into the future.
Otherwise, when I read, during a period of falling temperatures and ocean heat content,“Global warming is unequivocal,”* I hear, “Freddie Mac is cheap.”
- U.S. Climate Change Program, Key Finding #1, January, 2009, http://downloads.climatescience.gov/sap/usp/prd2/usp-prd-executive-summary.pdf
Michael R. Smith is CEO of WeatherData Services, Inc., An AccuWeather Company, and a Fellow of the American Meteorological Society. This weblog represents his personal opinion. AccuWeather’s Global Warming Blog can be accessed at: http://global-warming.accuweather.com/ .
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
“O/T. Space site has sunspots 12 but says on picture caption that sun is blank i.e.: spotless. What’s up with that?”
You aren’t the only one confused:
sc24.com: “The sun received a sunspot number of 12 today as Sunspot 1020 re-appeared. I was at work all day long and it looks like I missed the 2 small speck show. I dont know how it received a sunspot number, but it did.”
This isn’t really OT, because as you know all the ‘experts’ have been predicting a massive sc24 and now that this isn’t happening they are claiming it will be near average. So, to keep their prediction on target, the spots must follow the curve! The prediction graph indicates and uptick must start now, so they number every spot regardless of how long it lasted. But hey, at least if there is a Dalton minimum repeat we will know that even a cycle that has been recorded as “average” can produce the same phenomena experienced 200 years ago! 😉
“After the meltdown occurred, a second Forbes article stated, “All existing models for calculating risk, he [Nassim Taleb] says, should be thrown out because they underestimate extreme price swings. ‘The track record of economists in predicting events is monstrously bad,’ he says.” (February 2, 2009, p. 21) Of course, we learn this after our home values and values of our 401K’s are wrecked.”
Michael Smith is unfair to Nassim Taleb. He has been pointing out the perils of prediction for some years. His book “Fooled by Randomness” was published in 2004 and “The Black Swan” was published in 2007.
A major problem for our society is the inappropriate use of models in many fields. For example in my small home town a planning application has recently been submitted for a new supermarket. The town is notorious for its traffic delays and tailbacks during peak hours. Typical queue lengths at junctions are 60 to 70 cars. The supermarket has employed transport consultants to analyse traffic conditions. The consultants carried out a simple traffic count but did not measure queue lengths. They fed the data into ARCADY, a reputable model developed by the UK Transport Research Laboratory, and concluded that maximum queue lengths during peak hour are 2.65 vehicles long. This is a dramatic error!. The consultants, who clearly have failed to visit the town, have concluded that there is virtually zero queuing and ample capacity for more development.
In the British armed forces there is a saying, “Bullshit Baffles Brains”. The public official charged with assessing the consultants’ hugely flawed analysis has written: “The Marlow Society base their objection on what is considered to be a flawed computer analysis. It must be remembered that Arcady is only a tool and cannot model how people actually drive. Therefore the figures [that] are produced are theoretical and show the delays that would occur if all available road space was used. The important thing is that the increase in queue lengths, no matter how they are derived, is minimal. I accept Mr Post’s comments on the existing queue lengths but I would suggest this is due to the way people drive rather than a lack of capacity.”
So, if you visit our beautiful riverside town ever, be warned, Marlow-on-Thames drivers drive differently and fail to use all their available road space!
Forgive this off-climate deviation, but it seems to me that models, or more importantly, model users are a huge problem in all areas of society from climate change to pensions. Nassim Taleb is a prophet who should be lauded. If Michael Smith had thought to read Taleb before the recent financial meltdown, he would have been forewarned. As Yogi Berra famously said: “Prediction is very hard, especially about the future.”
When I am told by ‘believers’, of these predictions made from computer models, I ask them to imagine themselves in Edwardian England on a lovely summers day in May 1914 and please outline history to the year 2000. No one is game to make such a prediction although all the data is there. How can you predict any chaotic system to be modelled accurately ?
tom (23:02:43) :
I like the choice of words. What do “adults” think will happen?
What does this have to do with lifestyle? I actually have curiosity about what will happen in the future. Oh wait, if the climate is cooling does that mean we won’t be able to apply for grants?
All real scientists know that it is futile to model multi-dimensional chaotic systems. like our climate, and expect to get accurate forecasts of future trends. However, use of models which illustrate the outcome you want to see is a very effective propaganda tool for scaring people into accepting a political agenda which you wish to pursue.
I think that many climate modellers are similar to astrologers – they both use pseudo-scientific mumbo-jumbo to produce their forecasts, which tell their customers what they want to hear.
Has there EVER been a simulation model of a complex system which was long term accurate?
I doubt it, because you (1) you never each aspect of the system completely accurately — there is always a little uncertainty; and (2) such uncertainties tend to multiply and grow over time.
If a butterfly in China can cause a storm in New York, why can’t that storm in New York cause a Little Icea Age?
Excellent reality check. Thank you.
All Earth Science models are subject to the non-uniqueness principle. It’s not uncommon for multiple models to explain a particular set of observations equally well.
In the Gulf of Mexico, “bright spots” – seismic amplitude anomalies – are often associated with hydrocarbon accumulations. But, there are a lot of things besides hydrocarbons that can cause bright spots. Models are helpful; but they are far from conclusive. It’s very easy to build a model that predicts that bright spots are indicative of pay in sands where pay has no bright spot signature. These models, though honestly and competently constructed, are fatally flawed because they are predicting something that is highly unlikely and usually could have been ruled out by analogy.
“Well, to be fair, the models were assuming “normal” market conditions. What we had was a system greatly skewed by government manipulation. How would climate models hold up in the face of some sort of intentional manipulation by human beings?”
And this is characteristic of the failure of many (most?) modeling systems – they can’t accommodate the anomaly that places data outside the errors bars and yet cannot be removed as an outlier because the entire system has been perturbed by its presence.
Decades ago, Sir McFarlane Burnet was quoted as saying “the only thing you can be sure about in Biology, is that you can’t be sure about anything.” His reference spoke to the inability in biological modeling systems to both fully identify and control variables, most especially will. Chaos rules, and until you can predictably model unpredictable chaotic events, models will never accurately predict beyond the pre-pertubation ground state. Stated another way, the only thing they have some ability to do is model the inter-perturbation period.
Climate models aren’t predicting anything. They only project a time-linear consequence based on some of yesterday’s information. This is nothing more than navel-gazing raised to a high art.
From Junk Science
UAH MSU +0.05 °C. Rank: 16/31
Warmest May in this series was in 1998.
Average last 12 months: 0.14 °C.
GISTEMP +0.55 °C. Rank: 5/130
Warmest May in this series was in 1998.
Average last 12 months: 0.49 °C.
RSS MSU +0.09 °C. Rank: 16/31
Warmest May in this series was in 1998.
Average last 12 months: 0.18 °C.
http://junkscience.com/MSU_Temps/Warming_Look.html
Still waiting for the HadCRUT figure
Common sense is far too complex a thing to reproduce in a computer model so the results of computer models have to go through our own common sense filter. Problem is its an individual thing which can not be described or justified on paper so is often ignored by large organisations.
I predict that someone will predict our predictive model success rate to improve by 99.9% when the first projected prediction comes true.
Crosspatch – re your (21:11:17)
Very many thanks for that superbly clear explanation of how the crash came about. With your permission, I’d love to “cut and paste” it to a whole bunch of friends, some of whom still believe it was all down to greedy bankers. The fact that the party that was in power in the US when the whole house of cards was set up, is back with both Houses and the Presidency wrapped up should be worrying a lot of people.
Tim Bromige
“observer (20:41:02) :
O/T. Space site has sunspots 12 but says on picture caption that sun is blank i.e.: spotless. What’s up with that?”
It sublimated. It was predicted, so therefore space must be filled.
I cannot find a GONG, SOHO or UCCLE image of it.
Nobody drew it (Mt. Wilson, Catania, Uccle)
These spot phantoms keep popping up, like failed financials.
There are millions of pores on the Sun, and you can always count on one of them to be your ‘sucker born every minute’.
So, you can fake spots all day long by counting pores, but when you go outside tomorrow and project the Sun, you will see the reality of the situation. You cannot hide the Sun, it’s an Astronomical object with Equal Opportunity, anymore than you can fake the catastrophic Ocean rise.
Credit DeSpot Swap.
Fannie and Freddie were known to be tipping into failure. Congress was warned about it and disregarded the warning.
The economist like Peter Schiff, Marc Faber, Jim Rogers that predicted this correctly made a fortune shorting the financials.
Of course it is true that the consensus of economist missed the whole meltdown and now they are wrong again in predicting a quick recovery.
Here is Congress being warned about Fannie and Freddie.
In response they berate the person correctly predicting the coming failure.
http://blog.beliefnet.com/reformedchicksblabbing/2008/09/maxine-waters-we-do-not-have-a.html
Peter Schiff was right.
philincalifornia (23:43:52) :
Remember this one from the G-8 global warming deal. Oh how we laughed:
“Merkel and Blair also suggested a target of keeping global temperature increases to less than 2 degrees Celsius by 2050, which is not part of the deal.”
Armed with this new insight, the Global Thermostat is just months from reality:
http://www.physorg.com/news163861421.html
Nasif Nahle (22:24:48) :
Seriously, I don’t know why the WHO dictated that flu is now a pandemic
You are a biologist and you know what you say on this issue, so:Are they making a prediction as the other UN agency IPCC?
“Unless we announce disasters no one will listen.”
– Sir John Houghton,
“We need to get some broad based support, to capture the public’s imagination…So we have to offer up scary scenarios, make simplified, dramatic statements and make little mention of any doubts…
Each of us has to decide what the right balance is between being effective and being honest.”
– Prof. Stephen Schneider, Stanford Professor of Climatology,
lead author of many IPCC reports
http://www.green-agenda.com/
Another failed alarmist prediction:
http://blogs.news.com.au/dailytelegraph/timblair/index.php/dailytelegraph/comments/rain_denied/
Tim Blair
Saturday, June 13, 2009 at 04:57am
A Tim Flannery prediction from 2008:
The water problem is so severe for Adelaide that it may run out of water by early 2009.
We’ve hit 2009’s midway point, and Adelaide’s water supply is currently at 54 per cent of capacity. That’s more than 100,000 megalitres, with yet more to come:
South Australia is in for a wet weekend, including possible floods, as significant rainfall is forecast in all districts south of Port Augusta.
Up to 30mm of rain could fall in southern agricultural areas, including Adelaide, but up to 50mm is forecast in the Mount Lofty and Southern Flinders Ranges by the end of Tuesday.
Flannery’s been blown out of the water, so to speak, as is usual when he claims Australian cities are about to run dry. So how does Flannery respond?
Australian scientist and campaigner Tim Flannery, one of the conference organisers, said climate change was harming his home country. “Water resources have dried out to the point where they’re now affecting the future of some of our cities.”
Flannery is a rain denialist.
UPDATE. Flannery addresses the “carbon imbalance of the planet”:
Look at the link to see his BS speech.
This is the sort of speech for which Flannery once claimed to charge $50,000.
My pastor told a c-store clerk when gas was 4 dollars last july, it would be below 2 dollars before Christmas. The clerk said it would be 6 dollars at Christmas. He also on Wednesday night said fear was of the Devil in reference to global warming. He is a degreed engineer.
We have what he said last July on some audio tapes.
I of course said almost the same for 3 different reasons.
We had 3,500 oil tankers offshore waiting for crude to go up a dollar a day. They hustled to unload during hurricane season. Inventory figures were low because so much oil was not offloaded.
I also had learned some short positions that speculators had and knew it was a price inflated by speculation.
This is a great site and facts really take away emotionally jacked up fear. I sat in Mobils internation exploration conference room and was told in 1981 that we had 12 years oil left. We can’t do without oil consumption so that is qwhy they tax it. The excuse they give is it hurts our health.
Excellent article. I too am puzzled by the “random weather events will even out over time” shtick. Surely weather events are not “random” at all, no matter how much it seems that way to the average Joe. Things seem random when we don’t understand the way they work. If we understood weather enough to predict what it would be like in 2100, I don’t think it’s too much to ask that we be notified where and when hurricanes, tornadoes and severe thunderstorms will be forming in the next year. If we can’t even predict those sorts of things — at all! — how can we say temperatures will increase by 2 degrees in the next 100 years?
How many hits on that do you think they’ll get from WUWT readers?
Lots… I click on the WeCanSolveIt button every time… The irony that Al Gore & his band of believers could be paying Anthony is too delicious to pass up…
REPLY: Thanks for the thought, there’s no need. Unless you clear your cookies, or have them set not to accept, it counts as a re-view, not a fresh one. I think the cookies are 24 hours to expire. So a click a day, that’s all I ask. – Anthony
don’t tarp me bro (07:36:40) :
My pastor told a c-store clerk when gas was 4 dollars last july, it would be below 2 dollars before Christmas. The clerk said it would be 6 dollars at Christmas. He also on Wednesday night said fear was of the Devil in reference to global warming. He is a degreed engineer
———————-
Number of protons in carbon 6
Number of neutrons in carbon 6
Number of electrons in carbon 6
666 is the number of the beast, and he’s infiltrated all of us sinners !!
Trouble arises when there is the need to make predictions become true no matter how hard it is, and finally arriving to the conclusion that, at the end of the day, what matters is that at least a “paradigmatic reality” is created.
So, for example, “global warming” has life of its own, and as such a hundred or a thousand of blogs like WUWT won´t make it disappear.
[snip – no references to WWII propagandists, I’ve warned you about this before- Anthony]
As a layman, I wondered about the distinction between “weather” and “climate”. I was told that the reason global warming can be predicted is because “climate” is not the same as weather. As a layman I would be interested to hear the evidence for this, because as far as I can see, that distinction is just completely made up.
Joe Romm’s Climate Progress has a guest blogger talking about by 2050 he will be retiring underwater or on fire. When I called on this, all the other commenters agreed.
http://climateprogress.org/2009/06/12/after-bonn-a-safe-future-for-youth-still-in-doubt/