This article was sent to me by reader Peter Yodis. I found it interesting and germane to current events, so I’m sharing it here. Just a note for clarification, the very last sentence is in his original article, it is not commentary from me. – Anthony
Reposted from Machine Design.com from editor Leland E. Teschler Feb 17th, 2009
Amid all the hand-wringing about financial systems in meltdown mode, the subject of modeling hasn’t gotten a lot of notice. Banks and other financial institutions employed legions of Ph.D. mathematicians and statistics specialists to model the risks those firms were assuming under a variety of scenarios. The point was to avoid taking on obligations that could put the company under.
Judging by the calamity we are now living through, one would have to say those models failed miserably. They did so despite the best efforts of numerous professionals, all highly paid and with a lot of intellectual horsepower, employed specifically to head off such catastrophes.
What went wrong with the modeling? That’s a subject of keen interest to engineers who must model the behavior and risks of their own complicated systems. Insights about problems with the mathematics behind financial systems come from Huybert Groenendaal, whose Ph.D. is in modeling the spread of diseases. Groenendaal is a partner and senior risk analyst with Vose Consulting LLC in Boulder, a firm that works with a wide variety of banks and other companies trying to mitigate risks.
“In risk modeling, you use a lot of statistics because you want to learn from the past,” says Groenendaal. “That’s good if the past is like the future, but in that sense you could be getting a false sense of security.”
That sense of security plays directly into what happened with banks and financial instruments based on mortgages. “It gets back to the use of historical data,” says Groenendaal. “One critical assumption people had to make was that the past could predict the future. I believe in the case of mortgage products, there was too much faith in the idea that past trends would hold.”
Therein lies a lesson. “In our experience, people have excessive confidence in their historical data. That problem isn’t unique to the financial area,” says Groenendaal. “You must be cynical and open to the idea that this time, the world could change. When we work with people on models, we warn them that models are just tools. You have to think about the assumptions you make. Models can help you make better decisions, but you must remain skeptical.”
Did the quantitative analysts who came up with ineffective financial models lose their jobs in the aftermath? Groenendaal just laughs at this idea. “I have a feeling they will do fine. If you are a bank and you fire your whole risk-analysis department, I don’t think that would be viewed positively,” he says.
Interestingly enough, Groenendaal suggests skepticism is also in order for an equally controversial area of modeling: climate change.
“Climate change is similar to financial markets in that you can’t run experiments with it as you might when you are formulating theories in physics. That means your skepticism should go up,” he says.
We might add there is one other similarity he didn’t mention: It is doubtful anyone was ever fired for screwing up a climate model.
And yet the industry ploughs on with the forthcoming movie “The Age of Stupidity”
http://www.ageofstupid.net/
Dave, the problem is not the horsepower, the problem is the lack of a fundamental understanding of the systems involved. More horsepower will only produce the same answers more quickly! If one knew exactly how the systems really worked, then I would be willing to bet that my laptop would be sufficient horsepower. I am a little tired of the notion that more power makes a smarter computer. It all comes back to GIGO. Faster GIGO is just faster GIGO.
According to Moody’s President Ray McDaniels:
The models were based on the assumption that:
Homeowners would default on unsecured debt such as credit cards or auto loans before defaulting on their homes. This time, they protected the plastic and the cars and let the homes go.
In my paraphrase: “The models were based on the notion that people would screw the other guy before they screwed their mortgage companies.”
Now if that’s your business model, I would think that you are living on borrowed time….
Link to Moody’s quote:
http://globaleconomicanalysis.blogspot.com/2008/02/us-homeowners-confound-predictions.html
I somewhat agree with you here. Although I am a big proponent to using the right tool for the job, and I am not necessarily discounting Fortran, but I can’t help feeling that there are more advanced OO languages that may be better suited to the development of GCM’s (and perhaps financial models as well). Even with the latest morphing of Fortran towards OO (loosely), OO is still not its strength. I also believe its foundational construct and architecture is a bit dated. I presently develop application framework models, and I know what I am doing now would be difficult, if not impossible, to accomplish within a language such as Fortran. Now, that is not to say that the languages I am primarily dealing with at present, would necessarily be well suited to GCM’s either, but my gut tells me that they may be better suited than Fortran. I have been evaluating ModelE for a brief time now, and I just don’t see how Fortran fits well with that model at all. ModelE appears to be very clunky (besides poorly written) and cumbersome to me and I believe at least a significant portion of that can be attributed to the languages in use. Keep in mind, this is not a very educated opinion at this point, merely an educated guess from a fair amount of experience.
Oh rats, it didn’t work. I was quoting this:
Ron de Haan (12:38:54) :
“In Australia, the model based Global Warming Scare has resulted in the inevitable.
“A SKEPTIC POLITICAL PARTY to confront the political establishment from within.
http://heliogenic.blogspot.com/2009/02/new-political-party-in-australia.html
“With 90% of the population denying a human link to Global Warming this party has good perspectives to become a succes.
“This is a great idea for the USA to. Anthony for President?”
No preview function here, unfortunately.
/Mr Lynn
Having personally seen Risk Management Model at work in the chip industry, and watching in horror as the Financial Industry used it to stick their necks out of a speeding bullet train, I come to the following conclusion about it:
It is used to sweep as much risk under the rug as concienable. If there is something that can be done to mitigate risk, it will be disregarded as costing too much money. If there is nothing that can be done about the suicidal nature of what the company wants, then they will go ahead patting themselves on the back for having discussed it under the rug.
It’s just a dangerous tool to be using when lives and property are at risk.
I fear it is at work in climate modeling.
The model that preceeded it was sanity and common sense.
I knew it as Heinrich’s triangle.
It say that if you are having incidents, you are at risk for loss of life and/or property far beyond mere incidents.
For a solar cycle to be this late, the flux this quiet, the solar wind this backed off and the neutron count this high, the sunspots being this weak and losing contrast, it means that preparations should be in order for something about to snap.
We cannot control the Sun, but we can make preparations for sharp & marked climate change.
Just as in Heinrich’s Triangle, the biggie can come the first time you walk under the suspended load, or the 333rd time you walk under it.
You know that the numbers will eventually find you; you don’t know how sooner or later that will be. Ergo, we prepare ourselves.
Risk Management, on the other hand, is like playing deer in the headlights of an oncoming vehicle. No preparations are made, only marvelling at the strange incidents taking place.
Ergo the animal is wasted in it’s tracks.
The comments that connect CAGW cap and trade costs and energy costs with the financial disaster are probably well founded. What would all forms of energy cost if we had instead invested in more efficient and clean nuclear and fossil fuel technology? Would there be a water problem if we had abundant energy to desalinize sea water? What is the economic benefit of all crops increasing yield 15% due to increased CO2? In the two years that gas was very expensive the US alone spent over 700 billion more than necessary for energy.
Energy is the life blood of any economy. We are cutting off our blood supply at the worst possible time.
On a side note, a politician blaming capitalism for not regulating the banks is the epitome of hypocritical. The two worse institutions (Fannie Mae & Freddie Mac) are government sponsored institutions directly regulated by the senate finance committee. Now they are getting an additional 400 billion dollars from the government that started them.
Models are like the human capacity to reason. “You can reason yourself into or out of anything.
Fortran is a perfectly wonderful language for developing complex simulations, especially when they involve copious amounts of linear algebra routines like I would imagine atmospheric models do. I would trust the old linpack and eispack FORTRAN routines more than the C and C++ ports of them if I was developing the code. In fact, if you look into it in any detail you’ll be disappointed with the available math and statistics routines available in C, C++, C# and Java versus good FORTRAN. Personally, I think it has to do with FORTRAN’s inclusion of an intrinsic complex variable type, but I digress …
I’ll try to keep this short … I have a fair amount of experience in developing simulations and models of complex physical systems. My stomach turns when I hear these guys predict the earth’s temperature “N” years in the future. What a complete crock and I can’t wait to see how inaccurate they were. Don’t get me wrong, I strongly support the development of atmospheric models and it is great stuff but I think the ability to accurately model the time evolution of such a complex system is way out in the future if ever. Gas dynamics, fluid dynamics and heat and mass transfer are incredibly complex topics by themselves let alone folding in electromagnetic radiation and propagation models. Belief in these simulations’ predictions out multiple days, let alone multiple years is sheer madness.
I just laugh when I hear a politician or a biologist or ecologist tell us how real man-made global warming is. Ultimately, what these guys are doing is looking at simple data correlations and their predictions are no different than predicting the future share price of a company by viewing charts of its historical price. You can fit a mathematical model to a chart’s history and it might fit for some small time steps in the future, but soon you will be SOL because an event will occur that wasn’t in your model. The global warming doom and gloom scientists like that Hansen fellow are nothing but pure snake oil salesmen.
Anybody wanna guess where the term “horsepower” came from? There is evidence to suggest that what James Watt originally termed as “horsepower” was based on what a pony could pull X 50. Ponys were the original “little red steam engine that could” in mines. And many farms. Draft horses, the big kind, only came along after big bucks were made. Kind of like the difference between my little red putt-putt and the big-time farmer down the road with a blue tractor that has as many wheels as a semi truck, and as big as the cab.
I don’t believe there was any sort of “conspiracy” involved here (aside from Bernard Madoff, Robert Allen, etc., and others yet unknown). There are several Psychological driving forces that prompted the actions and continued behavior of those involved. A few that come to mind might be greed, euphoria, denial. And once you pack those into a collective reasoning environment, you really begin to develop a behavior almost akin to being drunk. The answer is, they simply didn’t care, they didn’t think it would collapse, they thought they could profit virtually forever, they listened to and paid attention to those things that most strongly supported their ambition (much like AGW). This is nothing new, this is human nature. The people involved were in positions where they could steer the ship where they wanted. They felt they had all of the control and dismissed any information to the contrary as it did not fit their ambition. Again, this is nothing new, everyone has done this at one time or another. Its just a matter of scale. When you have people profiting millions or even billions of dollars, that is a pretty powerful drug and has historically and does presently, drive people to do some pretty radical things. Explanation of this really is a “no-brainer” (after the fact of course). I think you said it best with ” .. in his right mind .. “
Awhile ago, I pulled apart the components of GISS’s Model-E and then extended the forecast it would have provided from 2003 (the end date of the data provided by GISS) to 2013, ten years.
The model would be off by about 0.15C in the first five years.
The more detailed version of this extension is here:
http://img175.imageshack.us/img175/2107/modeleextraev0.png
The simpler version.
http://img135.imageshack.us/img135/8594/modeleextend2013gi9.png
Another way to look at is they have huge GHG temperature impacts built in (no way to get to +3.0C without it) but they need to build in almost as big negative temperature impacts from other sources to keep the hindcast close to the actual temperatures we have seen so far.
One could conclude they are just plugging the big negative numbers into the hindcast after the fact to make it work.
Which is close to the point Leland Teschler was trying to make in this article.
http://img183.imageshack.us/img183/6131/modeleghgvsotherbc9.png
Without a large uptick in temperatures in the next few years, the modelers really have to go back to the drawing board (or they need to discover another “negative forcing” to keep the models on track to reality).
Wow! Very well said! I concur…
Wow, no ego there… And I couldn’t disagree with him (Gavin) more.
I agree with almost everything David Holliday (20:02:43) : said. Computers do not have any intelligence and a superfast human could do everything a computer could do. However, there are no superfast humans, so in reality computers can do many things us poor slow humans could never do or would even attempt to do.
I don’t think it will go to insulating your house, I believe it will go to insulating AGW from the heretic’s and deniers. I agree that he is probably fencing this one for a second term, but in the meantime, he needs to “keep the dream alive”, so of course he’s gotta shell out a little to keep the AGW fires fueled. Perhaps this will give us the necessary time to extinguish? Let us hope so…
Robert, you forgot to say sarc off at the end of your post. On the other hand, your post would make for a really cool movie. It would make a great movie about the Earth suddenly freezing its butt off.
sarc off
Predictions based on too much statistics is bad science. You may predict the climate this way or the solar activity, etc, and you might have a good chance of being right, but it doesn’t really give any new insight. Good science is prediction based on an understanding of the physics behind what you want to predict. The use of statistics is a great tool to identify the physical conditions, but it should be nothing more than that.
For Ron:
Yes, Ed Lorentz poked holes in a lot of weather modeling. Nova produced an excellent program on chaos years ago where Ed and many other pioneers on chaos theory talked a lot about sensitivity to initial conditions and other related subjects. I wish they would do a follow on show.
I don’t believe that most of the intellectual world is yet ready to try and understand the implications chaos theory puts on their models. I rest my case with the financial models and climate change predictions. Even this blog represents a lot of discussion about climate that is based on ignorance. For instance, the causes of natural variations in climate change are not well understood. Is it the sunspots, or lack there of? Is it the increased cloud cover with increased cosmic rays? Do increased flares from the sun heat the earths atmosphere? Does the sun drive the ocean oscillations? Let us not forget the Milankovitch cycles. Once these are at least partially understood, then how do they interact with each other? Are there tipping points we should be aware of on the cooling side of climate?
Climate is an elephant and people with a lot of little letters behind their name are experts in one toenail of the elephant. But they like to believe they know more. I believe they don’t study chaos theory or cybernetics because it makes them feel insecure when they realize it may be impossible to obtain enough timely data to use their statistics to make a prediction.
Think of all the historical data David Hathaway and group have about the sun, and look how pathetic their predictions on the start of cycle 24 have been. They fail to realize that chaos means randomness bounded by attractors. I believe they don’t even know what the attractor for sunspot cycles is. They may not even know what a strange attractor is.
I bet Ron or I cannot even bring anyone else here to discuss the chaotic nature of climate.
Dennis Sharp
David Halliday
“One of the biggest misnomers in Computer Science is Artificial Intelligence. There is no intelligence in a computer. And we’ve never been able to put it in there.”
We must have taken different classes in AI, then. Mine was from UCLA where the instructor wrote the AI for NASA’s Mars rovers. AI definitely exists, and I stand completely by my earlier assertions.
But I will not get further into a Did so! Did Not! contest, as it is fruitless and a waste of Anthony’s and moderators’ time.
I would agree that there are “some” things that computers can do that humans cannot. Computational speed is perhaps one, but that is just about where it ends. I have studied AI for quite some time and it was my primary collegiate focus, and I too play with neural networks from time to time just for fun. But the human brain by contrast, can perform many things that computers presently cannot do and some things that they may never do. One very humanly simple thing that computers are extremely poor at is pattern recognition. Humans process patterns with astounding accuracy and at an astounding rate. As a very simple example of this, I was recently sent an email from a colleage, the special thing about it was that the letters were all jumbled up. All words were written with the proper beginning and ending letters, had the proper number of characters and the correct characters as a whole, but, all inner letters were out of order. The interesting part of this is that you can read it almost as easily as you read anything else. As long as the words contain the correct letters, length and beginning and ending letters, it doesn’t matter. Your brain automatically compensates on the fly through pattern recognition. Its an interesting experiment and one that you can easily try for yourself. Now, one would say “so, a computer can do that”, yes, but through iteration and rearranging, not through first take pattern recognition, and certainly not with the efficiency of the human brain. And as for “fly-by-wire”, your brain handles more fly-by-wire than our entire fleet of Stealth bombers combined, every moment of your life, monitoring thoughts, temperature, body functions, heartbeats, internal clock, circulation systems, neural systems, and on and on, all in real-time. That’s pretty tough to beat. We may get close someday in the future, but for now, not even close.
As a tiny example, that goes to this and the model topic, have you seen and heard, even a short film that was completely computer generated, that you could not discern from reality? And that is the simple stuff.
Unfortunately, we still have actors (politician interchangeable)…
Suggestion; if you create an empty .htm file, using Notepad or Wordpad [for example] text editor,
then open that file using EDIT [right click],
PASTE your desired quote,
add your tags as needed, SAVE.
Then OPEN the file normally by clicking on it.
It will open in your browser and give you some idea of how your submission will look.
Not perfect, but works for me. Others may have better or more complete suggestions for a preview.
Robert Bateman. Always enjoy reading your posts! Most of you guys. It is so educational for a layman like me.
Begs the question: Why are some of these brilliant minds posting on this blog…not in charge on this planet for the direction of science?
Why is it still in the hands of the the megalomaniac Hansens, Holdrens, Gores, and other morons of our truncated politically-driven pseudo-science universe?
I understand we are in a continual “Randian” struggle and Atlas is shrugging a lot lately…but GEEZ.
Why must we CONTINUE to be held captive by bureaucrat-ideologues for centuries and centuries to come?
Gallileo is no doubt turning in his grave.
It is a shame…because the controversy of it all TAKES ON A LIFE OF ITS OWN and the noise from THAT…simply drowns out the scientific method.
On a grander scale, whether we are doomed by fire or ice or something in between…could be considered immaterial. (Natural selection at work!).
The REAL tragedy is this: Political ideologues who are polluting science and guaranteeing the human race to a slow, painful, and scientifically backward…demise.
Exempli gratia: WHY should it take the deaths of hundreds of Australian artists and thinkers who were forbidden to clear fire-lines around their homes…to prove this point??
But I digress….
By in large I agree with you. There seems to be a popular misconception that I think has been largely fueled by Hollywood. Computers cannot do the things you see on the big screen. Unfortunately, even my father suffers from this misconception, and he is a retired engineer from MIT! And the worst part is that he is eating up AGW like there is no tomorrow. I fight with him on the AGW subject daily.
BTW, to all, yes, AGW is most certainly a religion. I have seen this transformation in my father, and it is rather disturbing. I would never have guessed that I would be seeing this behavior from my father, but he’s clearly had too much kool-aid. I’ve always viewed him as perhaps the most rational and objective person I have known, but wow, not when it comes to AGW. It is some scary stuff!