This article was sent to me by reader Peter Yodis. I found it interesting and germane to current events, so I’m sharing it here. Just a note for clarification, the very last sentence is in his original article, it is not commentary from me. – Anthony
Reposted from Machine Design.com from editor Leland E. Teschler Feb 17th, 2009
Amid all the hand-wringing about financial systems in meltdown mode, the subject of modeling hasn’t gotten a lot of notice. Banks and other financial institutions employed legions of Ph.D. mathematicians and statistics specialists to model the risks those firms were assuming under a variety of scenarios. The point was to avoid taking on obligations that could put the company under.
Judging by the calamity we are now living through, one would have to say those models failed miserably. They did so despite the best efforts of numerous professionals, all highly paid and with a lot of intellectual horsepower, employed specifically to head off such catastrophes.
What went wrong with the modeling? That’s a subject of keen interest to engineers who must model the behavior and risks of their own complicated systems. Insights about problems with the mathematics behind financial systems come from Huybert Groenendaal, whose Ph.D. is in modeling the spread of diseases. Groenendaal is a partner and senior risk analyst with Vose Consulting LLC in Boulder, a firm that works with a wide variety of banks and other companies trying to mitigate risks.
“In risk modeling, you use a lot of statistics because you want to learn from the past,” says Groenendaal. “That’s good if the past is like the future, but in that sense you could be getting a false sense of security.”
That sense of security plays directly into what happened with banks and financial instruments based on mortgages. “It gets back to the use of historical data,” says Groenendaal. “One critical assumption people had to make was that the past could predict the future. I believe in the case of mortgage products, there was too much faith in the idea that past trends would hold.”
Therein lies a lesson. “In our experience, people have excessive confidence in their historical data. That problem isn’t unique to the financial area,” says Groenendaal. “You must be cynical and open to the idea that this time, the world could change. When we work with people on models, we warn them that models are just tools. You have to think about the assumptions you make. Models can help you make better decisions, but you must remain skeptical.”
Did the quantitative analysts who came up with ineffective financial models lose their jobs in the aftermath? Groenendaal just laughs at this idea. “I have a feeling they will do fine. If you are a bank and you fire your whole risk-analysis department, I don’t think that would be viewed positively,” he says.
Interestingly enough, Groenendaal suggests skepticism is also in order for an equally controversial area of modeling: climate change.
“Climate change is similar to financial markets in that you can’t run experiments with it as you might when you are formulating theories in physics. That means your skepticism should go up,” he says.
We might add there is one other similarity he didn’t mention: It is doubtful anyone was ever fired for screwing up a climate model.
A couple of points on the origins of the financial meltdown — slightly OT.
Years ago, a bank or other lending institution held the loans it made until maturity. The bank made money by a small upfront fee, and by the interest collected over the life of the loan. More recently, banks (as others above have pointed out) would sell the loan to obtain cash in hand in order to make more loans at a faster pace than simply waiting for the monthly payments with interest to come in.
The lenders made money by charging points and other fees on each transaction. The more loans that were made in a given month, the more money the bank made from those points and fees.
Where things began to unravel was derivatives. A mortgage loan, or more accurately, a package of mortgage loans, was sliced up into tranches, each paying a different interest rate, from high to low. Investors could then buy the tranche that suited their appetite for risk/return. The low-risk tranche may have paid as little as 5 percent on a loan that was made at 8 percent interest rate. The highest risk tranche may have paid 12 percent interest. But, if the property values declined, the highest risk tranche would be worthless, as it was the last in line to be paid if the properties were foreclosed. It did not require much of a decline in the real estate values for the highest risk tranches to be worth zero. And that is one of the primary causes of the present crisis.
One may have read about worthless securities, yet it is obvious that real estate prices did not decrease to zero. In many areas, real estate prices declined only 30 percent or so.
Former President Clinton was quoted recently as admitting he should have regulated the tranches and associated transactions more carefully during his 8 year presidency.
IMHO, until this problem is addressed, the financial markets will not regain stability, or investor confidence. Having a significant portion of a bank’s assets (or other institutions that purchase the mortgage-backed securities) with the potential to be worthless after a small decrease in real estate prices is a relatively new phenomenon and must be addressed. I try to follow this in my spare time, but have not read anything on changes to this aspect of finance.
LOL, if it was so catastrophic, how did he survive?
Give me a break…
Volcanos are to climate what government regulatory changes are to finance.
I see Mr . Teschler is an engineer. I notice an abundance of engineers that are AGW skeptics as well as skeptics of the many other wobbly ideas being sold to us today by our political leaders.
If you spend all day balancing the variables of complex systems so that they actually work, you start to see some patterns emerging. Pretty soon, you can look at any system and tell at a glance if it is grounded in reality.
Here’s to the engineers.
A quick response to Roger Sowell’s comment above. This proves my point about engineers. Engineers sure weren’t involved in the silliness of high risk tranches. No engineer worth their degree would ever allow a system like that to be built with zero margin for error. If it won’t stand up to a worst case scenario, it won’t stand up.
David Holliday:-)
That is just about bang on. I am so delighted to see so many mature heads making the point, GIGO. There will always be a human being at the end of it somewhere. Assume & presume nothing, ever!
As an engineer, & I know I have said this before, computers are little more than powerful calculators & number crunchers, sure they can churn out the numerical answers by the nanosecond where we mere fleshy lumps take minutes to do the same thing. However, the wee, wee, wee, wee, tiny flaw in the whole thing, is that if you get the design philosophy wrong, no amount of number crunching will lead you to the right solution, but only to many ways in which you prove you got it wrong! This point I would like to direct to Roger Sowell, yes computers are wonderful things, but they are after all just a tool to do a job;-) I spend many hours recommending to graduate engineers they sit down with a pad & a pencil & sketch things out by hand before they ever get near a computer programme. As a 51 yo luddite I mistrust computers, & with the current political administration losing personal data left, right, & centre I feel vindicated.
“For your consideration, I give you the new world order… We are being taken on a trip into a new dimension, where down is up and right is wrong. Energy and healthcare and housing are now free, and the only thing we must fear is earth’s newfound propensity to spiral into chaos at the slightest misstep of man. The scientists are the new priesthood and the beggars have become kings. The truly sane must recant their sanity or be shunned. The future has been written, and exists now only in the electronic mind of a supercomputer. This highway leads to the shadowy tip of reality: we’re on a through route to the land of the different, the bizarre, the unexplainable…We may go as far as we like on this road. Its limits are only those of hard truth. Ladies and Gentlemen, you’re entering the wondrous utopia of scientocracy. Next stop….The Twilight Zone.”
Sorry, off topic.
But yesterday, Putin warns US of the evils of socialism,
http://ibloga.blogspot.com/2009/02/putin-warns-us-democrats-against.html
Am I the only one wondering why this is not a top mainstream media story? Not even Googleing the NEWS.
A X-communist, the leader of Russia, warning of a socialistic agenda in the USA.
We’re living in Bizaro world………..
Sorry OT, I’m wrong about the date, it was over a week ago, and I just never heard of it till now.
Putin warning us of socialism sound really weird, until one talks to today’s Russians… and finds out that they consider him to be a nationalist.
I’ll have to check out that link. 🙂
Ah, but Pamela, I didn’t say that the Earth was going to suddenly freeze it’s butt off, though a big ding in the Sun’s output would certainly go a long way towards that. That would make me an alarmist predicting Global Freezing.
Sarc off – I am an alarmist when I see dirt being collectively swept under the rug.
As for the climate snapping, I believe that has occured before in the past.
We are much better able to adapt than at anytime in our history, as long as we know in advance that we should expect the climate to change. It’s the head-in-the-sand attitude with AGW that precludes preparation, whereby they have doomsday measures to put into play to force Earth’s climate in the direction they have pre-conceived it must go.
Sarc on – Why should I follow suit and stick my head in a different color of sand than the AGW mania does?
I don’t want to nuke volcanoes, dump insane amount of chemicals into contrails or perform massive cloud seeding programs to change the Earth’s climate, I just want to see man adapt as it unfolds. It will unfold differently depending on where one resides.
Here’s a great movie script Pam: Earth suffers massive climate blow out of the blue due to massive Y2K bug in AGW models. Prominent programmer tries vainly to warn the world, but in the end manages only to save wife, kids, grandma & grandpa in large suburban. Our heroes brave basketball size hail, flaming satellite debris and raging volcanic rivers as world goes nuts.
Dr. Adam and his wife Eve begin new civiilization on scenic mountaintop as they muse “Why wouldn’t they listen?”.
That must be one of the most intelligent and insightful articles I have read in a while.
Mark N (20:10:48):
“And yet the industry ploughs on with the forthcoming movie ‘The Age of Stupidity’
http://www.ageofstupid.net/ ”
—thus cementing in the minds of the public the underlying “truth” of AGW.
How do you counteract this kind of high-budget propaganda? Most people don’t read Realist blogs like this. Unless they chance upon the buried-on-page 20 article about a ‘skeptic’, or are fans of conservative talk radio, they never hear that there is another side.
How about a suspense thriller, in which our hero (a Tommy Lee Jones-like character) risks life and limb to uncover the sinister decades-long machinations of an organization, suspiciously like The Club of Rome, and an obese former politician building a commercial empire on ‘carbon’ trading (not unlike Algore), all intent on stopping Western civilization in its tracks by creating a gigantic hoax called “global warming.”
In the end the Club members who pushed the American President to close down the coal industry will be marched off in shackles, to hoots and cries of “Shame! Shame!”
It can be called “The Carbon Strategy.”
Who will fund this film? I know a screenwriter (she is a whacky liberal, but will write anything for money).
/Mr Lynn
Financial models have the problem that they are self aware. When analysts make predictions people invest using the predicted results and mess up the result. They buy because the prediction says the price will go up in the future, so the price goes up in the present instead and then they have to sell when the price goes down in the future because everyone rushed in and bought because of the prediction…
Climate modellers are in a bit of a fix too. If they suspect that their models are not much more than interesting computational exercises, they can’t say so out loud because they might lose their funding (ie, job). They are not qualified to do anything else.
This got me thinking about the similarities of Climate Prediction with Solar Cycle prediction. Both have long data set histories – The Sun’s is longer, with fewer ‘sites’. Both have long intervals. Solar Cycle is pretty will defined, The “Climate” interval is less well defined, but is long and permits only infrequent testing. Both have developed a physical basis in only the last 30 yrs. Dare I say that neither are very good? (But I expect a significant improvement when Cycle 25 is predicted – I don’t see the desire to improve Climate Models)
I am not aware of any field in which experts have been able to develop computer models of highly complex systems that work. So I have always been skeptical that climate modelers have succeeded where all others have failed.
Since I don’t know everything, I am curious whether anyone is aware of any computer models in any fields that involves highly complex and chaotic systems that make accurate long-term forecasts?
I work as a software engineer/consultant and I’m well aware of the problems of maintaining and updating code.
Remember the Y2k crisis? That came about because old code was never updated. Nobody knew if programs that had been in operation for decades would work and in many cases, nobody could dig through the layers of patches, bandaids, paperclips and hacks to decipher the internals of the programs, either.
I should not be surprised by the reported size of the Fortran code base, but I am. This language isn’t part of the Computer Science curriculum in any universities in this part of the USA. Is it still taught in the Engineering schools?
Mr Lynn,
The movie looks extraordinarily stupid and boring.
Sort of like AGW, under examination.
They could cover their butts on these models if they represented the error margins correctly. Of course then it would be revealed that they are full of crap…
What are they saying, 95% chance we’ll raise 2 deg C over 100 years or something? Where did they pull that number from after the billions of calculations and assumptions that went into the model? These people need to take some statistics courses.
The models are probably good for allot of stuff and can help us figure some relationships out, but big-picture long-term predictions are probably the absolute worst thing to use them for.
Mike Bryant (01:46:59) :
“For your consideration, I give you the new world order… We are being taken on a trip into a new dimension”
Really GREAT!…all this began with the french revolution. 🙂
Fortunately we, the savages of the third world, will save you.
John Galt: re Fortran still taught in engineering schools?
Yup. The University of Texas at Austin, for one. UT is a decent institute of higher education (and my undergrad alma mater). (not to be confused with University of Tennessee, another UT)
see http://www.utexas.edu
Click here for fortran class
Also, the other UT (Tennessee) teaches fortran . From this site: “For example, we’ve made changes in the NE [nuclear engineering] Fundamentals course in response to alumni feedback, bringing the Fortran computer language back to the curriculum in order to prepare graduates for the field.”
The Y2K fortran bugs were not that hard to fix. Refineries, chemical plants, power plants, etc. with fortran made it through midnight 12/31/1999 into 2000 just fine.
Just another scare-mongering non-event.
Jeff B. (23:14:04) :
“If you spend all day balancing the variables of complex systems so that they actually work, you start to see some patterns emerging. Pretty soon, you can look at any system and tell at a glance if it is grounded in reality.”
I’d rather employ someone who could actually make things work in the real world, and rely in their advice than someone full of theories and models that do not match what is going on in the world. Before calculators, we had sliderules. In order to use one effectively, the user had to be able to estimate what the answer should be. A computer is nothing more than a calculator that can concatenate many steps. It seems that the modelers have the answer they want in mind, and simply try to plug in the functions to get their ‘right’ answer. When the models can replicate the past centuries, only then will it be worthwhile to listen to them about the future. I’m not holding my breath…
Thanks for the update regarding Fortran in engineering schools.
pre-Y2K was a great time to be in software consulting. This business never made so much money. If you had asked me about the seriousness of the threat, I would have repeated the industry line about it being the gravest danger you could imagine.
Read an interesting piece on how illegal money laundering has not been mentioned by the media, yet may have played a role. Can you imagine being the poor dope at AIG who has to tell the mob he lost billions of their money?