Guest Post by Willis Eschenbach
Following up on his brilliant earlier work “The Black Swan”, Taleb has written a paper called Error, Dimensionality, and Predictability (draft version). I could not even begin to do justice to this tour-de-force, so let me just quote the abstract and encourage you to read the paper.
Abstract—Common intuitions are that adding thin-tailed variables with finite variance has a linear, sublinear, or asymptotically linear effect on the total combination, from the additivity of the variance, leading to convergence of averages. However it does not take into account the most minute model error or imprecision in the measurement of probability. We show how adding random variables from any distribution makes the total error (from initial measurement of probability) diverge; it grows in a convex manner. There is a point in which adding a single variable doubles the total error. We show the effect in probability (via copulas) and payoff space (via sums of r.v.).
Higher dimensional systems – if unconstrained – become eventually totally unpredictable in the presence of the slightest error in measurement regardless of the probability distribution of the individual components.
The results presented are distribution free and hold for any continuous probability distribution with support in R.
Finally we offer a framework to gauge the tradeoff between added dimension and error (or which reduction in the error at the level of the probability is necessary for added dimension).
Dang … talk about alarmism, that’s scary stuff. Here’s one quote:
In fact errors are so convex that the contribution of a single additional variable could increase the total error more than the previous one. The nth variable brings more errors than the combined previous n-1 variables!
The point has some importance for “prediction” in complex domains, such as ecology or in any higher dimensional problem (economics). But it also thwarts predictability in domains deemed “classical” and not complex, under enlargement of the space of variables.
Read the paper. Even without an understanding of the math involved, the conclusions are disturbing, and I trust Taleb on the math … not that I have much option.
H/T to Dr. Judith Curry for highlighting the paper on her excellent blog.
w.
As Usual: Let me request that if you disagree with someone, please quote the exact words you are referring to. That way we can all understand the exact nature of your objections.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Just to enlarge on my prior comment.
My example above is highly simplified. Imagine if instead of a simple equation like:
y = a1*x1 + a2*x2 + a3*x3
where all the variables assumed to be simply additive, we had an equation like:
y = ((a1*x1)^(a2*x2))*a3*x3
The propagation of error in this formula could be horrendous, with one bad parameter invalidating the whole effort.
So, with mathematical models of the REAL world, simplicity is highly to be desired. It is not rocket science.
What I didn’t understand about Taleb’s math was was he referring to the first equation that relates variables x1..x3 into the model y (linear) or the second equation (non-linear)?
Always consider first the obvious and if it does not work out then progress but I agree with weather forecasting we start small and go large.. if necessary.
Even without an understanding of the math involved, the conclusions are disturbing
Why?
Taleb is a fascinating man. I listen to him whenever I can even and has a wonderful gift for communication. Find him interviewed several times and in depth by Russ Roberts, professor of economics at George Mason, on his weekly podcast ‘Econtalk’. Who wants to listen to a weekly podcast on economics? Well I suppose I do, it’s a real gem, entertaining and fascinating every week. Give it a spin.
Russ did a great interview a week ago with Matt Ridley which I enjoyed very much.
Links ??
http://www.econtalk.org/archives/2015/01/nassim_nicholas.html
The Matt Ridley interview is here:
http://www.econtalk.org/archives/2015/06/matt_ridley_on.html?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+econtalk%2FxmOW+%28Econ+Talk%29%22%20title=
Sorry Grant. While I have a high regard for Taleb and most of his ideas he comes across as way too arrogant in most of the interviews, including those with Russ. I was also not very impressed with his argument against GM and about the need to take the AGW argument more seriously because of the fat tail issue. Like many smart math guys he sometimes gets lost in the detail and forgets to look around at the divergence between the prepositions and the reality. The Ridley argument on the fat tail issue was a good one. Taleb is being fooled by the data that he is given and is totally unaware that it does not reflect the actual measurements or the real distributions.
BTW, Am I the only one who when reading an interview of Taleb is reminded of the Jeff Goldblum character from Jurassic Park?
Dino:
In the June 3, 2007 New York Times Sunday Book Review, a survey “Read Any Good Books Lately?” featured celebrities giving their recommendations.
Michael Crichton’s response:
“Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable. The second volume by the author of Fooled by Randomness continues his theme — our blindness to the randomness of life — in an even more provocative, wide-ranging and amusing mode. A book that is both entertaining and difficult.”
OK, Crichton wrote “Jurassic Park” in 1990, but still…
Great minds think alike, apparently.
Chaos theory was already well advanced by 1990, but sadly ignored by climastrologists.
From that link you provided above,
Taleb on Hayek :
“So, Hayek was against–what? Against a top down social planner who thinks he knows things in advance, can’t foresee results. And makes–because the person first of all has arrogant claims that may harm us, but also because of mistakes–he’s not going to foresee his own mistakes; and mistakes will be large.”
Sounds very, very, familiar.
Which is really, really odd.
In 1987 James Gleick wrote “Chaos: Making a New Science” and his brother is none other than Peter Gleick, the eco policy scientist. I mean, don’t they even talk over the holidays?
“Chaos theory was already well advanced by 1990, but sadly ignored by climastrologists.”
True, and I always find that fascinating because one of the first great books on chaos was “Chaos: Making a New Science” by James Gleick, brother of Peter Gleick, climate alarmist extraordinaire. I read “Chaos” back in 1987 and it left me with a lasting skepticism about the ability of modelers to properly fence or account for complexity. I get the same thing from Taleb who seems to write extensively about what could be characterized as the unpredictable boundary between order and chaos. Taleb is an alarmist, obsessed with the sudden onset of turbulence after a prolonged period of laminar flow. He sees the potential for catastrophic discontinuities everywhere, a perspective reasonably colored by his experience with the sudden breakdown of civilized society in Lebanon. I enjoy his books, seeing them as an assault on sanguine central planners and the complacent certainty of modelers. But Taleb appears to buy in to the climate alarmism with a hearty “It may suddenly become so much worse than it now appears!” rather than applying his otherwise excellent skepticism to the elaborate models of the climate community. Clearly our world is at least somewhat Anti-Fragile or we would not still have life after all these billions of years. I sometimes wonder whether James is the same way when sitting down to lunch with Peter? “Yes Peter, we can’t predict the weather seven days in the future, but clearly these complex models predicting the global thermageddon are onto something.” Or does he quietly eat his bisque and roll his eyes at the certainty?
“Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable. The second volume by the author of Fooled by Randomness continues his theme — our blindness to the randomness of life — in an even more provocative, wide-ranging and amusing mode. A book that is both entertaining and difficult.”
For what it is worth, starting with Fooled by Randomness would be a good idea. It was brilliant and a great effort that shows Taleb’s genius.
Chaos theory, Catastrophe theory and Fuzzy sets are the three bad boys of mathematics. The mathematics is real enough. But by virtue of an excessively sexy name, these get over-promoted, over-hyped, misunderstood and applied inappropriately all over the place in areas outside mathematics; Jeff Goldblum pontificating about “nature finding a way” is just one example.
And don’t forget the mathematics of marriage: The Principle of Least Action.
I noticed that no one mentioned computer ALU round off errors due to using a finite number of bits. Also the step size of the calculations were not mentioned.
Thanks to the help of Chinese Checkers (a much more appropriate term than “hackers”)…a document internal to the “economic establishment” has been found, address this very publication. We reproduce it here in full:
Your Honor, allow me to present EXHIBIT A. of the general rabble’s case against Mr Taleb.
https://dl.dropboxusercontent.com/u/50282823/Propagation.pdf
It will become very clear that Mr. (is it Dr?) Taleb has not only had the “kutzpa” to make himself some number of millions of dollars, due to his “irrational” approach to the markets, and his belief of using
…shall we say, “intuitive” (some say GUT feeling) methods with regard to active floor trading, but he
has now…once again, produced a rigorous mathematical effort, to “show off” his considerable analytical
skills.
It should be noted, however, that as before, Mr. Taleb’s analytical skills are being applied for one primary reason, i.e. to expose our long standing fraud of the existence of any and all methods to use historical data, to predict future outcomes. (This could be applied to other fields outside of economics also, we may be able to recruit allies in our effort to quash Dr. Taleb.)
As you may well be aware, our livelihood is highly contingent on the number of FOOLS (excuse me, clients) for which we can establish and maintain a belief…by high sounding words, our own charts and graphs (either tending upward, downward or sideways) by which we can continue a delusion of our “economic invincibility”. One may recall how successful we were in preaching the “peak oil” mantra, with graphs of declining well curves, historical analyses of limit oil fields, growth of demand..and how we had
managed to maneuver the oil markets above $100 a barrel. (Adding immensely to the personal wealth of many of our clients.)
Unfortunately, to borrow a phrase from Mr. Taleb, a “black swan” occurred…(viz. N.D. oil, lateral drilling, modern fracking methods) which made much of the tenants of our predictions moot.
Now we have this INSULT to Dr. Taleb’s INJURY, in his saying …in a more direct analytical form, that OUR analytical methods have as much value as “stuff you spread on your garden to make it grow”.
It is the PLEA of our clients your HONOR that something be done to stop Mr. Taleb and put a halt to his dangerous activities, before we are brought before the court of pubic opinion for a complete session of ridicule and dis-belief!
Signed:
Yours, the ECONOMIC establishment
I think this all leads to the conclusion that all we are left with in the climate “prediction” (or projection) field is the good old “gut feeling”.
As uncertainty goes, it is much less prone to errors…
Isn’t this the same thing as saying that any errors in initial conditions will compound to the nth power where n=the number of subsequent re-iterations using said initial conditions?
That’s exactly what popped into my mind as I glanced over the article.
Not exactly how compounding is calculated, but I hope you get my point.
A more tangible example from my childhood comes to my mind:
Like attempting to saw 100 planks that are all 2,000 meter long +/- 2 millimeter.
Measure the first one with a tape measure.
And thereafter
– by eye only and no other regards to the size being reasonable –
always using the previous plank you sawed as a template for the next.
I tried that with 10 planks – my first experience with error propagation. 🙂
A reminder to American readers, that in Europe the comma “,” is used as a decimal point. Those planks aimed at two meters precisely in length, not two kilometers — which would have been truly memorable.
Please post a link to final publication when it is available.
This is yet another nail in the coffin of 19th century thought: That given enough time we could discover the underlying equations of nature and predict everything and control anything.
In the 20th century one theorem after another demonstrated that we couldn’t do that. Gödel’s incompleteness theorem. Turing’s work on the halting problem. The development of chaos mathematics. Quantum probability theory.
Few people, even those deep in science themselves, realise just how narrow is the field to which conventional linear analysis – the sort that has lead the way in one scientific breakthrough after another – is confined.
Looks like this paper is just another reminder of how little use (current) science and mathematics are in complex real world problems.
What the paper seems to be saying is that a function of a a large number of variables, all of which are well controlled within normally distributed variation, will exhibit much larger variation than its input terms: That is the probability of its output is not a linear function with respect to the combined probability of its inputs.
But that is the case for all non-linear functions.
Consider the case where a man stands under a ton rock suspended by a thin wire at which a sniper is firing rounds from a distance.
The rounds may be distributed at random but bell curve shaped distances from the wire. However if one strikes the wire, the rick falls on the man’s head. The outcomes as far as the man is concerned are not a smooth set of probabilities, but a binary case that means he either gets away unscathed or dies.
What I am not mathematician enough to understand in this case is whether Taleb’s analysis is somehow applying to linear functions in quantity as well.
my recollection is that Newton spent a considerable amount of time working on nonlinear functions.
its pretty much an accepted fact that linear analysis is appropriate for only a tiny fraction of phenomena in the world…
The best example of what he’s talking about:
Let’s say you want to calculate the exact motion of a drop of water molecules in a micro gravity.
You have to have positions for each molecule, temperture, gravity of the environment, and that’s before you start calculating the interaction of the molecules, what he’s saying is the error of the actual position quickly grows.
The higher the number of requirements the quicker the error grows, the more molecules you’re calculating the quicker the error grows. It’s possible that this particular problem can not ever be solved, but even if you could for 5 molecules, 6 might be impossible, or if you add a vibration to the 5 molecule problem it could become unsolvable.
And my example is simply compared to global climate.
The bell curve wrings from many, one / Describes all things; determines none.
Read Nassim’s earlier book Fooled By Randomness – I haven’t had time to read his latest, but I think the previous book is even better than The Black Swan. You can find PDFs of Fooled easily enough…well worth it!
Nassim also voiced opinions contrary to the ‘consensus’ back in 2009:
http://green.blogs.nytimes.com/2009/08/21/authors-climate-remark-ruffles-feathers/?_r=0
He certainly ruffles the GMO folks feathers…
The climate will do what does. When modeling, it may be wise to remember “the map is not the territory”.
Taleb originally wrote about the limitations of analysis in the financial sector.
I’ve long time thought that the same limitations and associated social responses are evident in Climate science.
And we see similar problems in other fields – pharmaceuticals, nutrition, etc.
There’s a book to be written about the rise and fall of the “analyst”…
Has anyone attempted to draw these clear parallels and offer a general thesis?
thanks,
Dan
Does anyone here know of a model/simulation that is used to find out NEW characteristics of effects?
My understanding is that a model/simulation is designed with correct functionality such as an aircraft flight simulator, and can then be used to train pilots.
Do we have any such models/simulations that have been created which are then used to ‘predict’ something we did not know?
I assume that the ‘center of gravity shift’ observed when an aircraft transitions to supersonic flight was found by physical testing, not as a result of a simulation.
Since, until we went supersonic we were unaware of the issue and could not ‘program it in’.
( I know this predates large computers but… )
So models interpolate well (except for phase boundaries). Extrapolate, not so much.
“tour-de-force”
Climate modellers – Read and Learn!!
Relating this to my own experience leads to the axiom that “No plan survives first contact with the enemy.” The variables spiral rapidly out of control no matter what you thought you knew in the planning phase. The solution is to train your commanders to adapt and if that does not work……drop a nuke. It would seem that the climate chatteratty have already reached that stage but it failed to go off and we are somehow still here in the same climate with the same weather at the same temperature as we have been for a lifetime. Sucks when your bomb doesn’t explode.
Ordered from Acme.
================
We could simply say that climate has roadrunner skills. Nothing wicked about it.
=======================================
Hah! I have known this for years. It is summed up in our family by the saying “Anything can happen in the next half hour”.
…I assume that the ‘center of gravity shift’ observed when an aircraft transitions to supersonic flight was found by physical testing, not as a result of a simulation. Since, until we went supersonic we were unaware of the issue and could not ‘program it in’….
Since air travels over different parts of an aircraft at different speeds, we encountered to problem gradually, as aircraft increased performance and reached high speeds in dives. It’s known as ‘Mach Tuck’, and can occur in aircraft susceptible to it at quite low speeds.
When the problem was first encountered it was replicated in wind tunnels, understood in theory, and so by the time planes were being designed to travel supersonically there was no problem with ‘programming it in’.
Two aircraft stand out in this research process – a modified Mk XI Spitfire flown at Farnborough in 1944, which achieved controllable speeds of M0.9 in a dive (over 600mph!) and the Miles M52 (which provided the design for the Bell X-1). The stories of both are well worth reading…
I think the aircraft that really stands out in this process was the DeHavilland 110 prototype, which tried to go supersonic at the Farnborough air show in 1952 and disintegrated, killing many people in the crowd.
Please se Prof John Christy of the University of Alabama at Huntsville, speaks on the subject of climate change. Christy is a climate scientist and responsible for the UAH satellite datasets.
https://youtu.be/BhFqgflaQVQ
This man is dead wrong about nuclear power.
Yes, it is ‘safe’ so long as it is operational and under control. But when it is NOT under control and something very bad happens and it is inevitable this can and will happen roughly every 20 years or so, the entire region around these dangerous entities is uninhabitable and everyone loses everything and is forced to flee forever.
Then the genetic destruction moves forward relentlessly and we have no idea how long but know that 40 years later, it is still very nasty where previous accidents happened and it seems it may be never-ending, we just do not know.
Sneering that ‘no one is dying’ is crazy when the deaths are slow, relentless and unavoidable and above all, invisible that is, you don’t know when you enter a death zone via any of your senses except perhaps if you notice a lot of deformed living things scattered all over the place.
ems
Your strong words seem to be out of place. So I’ll sneer at you.
As an old boomer sailor I know that nuclear power can be the ultimate power source. Of course, it is not something to go off half-cocked like you did above. Your research seems to be based on the fear that you have learned in your younger life and not from experience. After all, nuclear power is not a BOMB. Just like windmills are not placed in an area just to kill wildlife. And what about the mess that your thinking causes when the people begin to use the batteries for all the automobiles that are on the road. Buy yourself a golf cart and take care of it for a few years so you can gain experience as to the horrible mess it can cause when in the hands of the general public.
We all know that your side has in the back of their mind that OVERPOPULATION must be stopped without considering that we are already down that road without help from firm legislation.
So I suggest to you to TONE IT DOWN!!!!
“. . . every 20 years or so, the entire region around these dangerous entities is uninhabitable and everyone loses everything and is forced to flee forever.”
Not if the power plant is anchored 20 miles offshore in a specially designed fail-safe module.
How many nuclear power plants are ‘anchored offshore’ and would this work with tsunamis?
The answer is zero and then of course, violent storms and tsunamis are all too common ‘offshore’. Then there is warfare: #1 target in a war is a nuclear power plant.
emsnews, you say “the deaths are slow, relentless and unavoidable and above all, invisible that is, you don’t know when you enter a death zone via any of your senses except perhaps if you notice a lot of deformed living things scattered all over the place.” I assume (perhaps wrongly) that you are talking about the area around Chernobyl. I have watched the news about Chernobyl ever since the accident, now many years ago, as part of professional curiosity. It would appear that, other than the destruction that occurred from the perfectly ordinary industrial accident and explosion, that the release of radioactive particles has not harmed the flora or fauna of the area. The unfortunate deaths of the men who initially were sent in could easily have been prevented, by cycling people in-and-out of the area. The animals and plants still in the area seem to be thriving, with no apparent increase in deformities or genetic aberrations. There are even people who have moved into the area. with no immediately apparent problems. There is actually much more genetic damage that occurs from chemical exposures than from increased radiation exposure.
I realize that this will not make any change to your opinions, emsnews. However, for the other people that read this, I would advise that they do their own investigations into this. Radiation and radioactivity are often not reported on in a fair and even-handed manner, and there is a lot of misinformation out in the wild.
S0ren- thanks for posting the above video of John Chrisy. While watching it I was thinking of all the real time data I have been collecting. The collection process can be complicated and you can never be sure that the “data” is even real. I guess it is time for the other side to consider the aspect that they could be wrong. That their research is really based upon homogized data instead of past-ur-eyes data.
All modeled processes become unpredictable if sufficiently complex and/or if the time scale of the prediction is long. Even solar system motion is unpredictable at sufficiently long time scale. So I’m not sure that there is anything very surprising that adding variables increases unpredicability.
The usual assumption of novices to statistics is that variations (and errors) are independent. That is taught as nearly axiomatic in any introductory statistics class – the number that comes up on one die is unrelated to the number that comes up on the other.
When valid, this assumption allows reasonable estimates of the as-yet-unmeasured. Casinos thrives on the validity of this assumption for small, well-characterized and controlled systems.
The problem is that adding that nth variable can link the errors in the ith and jth variables in unanticipated ways so that they are no longer independent. From that point all bets are off. Only a fool would roll dice that are tied together.
Something similar can be seen in the calculations of chaos theory where the last (past) value and the current (present) value are used to calculate the next (future) value. Due to the iterative nature of the calculations a small variation or error introduced anywhere in the line is propagated indefinitely, and cannot be ‘corrected’.
Subsequent calculations can only continue to diverge, as we see daily in ordinary weather forecasts which only use a comparatively small ensemble of variables (typically air temperature, pressure, wind speed, direction, and humidity for a few score locations), yet have no predictive value at all beyond a week.
The subtleties of advanced mathematics and the connections to physical systems are often incompletely understood by the very investigators who work most closely with them. The very fact they are ‘investigating’ suggests they acknowledge to themselves there are aspects of the situation which they do not understand. This warrants the warning: “Do not try this at home.”