The Button Collector or When does trend predict future values?

So, you know now who to call if YOU loose a bu...

How many buttons will he have on Friday? (Photo credit: Wikipedia)

Guest essay By Kip Hansen

INTRO: Statistical trends never determine future values in a data set. Trends do not and cannot predict future values. If these two statements make you yawn and say “Why would anyone even have to say that? It is self-evident.” then this essay is not for you, you may go do something useful for the next few minutes while others read this. If you had any other reaction, read on. For background, you might want to read this at Andrew Revkin’s NY Times Dot Earth blog.

­­­­­­I have an acquaintance that is a fanatical button collector. He collects buttons at every chance, stores them away, thinks about them every day, reads about buttons and button collecting, spends hours every day sorting his buttons into different little boxes and bins and worries about safeguarding his buttons. Let’s call him simply The Button Collector or BC, for short.

Of course, he doesn’t really collect buttons, he collects dollars, yen, lira, British pounds sterling, escudos, pesos…you get the idea. But he never puts them to any useful purpose, neither really helping himself or helping others, so they might as well just be buttons, so I call him: The Button Collector. BC has millions and millions of buttons – plus 102. For our ease today, we’ll consistently leave off the millions and millions and we’ll say he has just the 102.

On Monday night, at 6 PM, BC counts his buttons and finds he has 102 whole buttons (we will have no half buttons here please); Tuesday night, he counts again: 104 buttons; on Wednesday night, 106. With this information, we can do wonderful statistical-ish things. We can find the average number of buttons over three days (both mean and median). Precisely 104.

We can determine the statistical trend represented by this three-day data set. It is precisely +2 buttons/day. We have no doubts, no error bars, no probabilities (we have 100% certainty for each answer).

How many buttons will there be Friday night, two days later? 

If you have answered with any number or a range of numbers, or even let a number pass through your mind, you are absolutely wrong.

The only correct answer is: We have no idea how many buttons he will have Friday night because we cannot see into the future.

But, you might argue, the trend is precisely, perfectly, scientifically statistically +2 buttons/day and two days pass, therefore there will be 110 buttons. All but the final phrase is correct, the last — “therefore there will be 110 buttons” — is wrong.

We know only the numbers of buttons counted each of the three days – the actual measurements of number of buttons. Our little three point trend is just a graphic report about some measurements. We know also, importantly, the model for the taking the measurements – exactly how we measured — a simple count of whole buttons, as in 1, 2, 3, etc..

We know how the data was arrived at (counted), but we don’t know the process by which buttons appear in or disappear from BC’s collection.

If we want to be able to have any reliable idea about future button counts, we must have a correct and complete model of this particular process of button collecting. It is really little use to us to have a generalized model of button collecting processes because we want a specific prediction about this particular process.

Investigating, by our own observation and close interrogation of BC, we find that my eccentric acquaintance has the following apparent button collecting rules:

  • He collects only whole buttons – no fractional buttons.
  • Odd numbers seem to give him the heebie-jeebies, he only adds or subtracts even numbers of buttons so that he always has an even number in the collection.
  • He never changes the total by more than 10 buttons per day.

These are all fictional rules for our example; of course, the actual details could have been anything. We then work these into a tentative model representing the details of this process.

So now that we have a model of the process; how many buttons will there be when counted on Friday, two days from now?

Our new model still predicts 110, based on trend, but the actual number on Friday was 118.

The truth being: we still didn’t know and couldn’t have known.

What we could know on Wednesday about the value on Friday:

  • We could know the maximum number of buttons – 106 plus ten twice = 126
  • We could know the minimum – 106 minus ten twice = 86
  • We could know all the other possible numbers (all even, all between 86 and 126 somewhere). I won’t bother here, but you can see it is 106+0+0, 106+0+2, 106+0+4, etc..
  • We could know the probability of the answers, some answers being the result of more than one set of choices. (such as 106+0+2 and 106+2+0)
  • We could then go on to figure five day trends, means and medians for each of the possible answers, to a high degree of precision. (We would be hampered by the non-existence of fractional-buttons and the actual set only allowing even numbers, but the trends, means and medians would be statistically precisely correct.)

What we couldn’t know:

  • How many buttons there would actually be on Friday.

Why couldn’t we know this? We couldn’t know because our model – our button collecting model – contains no information whatever about causes. We have modeled the changes, the effects, and some of the rules we could discover. We don’t know why and under what circumstances and motivations the Button Collector adds or subtracts buttons – we don’t really understand the process – BC’s button collecting because we have no data about the causes of the effects we can observe or the rules we can deduce.

And, because we know nothing about causes in our process, our model of the process, being magnificently incomplete, can make no useful predictions whatever from existing measurements.

If we were able to discover the causes effective in the process, and their relative strengths, relationships and conditions, we could improve our model of the process.

Back we go to The Button Collector and under a little stronger persuasion he reveals that he has a secret formula for determining whether or not to add or subtract the numbers of buttons previously observed and a formula for determining this. Armed with this secret formula, which is precise and immutable, we can now adjust our model of this button collecting process.

Testing our new, improved, and finally adjusted model, we run it again, pretending it is Wednesday, and see if it predicts Friday’s value. BINGO! ONLY NOW does it give us an accurate prediction of 118 (the already known actual value) – a perfect prediction of a simple, basic, wholly deterministic (if tricky and secret) process by which my eccentric acquaintance adds and subtracts buttons from his collection.

What can and must we learn from this exercise?

1. No statistical trend, no matter how precisely calculated, regardless of its apparent precision or length, has any effect whatever on future values of a data set – never, never and never. Statistical trends, like the data of which they are created, are effects. They are not causes.

2. Models, not trends, can predict, project, or inform about possible futures, to some sort of accuracy. Models must include all of the causative agents involved which must be modeled correctly for relative effects. It takes a complete, correct and accurate model of a process to reliably predict real world outcomes of that process. Models can and should be tested by their abilities to correctly predict already known values within a data set of the process and then tested again against a real world future. Models also are not themselves causes.

3. Future values of a thing represented by a metric in data set output from a model are caused only by the underlying process being modeled–only the actual process itself is a causative agent and only the actual process determines future real world results.

PS: If you think that this was a silly exercise that didn’t need to be done, you haven’t read the comments section at my essay at Dot Earth. It never hurts to take a quick pass over the basics once in a while.

# # # # #

Advertisements

  Subscribe  
newest oldest most voted
Notify of
Steve Obeda

“The only correct answer is: We have no idea how many buttons he will have Friday night because we cannot see into the future.”
— I would say that states things a little more harshly than is justified and that a more correct answer is “We cannot be sure, but the trend indicates a best guess of…” I forecast using trends all the time. A trend is useful even if it is only partially informative. No, we don’t know the underlying equation and we might be wrong but in many cases it doesn’t matter.
What’s missing here is the intended use of the forecast. If your boss is asking you to forecast sales, you’d better not tell him that there’s no way to know for sure. Before firing you he’ll say he knows that, but the business needs to make decisions on inventory and capacity. It doesn’t matter if you’re a bit wrong because you’ll simply adjust the forecast. But if your boss is deciding whether to decarbonize the global economy at a cost of several trillion dollars, then the strength of your model had darn well better justify it. Otherwise, the answer is going to be to keep studying and keep him posted on your progress, but hold off on the big spending.

I think that makes it very clear. It was also a fun read. Thank you.

Nicely put, thanks. Yes, too often people are lulled into thinking the pretty graph has meaning beyond being a useful display device.
Cause is everything, and when it comes to “global warming” there are nowhere near enough causal factors that can be relied upon to produce proof – it is mostly pure speculation, the mind boggles at the use of these graphs to bring in monstrous taxes and trading schemes for miniscule human introduced increases of one of the worlds most important elements for the survival of life on this planet

jim2

He will have between 0 and a googleplex of buttons. I’m 95% certain of it.

gnomish

if the underlying process is regular and invariant, then trend is, indeed, of predictive value with great accuracy.
for example, i need not specify planetary orbital paths nor gravitational constants to tell you that between now and a year from now there will be @ 365 cycles of day/night.
but this in no way validates ‘statistics’ as anything remotely scientific. it is a tool of numerologists and no less mystical than tea leaf reading.
nevertheless, i think it may be worth noting that statistics, like the broken watch, has its moments on occasion – if only to make sure this type of occurrence is properly classified as special and exceptional.

meemoe_uk

author is getting on a high horse over words. I’ve used trends to predict the future b4. Distinguishing between models and statistical trends is a grey area.

Steve M. from TN

jim2 says:
October 17, 2013 at 7:26 pm
He will have between 0 and a googleplex of buttons. I’m 95% certain of it.
I’m 100% sure he’ll have between 0 and infinity (inclusive).

vigilantfish

Based on current trends, how many months until WUWT reaches 2 billion views? I guess this cannot be safely predicted as unforeseen events may change current viewing habits.
I am astounded, as a dedicated visitor since well before Climategate, at how rapidly the number of views is mounting – around a million a week. Astounded, but not surprised. Congratulations, WUWT!

DAV

On the whole you’re right. However. you don’t need to know every cause. In fact, you can never know when you possess a complete set. You only need to know the more common and major ones. If this weren’t true, then statistical mechanics would have zero value.

bones

There are systems for which there are deterministic, physical causes of trends that can be characterized by parameters whose values are determined by fitting noisy data trends. Future average performance is projected by extrapolating the trend via its parameters. IPCC climate modelers would have us believe that future climate really does depend deterministically on CO2 levels. The obvious failure of their models to predict the current arrest of global temperature anomalies says that they have failed to account for some important physical variables.

Greg Cavanagh

It’s funny and sad that we have to remind scientists (and laymen) how to do statistics using pre-school techniques. Well done though, if the articles was shorter, perhaps science journalists would read it too (and not just the abstract).

Aynsley Kellow

Two weeks on, the number of buttons was found to have been constant for 17 days. I understand that this matter was then considered by the IPCC (Intergovernmental Panel on Clasps and Closures) and they reached a consensus opinion that, as 10 of the highest observations of number of buttons were made in the previous fortnight, the collection of buttons was clearly continuing to grow.

acementhead

“…he collects dollars, yen, lira, British pounds sterling, escudos, pesos(…)sic you get the idea. But he never puts them to any useful purpose, neither really helping himself or helping others…”

Sorry but the author is incorrect. Savings==investment. If the saver has chosen not to benefit then the benefit goes to somebody else. A few years ago wealth was transferred from some savers, whose saving were in US dollars, to savers whose savings were in bank shares. About a trillion dollars was stolen from one class of saver and handed to another. Saving always benefits SOMEBODY.
It is not possible to “neither help oneself nor others” by saving. Saving is always put to useful purpose.
The foregoing assumes that the saving are “real”, i.e. more was produced than consumed. Fake savings can be produced by favoured classes by permitting them to write up the value of imaginary assets(that’s what the banker thieves did. Oh and are still doing by the way(QE x)).

Mark XR

So, just because there has been a pause in rise of global mean surface temperature does not mean we can reject the physics of global warming? Rats.

Hoser

The trend just provides a first approximation for a model. The Pearson statistic may help you decide how much you want to bet on a linear extrapolation. It doesn’t take into consideration the consequences of being wrong. I’m betting I’ll wake up tomorrow, and that I’ll keep waking up each day for some time. Happily, if I’m wrong, I’ll never know. However, there would be consequences for others I’d rather they didn’t have. There is quite a bit of money on the line for insurance, and underwriting is a serious business. Back to trends again. Dealing with relatively large numbers of people helps. Do we have enough people for really good statistics (see DAV above)? We still don’t have our Harry Seldon, nor psychohistory.

Adam

Climate variables are not Martingale variables.

I believe that Nassim Taleb refers to this as the “falacy of the turkey” in both the Black Swan and Antifragile. Then again, what does he know…being a independently wealthy Wall Street “pit trader”, who AFTER achieving that success, went back to graduate school and earned his academic credentials. Let’s hope he’s no turkey!

Brian H

Trends which are stable despite a variety of potential disruptive events over the observed period will induce some confidence that it will continue. Trends which wobble in response to interventions may give some indication of causality. Climate science has neither.

Janice Moore

Well done!
Yes, as others have pointed out, we need to make guesses about the future and sometimes those guesses turn out to be accurate (the IPCC is merely precise, lol), but, in the end, all we have based on statistics which only describe the past or present is:
a guess.

dalyplanet

It was interesting and informative to read the comments at Dot Earth Kip Hansen. Even relatively intelligent posters have difficulty understanding this simple concept that trend of the past is not necessarily predictive of the future especially at longer time scales. It is clear that many do not understand the complexity of the solar, ocean, atmosphere coupled dynamics. It was especially interesting that your link to Marcia Wyatt’s new paper was ignored. Much has been ignored regarding the oceans while focusing on the atmosphere over the last twenty years.

Jquip

Kip Hansen: “The only correct answer is: We have no idea how many buttons he will have Friday night because we cannot see into the future.”
The correct answer is that you’ve made a mash of the whole topic. It’s certainly true that you cannot guarantee now the then that has yet to happen. But statistics are not and have never been a guarantee.
Nor is there any necessity, or even sanity, in attempting at self-fulfilling logical positivist approaches to causality. Newton went commando with Hypothesis non Fingo. Kepler was a pure data fiddler. So was Galileo. Point of fact, any postulated cause has less expectation for correctness, in general, than a long chain of validated predictions arising from a non-causal model. And that’s putting aside any expected issues that arise from the field of physics and it’s distinct reliance on impossible objects taken to every infinite asymptote they can find.
And yet the things still work out. The problem here isn’t statistics, nor trends generally, but one of repetition and validation of long chain of predictions. Especially when you have non-replicable observables and need to see your trend go both up and down. The problem isn’t the math. And it isn’t the models per se.
It’s a complete disregard for validating the results before launching into religious pronouncements that the world is doomed, that CO2 *is* the climate, or that there must be something terribly wrong with math.

Janice Moore

“Dewey Beats Truman”

Janice Moore

@ Aynsley Kellow (8:17pm) — Nicely put. Precisely!

Chris G

OMG buttons cause global warming.

thingadonta

“Models also are not themselves causes.”
Except the effect they have on the human mind.

DougS

Excellent presentation of the underlying problem Kip, right on target.

graphicconception

Statistics are strange things. Some thoughts:
Lord Rutherford is supposed to have said: “If your result needs a statistician then you should design a better experiment.”
90% of lung cancer victims have smoked. 100% of them have drunk water.
Odds of a million to one are considered remote. However, I can do something that has odds of, approximately 80,658,175,170,943,878,571,660,636,856,403,766,
975,289,505,440,883,277,824,000,000,000,000 to 1 against any time you like. I just need to put a deck of cards in a line.

davidmhoffer

I liken it to a person walking up a hill and down the other side, and measuring their altitude at every step. Even if we know exactly what the length of their stride is, we cannot predict from the past data what their altitude will be on the next stride unless we know exactly what the shape of the hill is. In fact, if all we have is the data from the first 25% of the journey, the data would provide a trend suggesting the person is headed for outer space.
Interestingly, if we stopped our data collection when the person was 5 steps past the crest of the hill and on their way back down, we could say, and correctly so, that the person’s last 10 steps were the highest in the entire record of the journey. It would not change the fact that the person is going down hill.

Karl

It is amazing how many commenters still, even after reading the article, cling to the idea of a predictive trend.
A trend never predicts future values. By definition, as the author pointed out, a trend[line] is fit to the data (not the other way around).
Some commenters seem to be unable to uncouple the trend from the process beneath.
Measurements Monday, Tuesday, Wednesday, and Thursday, of empty milk bottles on the front porch being, 1,2,3, and 4 respectively create a trend that predicts 5 empty milk bottles on Friday.
Unfortunately for the trend based prediction user, there are only 4 quart bottles of milk, and the milkman brings new milk on Fridays, so the correct answer is zero.
The trend is utterly useless as a predictor — always.

HankHenry

Trends are an illusion

Gogs

This is what worries me about share traders who use charts and “technical analysis” as predictions. Expressing the price as trying to break resistance lines and all that.
Prediction, it ain’t!

jimmi_the_dalek

But half the articles on this site back predictions based on trends, usually of the form “it is going to get colder”. Are you saying all these articles are worthless?

observa

Young Antounie Caen’s schoolteacher-
Swans are white, always have been white and always will be young Antounie, so get that through your thick head and pay attention in class meboy.
http://www.svswans.com/black.html
@gnomish-
“if the underlying process is regular and invariant, then trend is, indeed, of predictive value with great accuracy.
for example, i need not specify planetary orbital paths nor gravitational constants to tell you that between now and a year from now there will be @ 365 cycles of day/night.”
Black swan event and poof! Just that we are lulled into thinking that it will be so from trend but should we ever have reason to believe otherwise then a helluva lot of other theories and science will evaporate instantly.

Fred the Statistician

“The trend is utterly useless as a predictor — always.”
This is absolutely incorrect. In the absence of any other data, past trends are the BEST predictor of future values.
This entire article shows a complete failure to understand statistics. Statistics can be used to make predictions, based on certain assumptions. If those assumptions are correct, the prediction is MORE LIKELY to be correct. If the assumptions are wrong, then the prediction is LESS LIKELY to be correct. No where are guarantees made.
The correct prediction in the article would have been “Given the assumptions we are making (iid, normal, etc), we predict that it is most likely that there will be 110 buttons, give or take (a very large number).” No statistician would EVER say “There WILL BE 110 buttons.” Because statisticians actually understand statistics.
“We can’t know the future” is an utterly silly argument. Will the sun rise in the East tomorrow? I hope you didn’t say “yes”, because you can’t know the future! If I let go of this pencil I’m holding in the air, will it fall to the ground? I hope you didn’t say “yes”, because you can’t know the future!
Bah. The only thing this article shows is how many supposedly educated people can utterly fail to understand simple concepts. Statistical predictions are not Truth, they are well formed Guesses. And guesses can be wrong. Anyone, like the author, that fails to understand this should stay well away from statistics.

Geoff Sherrington

This analysis fails to mention response times when you are dealing with a time series. If you consider response times, you are left with a strange result, that although a trend cannot be used to predict a future term, it can estimate unpredictable future terms with various degrees of lack of confidence.
An earlier blogger has used the case of planetary orbits. These have a long response times in regard to position relative to a datum. If I am given a graph of daily positions of a planet, I can estimate a position in the next day to a high degree of accuracy.
If, however, I am counting buttons, which my wife can throw away in a second, then I would not try to estimate ahead. The response time can be very short, like her temper can.

observa

@Fred the Statistician-
“This entire article shows a complete failure to understand statistics. Statistics can be used to make predictions, based on certain assumptions. If those assumptions are correct, the prediction is MORE LIKELY to be correct. If the assumptions are wrong, then the prediction is LESS LIKELY to be correct. No where are guarantees made.
That’s what the article points out- ie the bleeding obvious and it’s just reminding us all about the implicit assumptions underlying the sun rising tomorrow or the pencil dropping. We answer ‘yes’ and tend to ignore the qualifying assumptions underlying it.

Mike McMillan

Here’s a Revkin interview that’s almost reasonable.

Other_Andy

Mark XR says:
October 17, 2013 at 8:23 pm
So, just because there has been a pause in rise of global mean surface temperature does not mean we can reject the physics of global warming? Rats.
If you mean, post modern IPCC physics, that CO2 is the key driver of modern climate change, yes we can. Historical evidence has all but debunked the correlation between global temperature and CO2.

Leonard Lane

Steve M. from TN says:
October 17, 2013 at 7:43 pm
jim2 says:
October 17, 2013 at 7:26 pm
He will have between 0 and a googleplex of buttons. I’m 95% certain of it.
I’m 100% sure he’ll have between 0 and infinity (inclusive).
Not exactly. He will have between 0 and a countable infinite number of buttons, all terms of the infinite series being positive even numbers (unless he goes “broke” and winds up with zero). He will not have any odd number of buttons.

albertalad

You are correct – stats are only as good the information entered. And as always there are as many economists predicting the future as there are economists. One may as well watch Star Trek. No matter what anyone says here – whatever their forward programing it is a guess based on a past that may or may not be anywhere near accurate. Regards to inventory – how often shelves are without a product is common. If climate is an indication then it is no wonder why models are next to useless. What we do know is there are ice ages and interglacial periods. The rest our present understanding is next to nil – like the weather.

Scarface

What ever happened to ‘The trend is your friend’?
Certain kinds of trading rely heavily on this principle.

How do you know that BC is reporting the number of buttons accurately? What if his buttonometer hasn’t been calibrated. What if he is in an Urban Button Island. Maybe he adjusted the count to conform to the Summary for Button Makers.
Heh – there haven’t been any buttons for 17 years.

thingadonta

Gary Larson’s The Far Side’s 4 personality types.
1. The glass is half empty.
2. The glass is half full.
3. Half empty!..no wait.. Half full.!….no wait
4. Hey I ordered a cheeseburger.!
1. The buttons are decreasing.
2. The buttons are increasing
3. We don’t know whether they are increasing or decreasing, or staying the same.
4. Unless we reduce our button consumption, things will get much worse than we thought, and the world will end as know it.

DrJohnGalan

In the UK, I think the few minutes spent reading this article would be of great benefit both to Ed Davey (if he is capable of understanding it) and Sir Mark Walport, as well as several members of Parliament’s Energy and Climate Change Committee.
“Back to basics” on the use of statistics, with a reminder of the number of potential interconnected variables that might contribute to a model of global climate, should get them to pause for thought. Clearly this will not happen, because we are dealing with a religious type of belief (which history shows us is all about the political power to soak the poor), not statistics or science.

A Crooks

If I could be bothered reading all this, I think I would disagree.
Sure, the trend is a product of the data, but if the data is measuring something which has inertia we can expect data to continue on trend. Given that global weather has inertia one can expect trend to continue. If there is a sixty year cycle that is obvious over one hundred and fifty years of data one would expect a certain amount of time to pass before that cycle decays and a new cycle / trend is established. There is a long trend out of the Little Ice Age. That will not just reverse. The value of the trend is that it enables you to see when the data is starting to stray off the trend. This is something the IPCC never worked out. They were silly enough to draw a straight line through a cherry-picked bit of the data and call it significant when the bigger picture screamed something else. Their model is broken and they just dont like the reality imposed by the real data (trend).
If you think this is wrong – consider investors in stock markets whose whole living is dependent on picking trends in data and then working out when the data has gone off trend in a significant way. Like climate researchers and climate , each investor does not need to know the detailed internal workings of a company to be able to see when the data is straying off trend and a buy/sell decision be made. I dont need to know anything about the science of climate change to see its drifted off the IPCCs trend and the whole AGW/CO2 business is now a SELL option. There was an excellent post just the other day on this very topic.
To say that you can never predict the future based on a trend leaves us saying we can never be sure the sun will come up in the morning. This may be true but it is an excellent working hypothesis.

Gareth Phillips

Past performance etc does not guarantee future returns, is a caution given to any small investor in the UK. However past behaviours do indicated the likelihood of future behaviours. When working in forensic mental health we used various tools to assess the risks patients with mental health problems who were also offenders posed to society. These assessments were in the main pretty useful. These type of assessments can also extend to how a person will react to a given piece of research. If they have always rejected a certain stance in the past, they are likely to do so again whatever the quality of the study. This underpins my belief that there is one heck of a lot more subjectivity in Climate science than we recognise, and a stance that makes me unpopular with both the consensus and skeptic side, many of whom like the reassurance of definite indisputable facts. From a personal perspective I see two general trends in opinions, but millions of different specific views on the issue, a number usually correlated to the amount of people who are asked.

LevelGaze

About trends –
Yes, the trend is indeed your friend.
But wise stockbrokers know that a trend continues until it doesn’t.
And, @ Observa
Here in Australia all our swans are black.

Kelvin Vaughan

Definition of Trend:
1. The general direction in which something tends to move.
2. A general tendency or inclination.
3. Current style.

Kelvin Vaughan

observa says:
October 17, 2013 at 10:38 pm
Young Antounie Caen’s schoolteacher-
Swans are white, always have been white and always will be young Antounie, so get that through your thick head and pay attention in class meboy.
He forgot to add “The science is settled”!

If you think this is wrong – consider investors in stock markets whose whole living is dependent on picking trends in data and then working out when the data has gone off trend in a significant way. Like climate researchers and climate , each investor does not need to know the detailed internal workings of a company to be able to see when the data is straying off trend and a buy/sell decision be made. I dont need to know anything about the science of climate change to see its drifted off the IPCCs trend and the whole AGW/CO2 business is now a SELL option. There was an excellent post just the other day on this very topic.