First, a bit of a primer. Wikipedia describes a random walk is a mathematical formalisation of a trajectory that consists of taking successive random steps. For example, the path traced by a molecule as it travels in a liquid or a gas, the search path of a foraging animal, the price of a fluctuating stock and the financial status of a gambler can all be modeled as random walks. The term random walk was first introduced by Karl Pearson in 1905.
From the Financial Post: A 2011 study in the Journal of Forecasting took the same data set and compared model predictions against a “random walk” alternative, consisting simply of using the last period’s value in each location as the forecast for the next period’s value in that location.
The test measures the sum of errors relative to the random walk. A perfect model gets a score of zero, meaning it made no errors. A model that does no better than a random walk gets a score of 1. A model receiving a score above 1 did worse than uninformed guesses. Simple statistical forecast models that have no climatology or physics in them typically got scores between 0.8 and 1, indicating slight improvements on the random walk, though in some cases their scores went as high as 1.8.
The climate models, by contrast, got scores ranging from 2.4 to 3.7, indicating a total failure to provide valid forecast information at the regional level, even on long time scales. The authors commented: “This implies that the current [climate] models are ill-suited to localized decadal predictions, even though they are used as inputs for policymaking.”……
More here: http://opinion.financialpost.com/2012/06/13/junk-science-week-climate-models-fail-reality-test/
h/t to WUWT reader Crispin in Waterloo
Previously, WUWT covered this issue of random walks here:
Is Global Temperature a Random Walk?
UPDATE: The paper (thanks to reader MT) Fildes, R. and N. Kourentzes, 2011: Validation and forecasting accuracy in models of climate change. International Journal of Forecasting. doi 10.1016/j.ijforecast.2011.03.008
and is available as a PDF here
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
In my view any reconstruction of the global temperatures may encounter problems unless two hemispheres are considered separately.
Recently I looked at the Northern Hemisphere, more complex but still possible to achieve a reasonable degree of correlation using the known natural parameters as shown here:
http://www.vukcevic.talktalk.net/NH-Recon.htm
Note: all variables are detrended.
Despite possibility that the natural oscillations in 1960-70s and 2000-2011 could have been suppressed by man made factors the longer term values or trends are unaffected.
If this is correct then both the sulphates and CO2 effects are only short term.
Reasons for anti-phase in the late 1930s are not as obvious.
(p.s. if there is any interest in method and parameters used, I may put more info on my website some time soon, if there isn’t I’ll do it anyway).
re: Mindert Eiting 4:23 AM 6/14/12
I suspect this is wrong. First no human is going to be totally clueless of a given subject. Say the subject was the nomenclature used in modern dance. I don’t think I’ve ever read anything about it, but I’ll bet I’d score well above 50% in any actual test constructed in the English language. This is because any human test of knowledge is expressed using the common knowledge embedded in the language it’s constructed in. Depending on whether the test is constructed to be as difficult as possible or to best test the actual knowledge of the test taker, the average score might be 17 or 25 say.
The main climate model forecasts over the years versus the temperature observations (up to April and May 2012).
http://img441.imageshack.us/img441/5987/climodfrcstsobsmay2012.png
A fan of *MORE* discourse says:
June 14, 2012 at 6:22 am
“Yet global climate changes are constrained by strict conservation of mass and energy and global increase in entropy, and thus *CAN* be predicted”
If you look at reference B from my first post you will find that the models need to be “corrected” during the numerical integration – the specific reason being given that they violate conservation of mass. Any model thjat needs to be “corrected” in order to restore conservation of mass can not provide useful information.
Philip Richens says:
June 14, 2012 at 6:36 am
Hi Philip,
A similar point was made on Tamsin’s blog: “On filtering, you were given a clear answer in the Met Office’s reply: the low-pass filter described has nothing to do with the workings of the model as it is running. It’s a technique used to aid analysis of a specific climatic phenomenon after the model run has completed.”
My reply was that any filtering “to aid analysis” must result in loss of information, given that there is no a priori knowledge of the signal.
Do you agree?
mt says:
June 14, 2012 at 4:47 am
That’s not a random walk, that’s a persistence model, a.k.a a straight line.
========
every walk starts from the position of the last step. However, for it to be a straight line, the direction of the next step must remain constant. If the direction changes randomly at each step, it is a random walk.
In Australia they found it was more accurate to simply forecast yesterday’s weather today than to rely on the zillions of dollars they were spending on weather forecasting.
In climate, it is much more accurate to forecast that the climate will be pretty much what it was 60 years ago than it is to rely on the IPCC models. This decade will be like the 1950’s and the 1890’s. We are at the beginning of a cooling period that will last 20 more years, just like the 1950’s and the 1890’s. During the next 20 years, until about 2035, the warming we have experienced since the bottom of the LIA will remain halted, and temperatures will be stable to declining.
After than time, warming should resume. However, there is plenty of evidence that temperatures overall have been declining for 8000 years, and we are likely due to start a rapid slide back into ice age conditions in the not too distant future. At which time you can pretty much be certain that real estate prices outside the tropics with collapse. The money that went to prevent global warming? That money will be gone. Instead you will then be taxed to prevent global cooling. None of which will have any effect except to line some else’s pockets.
Wonder what these folks would say about the models:
http://statistics.ku.dk/
Sort of reminds me of an argument between my physics professor and philosophy professor in college. The physics professor was trying to prove a point that (essentially) physics had all the answers and how it was superior to philsophy. To prove it he said to the philosophy professor “you see that pen you’re holding: If you dropped it I could tell you everything about it…it’s velocity at every point in time, it’s momentum, how far it falls, how hard it hits, how high it bounces, etc.” to which the philosophy professor said “but can you tell me if and when I’ll drop it”?
@RockyRoad read the paper. As far as I can tell, they’re giving scores based on looking at future data (2007-2017 and 2007-2027). How is it possible to say what model is “right” and “wrong” versus data that doesn’t yet exist?
Please note there are four papers: the original paper, two critical commentaries, and the authors’s postscript. This is a welcome methodological critique of models, not of the science behind them.
From the original paper’s conclusions:
“…there is no support in the evidence we present for those who reject the whole notion of global warming: the forecasts still remain inexorably upward, with forecasts which are comparable to those produced by the models used by the IPCC.”
From the postscript:
“Climate change is a major threat to us all, and this requires the IPCC community of climate modellers to be more open to developments and methods from other fields of research.”
Yeah, but if you take a bunch of them and average them together it’s better, right? Oh, I know what the problem is here; These guys aren’t climate scientists, so it’s just too complex for them to understand. I’ll bet they forgot to turn some of the data upside (you have to do that sometimes).
The GCMs are run almost continuously trying to predict weather. They are in general disagreement out after a few days but they keep trying. I have been amazed at their persistence over the years. At 10 days out their guesses at what weather will happen becomes laughable.
I started paying attention to this in 2006 and so far there has been no improvement. They just keep trying. You can become aware of much of this effort over at weather underground where a bunch of modeler kids try to fathom their rationale for hurricanes and tropical storms. At 3 to five days out they are all pretty accurate but by 10 days out their really fail big time. It remains very disappointing after seven years looking for improvements.
This post and comments include statements with the terms random and chaotic with regard to weather and climate. I wonder who gets to define the terms, outcomes, and miss or match of the results? For example, if the weather forecaster looks at a computer output and it says the High Temp for tomorrow will be X.23 and then posts X and then the actual High comes as X +2, is that forecast considered wrong? In the strictest sense, if the actual High was measured as X.23 but the forecast was X — is it still wrong? On a different level, if, for 999 times out of 1,000, tomorrow (June 15) will be more like today than if will be like January 15, 2013, is it useful to think of weather as random?
In a similar sense, with climate there seems to be a sense of sameness involved. Sometimes the terms average or normal or climatology enter into the discussions. That a person or model, or tea leaves, cannot say exactly what the high temperature and total precipitation will be for June 15, 2020 may not constitute a FAIL in the sense of climate sameness. Maybe the sameness should be compared to the entire month and not one day. And how close should the numbers be to declare “same/not-same” statements?
If June 2020 will be more like June 2012 than it will be like January 2013, is the climate chaotic? I cannot recall a June when this comparison – June to June, versus June to January – hasn’t produced the expected result. Yes, I know. There have been instances when it was cold and snowed in June (southern hemisphere folks, please don’t take offence at this NH statement) — but this is within the meaning of “sameness” in my use of the term. I don’t consider that every day has to be very close to the averages for the month for June to be “June” and not some surprising anomaly.
Consider the statements in the opening paragraph: “ . . . the path traced by a molecule as it travels in a liquid or a gas, the search path of a foraging animal, . . . ”
One has to define these in a way to make them both “random walks” because the animal is not random in the step-by-step fashion while actually foraging. A horse, say, almost never backs up while foraging in a pasture. They tend to defecate from one end and take in food at the other and they have an aversion to ingesting plant material they have just soiled. To call their movement a “random walk” requires a definition that eliminates the step-by-step movements of the horse. Perhaps, one only plots the position once every half hour. Anyway, a step-by-step movement of a molecule in a gas does not involve the definitional issue as does the foraging horse.
The people who define the path of a molecule and a horse as equivalent must also be the sort of people who can define weather and climate as random and chaotic. The phrase “The devil is in the details” seems appropriate.
This is my bread and butter — random numbers and stochastic (e.g. Markov chain) models. I can do no better but to refer people to Koutsoyannis — here is a brief post on CA that indicates McIntyre’s opinion of him:
http://climateaudit.org/2006/01/05/demetris-koutsoyannis/
In a nutshell, Koutsoyannis asserts that climate is best described by Hurst-Kolmogorov statistics, which is basically a biased random walk. The decadal transitions are locally all nearly random and discrete, with nearly unbiased noise persistent between transitions. The direction of the transitions is nearly random, modulated by truly long term trends and non-Markovian dynamics.
Personally I think he is brilliant, and brings an extraordinary and much-needed degree of statistical competence to a field (climatology) that operates under the fantasy that simple linear physical models can capture and underlying nonlinear chaotic process. Koutsoyannis builds moderately successful models USING statistics that are closely related to a random walk.
IMO the right way to approach modeling is via e.g. a Langevin equation — a stochastic integral equation solution to a set of coupled nonlinear ODEs with random terms that model noise, where the noise terms are LARGE (possibly dominant) and not SMALL the way they are now. That’s HK statistics in a nutshell. It’s not that there isn’t long term dynamics and trends driven by physics, it is that resolving this long time scale trend from the short and meso-scale noise is nearly impossible because the noise is actually an order of magnitude greater than the signal. Under these circumstances, understanding response functions to local perturbations of the underlying dynamics is nearly impossible — detecting the “global warming signal” when the noise is an order of magnitude greater on decadal time scales, for example. Doubly so when the underlying dynamical model being “tested” is almost certainly not correct within a factor of three to five…
rgb
“…The mathematical equations describing the evolution of the atmosphere and oceans cannot be solved analytically. Instead they must be solved numerically using computers. This process frequently involves a compromise between accuracy and stability…”
If the models uded by the Met are a compromise between accuracy and stability, then that stability must be the proverbial “immovable object”.
We’ve already seen their accuracy…
And, as a reply to mt (June 14, 2012 at 4:47 am)
“…Assuming we’re talking about the persistence model above, all the test is doing is scoring the amount of change from the baseline. This says nothing about what is right or wrong. This is at best misleading to try and judge the performance of climate models using a test like this…”
And you could say the same about the anomaly charts they pass off as “global temperatures”.
All they do is show the amount of change from their chosen baseline (averaging period). Not all charts use the same baseline, or even use the same reporting stations.
So those charts say nothing about what’s “normal”, or if we’re above or below that. This is at best misleading to try and judge “rise” in GLOBAL temperatures (with values less than a degree for most) using charts like this.
Thanks to all of you I am beginning to get a clearer picture of how those on the AGW bandwagon can see things so clearly where, we sometimes we shake our heads and wonder how they can be so dense.
It’s simple really. If you add one little routine to the random walk that discards all random walks that do not come close to matching the magenta random walk, since after all that one HAS to be right, (since it comes closest to tracking the relentless upward climb of that, that, nasty, evil mosquito of a compound CO2 from the year 18–, no 19whenever) then you can use those logically, correctly selected random walks, since by inference they modeled it rightly, to randomly walk into the future, and voila! within a hundred years we have . . . . . wait for it . . . GLOBAL WARMING! wait no? . . . . ok CLIMATE CHANGE!! See, I told you so! . . .
/sarc off.
“””””…..Helen Ap-Rhisiart says:
June 14, 2012 at 12:43 am
Surely a diagram on a flat surface is in two dimensions?…..”””””
Not so surely Helen. If you do the exact same random walk yourself out in some safe place so you won’t step off a cliff; how about aschool playing field; you step forward or you step backward, depending on whether the referee calls heads or tails. You can do this till the cows come home, and you will never move sideways. so you either make it to your goal line or you do a wrong way Corrigan score at the enemy’s goal, but you never move sideways.
If Anthony plotted your path exactly as you walked it, you would see all of your steps on top of each other and you couldn’t tell anything, except the length of the line.
Time was invented to preventing everything from happening all at once, so the bottom scale on Anthony’s chart is just to time separate the moves so you can see them all. So it is one dimensional steps. In a two dimensional case, you would need to step sideways sometimes depending on some random signal.
to which the philosophy professor said “but can you tell me if and when I’ll drop it”?
To which the physics professor then replied, “Not even you with all of your philosophy and the pen in hand can do that — but at least I understand why.”
Who is it, after all, that drops the pen?
rgb
“””””…..ferd berple says:
June 14, 2012 at 7:07 am
Philip Richens says:
June 14, 2012 at 2:27 am
Pretty certain that what RB has in mind is that (A) short term random noise (white noise) prevents models from making accurate forecasts over weeks and months and (B) the white noise is averaged out over climate time scales (decades)
=========
Correct, the climate models make the assumption that climate is normally distributed (constant mean and deviation), that the errors plus and minus average out over time……”””””
Well white noise may average out to zero after long time scales; but only in mathematical models, which have a Gaussian distribution. Physical systems don’t have such an ultimate error distribution. Most of them eventally show the presence of 1/f noise, where the amplitude of an error or step, can grow without limit, but at an occurrence frequency that is inversely proportional to the size of the error or step. One can make the argument that 1/f noise is a consequence of Heisenberg’s Principle of Uncertainty. Maybe the Big Bang, was simply the bottom end of the 1/f noise spectrum.
So in real physical systems, noise does not average out to zero no matter how long you wait, because there is always the chance of a step much greater than anything previously seen. And no; 1/f noise does not violate any thermodynamic limits, such as the total power or energy growing without limit, which it would in a white noise system. It is simple mathematics, to show that with 1/f noise each octave of frequency range contains the same amount of power as any other, no matter how large the amplitude gets.
A 2011 study in the Journal of Forecasting took the same data set and compared model predictions against a “random walk” alternative, consisting simply of using the last period’s value in each location as the forecast for the next period’s value in that location.
That isn’t a ‘random walk’!
Not to be too pedantic, but Random Walks do not model the motion of foraging animals very well.
Hey, Anthony!
That’s a nice graph you have there. Very evenly splayed out. Did you pick it yourself? 🙂
OK, just teasing. I know it’s only for illustrative purposes.
Still, I thought it looked suspiciously ‘good’ with that attractive symmetry and those nice convenient outliers either side at about the 2 SD mark at the end, so I ran a few for myself without cherry picking — a straight sequence of runs — just to check my suspicions. Tut tut, Anthony. 🙂
They’re not as pretty as yours but you can see the results here: http://i46.tinypic.com/10rplki.png
Go on, tell us. How many runs DID you try until you found that nice one?
Too much exposure to the antics of those delinquents over at RealClimate might be leading you astray. They’re a very bad influence. You need to be careful. I’d stay well away from them from now on if I were you. 🙂
The climate models are absolute bullshit. And they have failed, all of them. And the scare-mongers predictions of doom, going back over 4 decades, have failed, all of them. And the foundation of trumped up AGW theory, the CO2 / temperature correlation, has been debunked (see algor repeat the lie: http://www.youtube.com/watch?v=WK_WyvfcJyg). The hockey has been shown to be a fabrication. There is nothing wrong with the climate.
What amazes me is just a few short years ago nearly everybody seemed to be going along with the scam, even conservatives. Now skeptic Senator Inhofe’s position on climate change is the “new normal,” and he is leading the way: http://epw.senate.gov/public/index.cfm?FuseAction=Minority.Blogs&ContentRecord_id=eb26d140-802a-23ad-4a8f-cc2a20647095.
But, amazing yet so sad quote from the link, Inhofe said: “I was all alone starting in 2001. When you’re an army of one you don’t get much attention.” How was it that the leftist econuts and their liberal political allies duped conservatives, virtually across the board?? Someone needs to tell the story.
Roger Longstaff @ur momisugly June 14, 2012 at 7:33 am
I think the problem is that your question asked whether they thought it was reasonable practice to use numerical models that use low pass filters for multi-decadal forecasts. According to both their answer and the paper you mentioned, a filter is used specifically on the model output. This means that even though filtering, smoothing, averaging etc will remove information from the model output, this will only happen after that output has been fully calculated by the model.
Thanks for pointing out the discussion on Tamsim’s blog – it is very interesting and Tamsin’s style is very nice and friendly. I notice that Judith Curry has a comment at June 14, 2012 – 3:49 pm, which is asking the questions that interest me, plus quite a few others that I should be interested in. Again, I think the crucial issue is the low decadal variability in the models.
John Ray quote from 2009 on the longest temperature record ever, that of Central England:
“It is clearly a random walk and any trend up or down is a statistical creation rather than anything real.”
http://www.stoptheaclu.com/2009/11/28/30129/
His entire series of news digest blogs presents a calmly welcome break from the statistics bickerfest that skeptics risk addiction to, starting with http://antigreen.blogspot.com and branching out to academic politics and medical report skepticism that oft ridicules epidemiology as statistical dredging.