First, a bit of a primer. Wikipedia describes a random walk is a mathematical formalisation of a trajectory that consists of taking successive random steps. For example, the path traced by a molecule as it travels in a liquid or a gas, the search path of a foraging animal, the price of a fluctuating stock and the financial status of a gambler can all be modeled as random walks. The term random walk was first introduced by Karl Pearson in 1905.
From the Financial Post: A 2011 study in the Journal of Forecasting took the same data set and compared model predictions against a “random walk” alternative, consisting simply of using the last period’s value in each location as the forecast for the next period’s value in that location.
The test measures the sum of errors relative to the random walk. A perfect model gets a score of zero, meaning it made no errors. A model that does no better than a random walk gets a score of 1. A model receiving a score above 1 did worse than uninformed guesses. Simple statistical forecast models that have no climatology or physics in them typically got scores between 0.8 and 1, indicating slight improvements on the random walk, though in some cases their scores went as high as 1.8.
The climate models, by contrast, got scores ranging from 2.4 to 3.7, indicating a total failure to provide valid forecast information at the regional level, even on long time scales. The authors commented: “This implies that the current [climate] models are ill-suited to localized decadal predictions, even though they are used as inputs for policymaking.”……
More here: http://opinion.financialpost.com/2012/06/13/junk-science-week-climate-models-fail-reality-test/
h/t to WUWT reader Crispin in Waterloo
Previously, WUWT covered this issue of random walks here:
Is Global Temperature a Random Walk?
UPDATE: The paper (thanks to reader MT) Fildes, R. and N. Kourentzes, 2011: Validation and forecasting accuracy in models of climate change. International Journal of Forecasting. doi 10.1016/j.ijforecast.2011.03.008
and is available as a PDF here
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Climate Models: If a frog had wings,……….
One of the reasons that climate models fail is that they include CO2 which in fact lags temperature rather than being useful in a forecasting model for temperature, rain, wind etc. The so-called climate scientists basically have no understanding of heat transfer and reaction kinetics. As Willis has been showing they need to look at heat from the sun, and clouds in their various forms. They also need to look at the chemical reactions at the top of the atmosphere such as ozone formation.
Heh — the magenta random walk produced a pretty decent hockey stick…
One has to admire McKittrick’s clear and direct style.
I like this long-term precipitation record – Central England & Wales, 1766-now:
http://climexp.knmi.nl/data/pHadEWP_monthly_qc_mean1_1766:2011.png
There is some multidecadal oscillation, but nothing exceptional whether it was LIA or modern WP.
This surprises me – and probably most people here – not one bit.
As long as the input assumptions are flawed, all that the climatologists will manage with larger and more expensive supercomputers (as demanded by the UK MET) will be to compound their errors.
Surely a diagram on a flat surface is in two dimensions?
the models are screwed up…we have been saying that for a long time…there are too many factors…many poorly understood…co2 is already absorbing all the infrared it can…adding more will not make any difference…these climatologist guys dont seem to understand basic physics
For some time I have been unable to understand how the UK Met Office models can predict climatic conditions decades in the future, when they are admitted to be of “low skill” (by Richard Betts) in predicting the weather / climate over periods only weeks or months in the future. I asked Richard for references to the modelling procedures and he kindly provided several on various threads at Bishop Hill. This raised more questions and I finally emailed the Met Office with the following:
“I would be grateful if you could let me know if you think that it is reasonable practice to use numerical models for multi-decadal forecasts that:
A. Use low pass filters “chosen to preserve only the decadal and longer components of the variability” ( http://www.geosci-model-dev.net/4/543/2011/gmd-4-543-2011.pdf (quote from page 21), and,
B. Accommodate errors that cause instabilities “by making periodic corrections, rather
than by fixing the underlying routines (http://www.cs.toronto.edu/~sme/papers/2008/Easterbrook-Johns-2008.pdf (quote from page 7).”
The Met Office kindly replied with the following answers:
“(A) The models themselves do not use low pass filters. Indeed they simulate weather on timescales of minutes. Low pass filters are used to analyse the model output in order to focus on timescales of interest.
(B) The mathematical equations describing the evolution of the atmosphere and oceans cannot be solved analytically. Instead they must be solved numerically using computers. This process frequently involves a compromise between accuracy and stability. During model development much research is undertaken to find the numerical schemes that provide the best accuracy whilst minimising instabilities. At the Met Office the same numerical schemes are employed for weather and climate predictions, and the performance of the forecasts are continuously assessed across a range of timescales from days to decades.”
I am still struggling to find an answer to my original question, and perhaps my questions to the Met Office were the wrong ones. Is there anybody here who understands how numerical models can be “low skill” over short timescales but could possibly be accurate enough to justify massive intervention over multi-decadal timescales?
LevelGaze: “This surprises me – and probably most people here – not one bit.”
Yeh no. I’m not simply surprised but mortified. It’s one thing to hold that simply gluing together words ‘climate’ and ‘science’ doesn’t mean there’s much of either in ‘climate science’. It’s quite another to state that the “consensus science” is so stupidly distant from a random walk, on the wrong side of things, that we’re beyond Jonestown and that we’ve gone beyond Ludicrous Speed.
Each of these models, which the article refreshingly notes, are a scientific theory about the climate. And they are when empirically tested make errors more than twice that achieved by a monkey throwing darts. Under any notion of science every one of those models is a miraculously falsified theory.
And of course it serves to note that the average stock broker does slightly worse than an actual monkey throwing darts at a dartboard — for those that remember the MonkeyDex. Slightly worse. And it’s a felony for stock brokers to make too much of their own prognostications without a disclaimer to the effect that they are not Delphic Oracles, Palm Readers, or merely reliable. And that’s just for ‘slightly worse’.
So while I’m unsurprised that the models are terrible, I’m utterly aghast that they are so grotesquely biased as to make the static on an empty television channel the acme of rigorous statistical predictions in climate.
Yes, but only ONE of the dimensions is length. The other is time. Think of a bead on a string. At time t=0 the bead is at zero, at time t=1 the bead is at -1 (say), at t=2 the bead is at +5, and so on. The graphs shows the position of the bead on the string as a function of time.
Oops. Brain now switched on.
Could someone share the title of the paper this was based on? I’ve looked through the 2011 paper in the forecasting journal but couldn’t find any likely titles. Perhaps I just missed it.
Cheers!
Doesn’t surprise me in the least.
All the climate models do is project the modellers belief in the net forcing of various factors, primarily CO2, forward into the future. That the models do worse than random chance is proof that some or all of the believed forcings are wrong, and specifically the believed CO2 forcing is wrong.
No other conclusion is possible, except perhaps serious programming incompetence by the modellers.
In a nutshell a random walk has a finite chance of being correct but if you start with a wrong model it will always be wrong.
This is the best ever article on climate modelling (although Ross McKitrick’s article just published comes a close second).
Please read it carefully – it’s worth it –
http://climateaudit.org/2011/05/15/willis-on-giss-model-e/
@ur momisugly Roger Longstaff
“Is there anybody here who understands how numerical models can be “low skill” over short timescales but could possibly be accurate enough to justify massive intervention over multi-decadal timescales?”
No, because this is not remotely feasible. The climate is a chaotic system so a tiny variation in an initial condition will result in entirely unpredictable (or random) outcomes.
Roger Longstaff says @ur momisugly June 14, 2012 at 1:03 am
Hi Roger,
Pretty certain that what RB has in mind is that (A) short term random noise (white noise) prevents models from making accurate forecasts over weeks and months and (B) the white noise is averaged out over climate time scales (decades) and hence the models should end up providing reasonable average behaviour over the longer time scales.
The reason that (B) is wrong is simply that the noise in the system does not average out over the decadal time scales. Over these time scales, the pink noise mentioned in the “Is Global Temperature a Random Walk?” link begins to dominate. Models are not currently able to simulate this behaviour, which means that their use in decadal projections is unlikely to be useful globally, let alone regionally.
It’s not too surprising; a random simulation can be right some of the time, but a model rigged to be wrong will consistently be wrong.
GCMs are like communism, they require global venue to succeed as their supporters say; but fail miserably at a local level as reality states.
This is great news! With this breakthrough in modeling, we will be able to improve climate forecasting while at the same time reduce the amount of computational performance needed, making for more sustainable modeling. Simply replace the current climate models with random walk models, do an ensemble run and average over it.
I can’t wait to see the headlines:
“Scientists baffled – New Climate Model predicts no change in temperature ever!”
The UK. Met. Office have yet to learn that when you have dug yourself into a hole you stop digging.
Their reply to Roger Longstaff is typical of QUANGOs which the Met Office has become. At least when it was officially under RAF control it did apologize for errors it made.
For non UK readers a QUANGO is a Quasi Non Governmental Organization that is formed to advise the government and in doing so get paid money. The Met Office has dreams of operating the worlds biggest climate computer so needs to scare the government into paying more money. It can’t be bothered with members of the public, who actually provide the money through taxation, asking what it considers stupid questions.
Keep trying Roger.
People here waste too much time in armchair speculation on why climate models can’t work “in theory”. The key issue is validation. Why should I believe what this model tells me? What is the evidence for it’s efficacy? If I have to evaluate the merits of a complex surgical procedure I want to look at the study results. I don’t care if the surgeon is the world’s greatest or that medical science is amazingly good, or if there is a consensus on something related to it, or any other non sequiturs of that type.
Random walk? What random walk? The yellow line and the black line OBVIOUSLY prove my **ULTIMATE THEORY!**
If you people just can’t see that, well, I hope Schroedinger’s Cat bites you!
– MJM
Back to discuss about random walks after these ad’s….
‘the financial status of a gambler ‘, I spend 4 pounds a week on the lottery and always lose (using this years figures as I ain’t won a penny) and my friend lost the £120,000 given to him when his farther passsed away plus the £40,000 he stole from work, so I promise you gamblers don’t generally have random walks just declines and even when they do win these are generally smoothed out by, yep, the loses.
Welcome back…..
Yes,, this is exactly how they are supposed to work.
If you have two climate models, one says ‘rains and floods will increase’ and another that uses the same data says ‘droughts and hot weather will increase’ then depending on the weather at the time you just refer to the model that got it correct and blame it on climate change. I thought you all knew the science behind AGW ? /sarc