Guest essay by Eric Worrall
The UK MET is celebrating that their new £97 million computer can now create slightly better 12 month predictions than tossing a coin.
The Met Office has shown it can predict the weather one year in advance with its new £97 million supercomputer.
Scientists believe they can now forecast with some accuracy the North Atlantic Oscillation (NAO) weather phenomenon in the Atlantic Ocean which largely governs the British winter.
The phenomenon forms because of low-pressure over Iceland and high pressure over the Azores in the Atlantic.
A large pressure difference brings increased westerly winds, cool summers and mild, rainy winters. In contrast when the difference is small there are fewer winds and Britain shivers in a big freeze during the winter months.
It was previously thought that the NAO was a chaotic system which could not be predicted but the Met Office has used a technique called ‘hindcasting’ to check whether their new supercomputer could have predicted past winters.
After looking back at weather data going back to 1981, they discovered that they could largely predict what the winter weather would have done for the past 35 years, a year in advance, with 62 per cent accuracy.
The abstract of the study;
Skilful predictions of the winter North Atlantic Oscillation one year ahead
The winter North Atlantic Oscillation is the primary mode of atmospheric variability in the North Atlantic region and has a profound influence on European and North American winter climate. Until recently, seasonal variability of the North Atlantic Oscillation was thought to be largely driven by chaotic and inherently unpredictable processes. However, latest generation seasonal forecasting systems have demonstrated significant skill in predicting the North Atlantic Oscillation when initialized a month before the onset of winter. Here we extend skilful dynamical model predictions to more than a year ahead. The skill increases greatly with ensemble size due to a spuriously small signal-to-noise ratio in the model, and consequently larger ensembles are projected to further increase the skill in predicting the North Atlantic Oscillation. We identify two sources of skill for second-winter forecasts of the North Atlantic Oscillation: climate variability in the tropical Pacific region and predictable effects of solar forcing on the stratospheric polar vortex strength. We also identify model biases in Arctic sea ice that, if reduced, may further increase skill. Our results open possibilities for a range of new climate services, including for the transport, energy, water management and insurance sectors.
Read more (paywalled): http://www.nature.com/ngeo/journal/vaop/ncurrent/full/ngeo2824.html
It will be fascinating to see whether the new forecast system delivers. A lot of models which can hindcast successfully don’t survive contact with reality.
Imagine building a model for predicting lottery wins. With enough effort your model could be coerced into accurately hindcasting past lottery results. But it is very unlikely your lottery model would be able to predict future draws. Fitting a model to a limited data set is often not the same thing as accurately modelling the physics behind the data.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Well, there’s always a good chance tomorrow’s weather is going to be like todays; so maybe next year’s weather will be like this year’s.
They didn’t mention what “62 per cent accuracy” actually means.
ahhh.. yes. The cursed recursive. Been to that dance in my arrogant youth, and suitably humbled, stood in awe at the elegance of it all, knowing I knew nothing.
https://notonmywatch.com/?p=679
To be fair, UKMET is one of the better models used to predict things like hurricane development, and its generally fairly accurate up to 4-5 days out.
Put that game – hurricane forecasting – shows the perils of simplified models. Often one or two models will generate completely different hurricane tracks. So its clear that subtle assumptions in models can make huge differences to the predictions.
Frankly we need a new way to deal with complex non linear phenomena. Treating them as linear in the limit ends up with huge maths and poor accuracy, or simplifying them into broader liner equations results in so much loss of accuracy that its pretty useless.
We need new mathematical tools.
To be fair, forecasting UK weather is a difficult task. The prevailing conditions are determined by swirling high and low pressure regions that roll in across the Atlantic, and the paths of these regions can vary sufficiently that they sometimes miss us altogether. Thus, the best the forecasters can do is to say that there is a risk of high winds, heavy rain or whatever associated with a particular weatherfront approaching from the Atlantic. They cannot say with any certainty that it will hit, at least not until it’s a few hours away.
Add to that, the media presenters want the forecasters to give very simple and direct advice to the public, but it isn’t a simple situation.
It’s a chaotic system ! That is why it is pointless trying to do anything more than look at existing weather patterns 24h ahead.
A small niche group have manage to build a lucrative career path out of making spurious long term “predictions” using big computers. In past time they would have used sheep’s entrails and been equally held in awe.
Society is currently more aligned to believe in tech than sheep’s guts but it’s all a form of witchcrafft.
Greg,
Tonight, my near neighbours, down the hill, had some lovely haggis.
That allows me the predict a general cooling here in London over the next two to three months.
Memo to HM Government:
I am about 20 % up to MO standard, already, so please send hardware cheque of 20% of 97 Million [incredible shrinking] pounds by return.
Auto
“With enough effort your model could be coerced into accurately hindcasting past lottery results. But it is very unlikely your lottery model would be able to predict future draws. Fitting a model to a limited data set is often not the same thing as accurately modelling the physics behind the data.
No it couldn’t as picking numbered random balls, is just that – random.
Impossible to *predict* hindcast or otherwise.
A model is a tool to learn.
So as in early NWP, it has limited, but it seems some, skill, via applying “…two sources of skill for second-winter forecasts of the North Atlantic Oscillation: climate variability in the tropical Pacific region and predictable effects of solar forcing on the stratospheric polar vortex strength….”
Such that they learned of “model biases in Arctic sea ice that, if reduced, may further increase skill”.
That’s how it works.
And these *forecasts* may be useful to:
“the transport, energy, water management and insurance sectors.”
They are not “Push(ing) Inaccurate Forecasts out to 12 Months”.
They are reporting “modest but significant skill” in their endeavour. That is all.
http://www.metoffice.gov.uk/news/releases/2016/winter-forecast-skill
Thanks for the link. Curiously the omit any meaningful reference to the paper in question. No pre-print , no doi , not even the title of the paper and the authors.
PR puff piece they are scared to put up for inspection?
You go the their own web site to git it from the horse’s mouth and find they present you will the wrong end of the animal in question.
“Our latest research builds on that gain in skill in predicting the coming winter, but also reveals, for the first time, modest but significant skill in predicting the phase of the NAO one year ahead. This is an exciting first step in developing useful winter climate predictions on longer timescales.”
This discovery was made possible thanks to increased Met Office supercomputer capacity. This allowed the team to increase the resolution of the climate model and test the retrospective skill of their forecasts over the 35-year period from 1980 onwards.
What ? “test the retrospective skill of their forecasts ” That is one of their top scientists speaking?
If you test hind-casts through your calibration period you are testing your calibration not the predicitive capabilities of your model.
If they do not understand that subtly they should be being payed to do computer modelling. If they do they are being deliberately deceitful in order to justify continued funding.
Take your pick on which one it is.
“Curiously the omit any meaningful reference to the paper in question. No pre-print , no doi , not even the title of the paper and the authors”
It was in the post from Worrell my friend:
Again….
http://www.nature.com/ngeo/journal/vaop/ncurrent/full/ngeo2824.html
Oh, and seeing as it’s you…
Here’s the paper in full:
http://sci-hub.bz/10.1038/ngeo2824
Does it still meet you definition of puffiness?
In the absence of any link to their alleged paper it’s difficult to evaluate exactly what they are claiming.
There’s no mention of 62% accuracy in that release. I’d be surprised if the Met Office are making any such claim – it’s just the Telegraph are misunderstanding the r value of 0.62 quoted in the graph.
“DePreSys3 hindcasts of the first winter (Fig. 1a) skilfully predict
the year-to-year variability in the observed winter NAO index with
a highly significant correlation skill of r =0.62 (p=0.001 using a
two-tailed Student’s t-test).”
“DePreSys3 hindcasts of the first winter (Fig. 1a) skilfully predict
the year-to-year variability in the observed winter NAO index with
a highly significant correlation skill of r =0.62 (p=0.001 using a
two-tailed Student’s t-test).”
Yes, that’s what I meant. The Telegraph reporter described the correlation coefficient of 0.62 as “62% accuracy”. This has obviously confused a lot of people here who are interpreting the 62% as meaning they are right 62% of the time and wrong 38% of the time.
I think the Telegraph are making another mistake as the 0.62 correlation is for the “first winter” forecast, that is the forecast made in November for the coming winter. The year ahead forecast had a (statistically significant) correlation of 0.42.
The Thames will freeze JAN – FEB 2017. Will their supercomputer confirm this before the freeze or after the fact?
The Thames will freeze Jan – Feb 2017. The NHS will have difficulty keeping up with the casualties.
From The Daily Telegraph article on this story:
“After the 2007 prediction of a “barbecue summer” ended in a soggy washout, the Met Office pointed out that its seasonal forecast offered only a 65 per cent chance of being right.”
So spending £97 million has reduced the chance of being right from 65 to 62 per cent. Some progress!
It would cost £167 million to reduce it further to 50% – that is the next step
Will not be long before the start bleating about upgrades…
I haven’t noticed much from Piers Corbyn lately. He claims to be accurate. He was able to pay the rent with a series of bets on the weather until the bookies cut him off.
Piers Corbyn – the older brother of Jeremy Corbyn
Yep, that’s the one.
Ha, ha, haha “But it is very unlikely your lottery model would be able to predict future draws.”
Not “very unlikely” but impossible.
Not impossible. A Lotto prediction program has the same odds of being right as the little old lady who goes down to the store each week to by Powerball tickets.
I wouldn’t bet on it. But hey, she does. ^¿^
I tell you that a simple laptop computer is as inadequate in predicting a lotto outcome as a super computer of a trillion dollars.
Sorry I initially missed the joke.
Now, if only they could tell us what ‘62% accuracy’ means in a weather forecast!
62% of a day is about 15 hrs. So if it rains sometime between 6am and 9pm a forecast of rain is 62% right?? Or does it mean that if rains more than an hour sometime between 6am and 9pm over 62% of the forecast area it’s counted as a correct fore cas?. Is a forecast of mostly cloudy today correct of it’s solid cloud between 6am and 9pm and totally clear overnight a correct one or is it correct if about 2/3 of the sky has clouds for 24hrs?
There is a huge difference in utility of a forecast when using such vague terms as 62% correct. Gusts of wind at 15 knots for 20 minutes every hour is not the same as 5knot breezes.
Well no, they haven’t. They’ve shown their £97 million supercomputer can ‘predict’ the past with the data from a year earlier less then 2/3 of the time. To show they can predict the future a year in advance they need to make a prediction for a year from now and then wait to see if they were right. Then they need to do it another half dozen times at least to show it wasn’t a fluke.
Typical headline inflation.
How many watts does a £97 million supercomputer require these days ?
Looked it up for you. UKMet says 2.7MW.
That’s much better than I expected .
About 2000 horsepower 24 7 365 .
Oh yeah? Well over here we’ve got the Old Farmer’s Almanac with an 80% accuracy rating. No government funding either.
Having cocked up over trying to forecast what our climate will be in 100 years time they have now decided to spend even more of our money to do the job they were employed to do in first place, forecast our weather with some degree of accuracy though stretching that ideology to one year hence with only a 62% chance of being almost correct is somewhat stretching reality. What is more to the point is that they are making this gambit before in fact their new computer has even been installed which leads me to believe that their declaration today is more in hope than expectation. This is only what they believe they can do and the chimps currently in place at Westminster are falling in line as they always do with any trumped up nonsense that allows them to spend our money on anything other than what it should be spent on to curry favour and appease the sad halfwits whose beliefs continue to mitigate against what is left of the single electron that flits between one mutant alarmist and another looking for its final resting place but maybe not before what is left of our environment has been destroyed by solar panels and wind turbines. I wish I believed in God.
Global anthropogenic SO2 emissions are being tracked very closely, and it should be possible to make an accurate estimate of the amount of reductions expected for the coming year.
Give me that number, and I can predict the average global temperature at years end with an accuracy of greater than 90% with my $1.00 calculator.(after accounting for the temporary effects of El Ninos, Recessions, La Ninas, and volcanic eruptions)
(This claim is based upon temperature changes that occurred due to SO2 reductions 1975-2011)
This information could probably also be used to make regional predictions.
What happened to the 97% standard? Anything less just isn’t Climate Science.
So, are they actually going to predict something or waffle about with weasel words? I’ll bet you anything that 97M will produce nothing but weasel words and a whole list of excuses.
It’s only a matter of time before that improbably accurate12 months hence forecast will be pushed to an absolutely accurate fifty years hence we are all going to fry forecast.
Just how is accuracy rated?
I would be happy with an accurate 5 day forecast. Let me know when that happens.
“Let me know when that happens”
http://www.cma.gov.cn/en2014/research/News/201509/W020150907602390847882.jpg
“Forecast skill is the correlation between the forecasts and the
verifying analysis of the height of the 500-hPa level, expressed as the anomaly
with respect to the climatological height. Values greater than 60% indicate
useful forecasts, while those greater than 80% represent a high degree of
accuracy. The convergence of the curves for Northern Hemisphere (NH) and
Southern Hemisphere (SH) after 1999 indicates the breakthrough in exploiting
satellite data through the use of variational data100.
http://www.nature.com/nature/journal/v525/n7567/abs/nature14956.html
The Met is attempting to predict the North Atlantic Oscillation — and in hindcasts, got it right 62% of the time.
Nothing wrong with that — and nothing wrong with the quite possible idea that the NAO is periodic, cyclical, and predictable from other knowable climate factors.
I bet that the best weatherman in England could do the same from experience alone — looking back at past winters and world weather patterns leading up to them. This kind of prediction is well within abilities of modern climate models — broad general patterns suggesting near-present future patterns.
“I bet that the best weatherman in England could do the same from experience alone — ”
Nope.
Nowhere near.
Toneb ==> Gee, the bar is pretty low….they only have to predict either “mild, rainy winter.” and “big freeze winter” and only need to get it right 62% of the time (a bit over the chance guess 50%).
In the US, some credit the Farmer’s Almanac with a near 80% success rate on similar predictions of winter conditions (mild or hard).
Might as well if CO2 is their only variable.