UK MET Office to Push Inaccurate Forecasts out to 12 Months

UK MET Office Exeter
UK MET Office Exeter – By Richard Knights, CC BY-SA 2.0, Link

Guest essay by Eric Worrall

The UK MET is celebrating that their new £97 million computer can now create slightly better 12 month predictions than tossing a coin.

The Met Office has shown it can predict the weather one year in advance with its new £97 million supercomputer.

Scientists believe they can now forecast with some accuracy the North Atlantic Oscillation (NAO) weather phenomenon in the Atlantic Ocean which largely governs the British winter.

The phenomenon forms because of low-pressure over Iceland and high pressure over the Azores in the Atlantic.

A large pressure difference brings increased westerly winds, cool summers and mild, rainy winters. In contrast when the difference is small there are fewer winds and Britain shivers in a big freeze during the winter months.

It was previously thought that the NAO was a chaotic system which could not be predicted but the Met Office has used a technique called ‘hindcasting’ to check whether their new supercomputer could have predicted past winters.

After looking back at weather data going back to 1981, they discovered that they could largely predict what the winter weather would have done for the past 35 years, a year in advance, with 62 per cent accuracy.

Read more: http://www.telegraph.co.uk/science/2016/10/17/met-office-can-now-predict-winter-weather-one-year-in-advance/

The abstract of the study;

Skilful predictions of the winter North Atlantic Oscillation one year ahead

The winter North Atlantic Oscillation is the primary mode of atmospheric variability in the North Atlantic region and has a profound influence on European and North American winter climate. Until recently, seasonal variability of the North Atlantic Oscillation was thought to be largely driven by chaotic and inherently unpredictable processes. However, latest generation seasonal forecasting systems have demonstrated significant skill in predicting the North Atlantic Oscillation when initialized a month before the onset of winter. Here we extend skilful dynamical model predictions to more than a year ahead. The skill increases greatly with ensemble size due to a spuriously small signal-to-noise ratio in the model, and consequently larger ensembles are projected to further increase the skill in predicting the North Atlantic Oscillation. We identify two sources of skill for second-winter forecasts of the North Atlantic Oscillation: climate variability in the tropical Pacific region and predictable effects of solar forcing on the stratospheric polar vortex strength. We also identify model biases in Arctic sea ice that, if reduced, may further increase skill. Our results open possibilities for a range of new climate services, including for the transport, energy, water management and insurance sectors.

Read more (paywalled): http://www.nature.com/ngeo/journal/vaop/ncurrent/full/ngeo2824.html

It will be fascinating to see whether the new forecast system delivers. A lot of models which can hindcast successfully don’t survive contact with reality.

Imagine building a model for predicting lottery wins. With enough effort your model could be coerced into accurately hindcasting past lottery results. But it is very unlikely your lottery model would be able to predict future draws. Fitting a model to a limited data set is often not the same thing as accurately modelling the physics behind the data.

1 1 vote
Article Rating
120 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
October 17, 2016 11:50 pm

The Met Office will not be pleased with that telegraph article. It over eggs the pudding somewhat.

CC Reader
Reply to  David Johnson
October 18, 2016 10:28 am

When I think of hind casting, this picture comes to mind. http://www.lifeisajoke.com/pictures388_html.htm

auto
Reply to  CC Reader
October 18, 2016 2:44 pm

And the famous ‘ We are in drought’ London bus in a downpour . . . .
http://randomlylondon.com/is-there-any-doubt-we-are-in-drought/
Auto – well aware that 62% accuracy should beat [I nearly wrote trump(!)] tossing a fair coin.
Not sure it beats – ‘same as before’ – even in the UK . . . .

John Edmondson
October 17, 2016 11:58 pm

To be fair the the UK Met Office, their claim that they can predict the state of the NAO with 62% accuracy is pretty impressive.
However, it will give no clue whatsover as to the detail of the winter’s weather.
Given the nature of the climate here in the UK, predicting with any degree accuracy at more than 5 days is while nigh impossible.
So, for that reason, I don’t get what the Met Office has to gain from this claim. They should stick to their short term forecasts, which are very good.

Stephen Richards
Reply to  John Edmondson
October 18, 2016 1:11 am

They are back-justifying their need for a new £97m computer, its that’s simple.

Alan the Brit
Reply to  Stephen Richards
October 18, 2016 3:31 am

Totally agree!

Frederik Michiels
Reply to  Stephen Richards
October 19, 2016 6:33 am

i guess they needed an upgrade so they could play the latest games in decent framerates? :-p

AndyE
Reply to  John Edmondson
October 18, 2016 1:19 am

But the figure 62 isn’t that far from 50, is it – which is chance!!

Reply to  AndyE
October 18, 2016 2:59 am

It’s even closer to 66%, which would be pretty impressive for a year in advance.

Greg
Reply to  AndyE
October 18, 2016 4:01 am

Statistical significance is not a simple comparison to coin tossing.

In a system with strong auto-correlation I doubt the 62% is random chance but it needs to assessed with more thought than tossing coins.

After looking back at weather data going back to 1981, they discovered that they could largely predict what the winter weather would have done for the past 35 years, a year in advance, with 62 per cent accuracy.

But this NOT hindcasting since this the calibration period for the model.
Do the same test for 1920-1945 or 1950-1975 and tell us how well it works out.
So what they really have shown is that having tuned to model to a certain period of data , even then it barely reproduce what is happening to better than random variations.

Reply to  John Edmondson
October 18, 2016 3:14 am

with 62% accuracy is pretty impressive
Suppose they are 100% wrong… How many years do we have to go out before we can invalidate the 62% accuracy? At least 3 years of Wrong, Wrong, Wrong before you can invalidate a success rate of 62% or better at 90% confidence.

Reply to  Stephen Rasey
October 18, 2016 11:39 am

I thought I remembered reading was if you predict tomorrows weather was going to be the same as today’s you ended up being right about 60% of the time. Guessing that might work year over year???

Reply to  Stephen Rasey
October 18, 2016 5:27 pm

I wonder how that 62% compares to the Old Farmer’s Almanac. There is a post in Wikipedia (from sources not on line)

John Walsh, University of Illinois Atmospheric Sciences professor emeritus, reviewed the accuracy of five years of monthly forecasts from 32 weather stations around the county and found 50.7% of the monthly temperature forecasts and 51.9% of precipitation forecasts to correctly predict a deviation from averages.[31][32]

So now we have to define how accuracy is defined. If you predict the weather from weekly averages over the past 30 years, are you accurate if you are near the mean? What if accuracy is defined as within the posted spread — and the posted spread is +/- one standard deviation. By statistical definition, you would be within the range only 68% of the time. And deviate from the range 32% of the time. (assuming normal distribution, which don’t apply to precipitation and temperature uncertainty.)
So let’s see if the UK supercomputer can do better than the Farmer’s Almanac under the same test of accuracy.

Ian H
Reply to  John Edmondson
October 18, 2016 3:24 am

I don’t get what the Met Office has to gain from this claim.

It is a worthwhile objective if they can figure out how to do it. Good on them for giving it a go. To a farmer just knowing generally whether to expect a mild wet winter or a dry cold one would be very useful. It could help determine which varieties of crops to plant or how much feed to buy in.

Reply to  Ian H
October 18, 2016 5:55 am

Ian H, various copies of different Farmer’s Almanacs have been providing that “need” for more than 100 years.

jorgekafkazar
Reply to  Ian H
October 18, 2016 5:15 pm

Good on them?? It’s only good if it were cost-effective. Spending $97 million to emulate a coin toss gets no kudos from me. Whom the gods would destroy, they first drive mad.

QV
Reply to  John Edmondson
October 18, 2016 4:03 am

“which are very good”
What do you base that on?
My own analysis of the next 24 hour web forecast for the nearest MO station to where I live (Albemarle) was that MO forecasts were at best 37% accurate.
Based on hourly forecasts symbols v observations for April 2015..
Otherwise they are wrong about 6 out of 10 times on timing and weather type.
Don’t believe what the MO say about accuracy but check out the accuracy for your own location.

bazzer1959
Reply to  QV
October 18, 2016 4:42 am

I did exactly that for a year, about my home town here in the south of England. I showed their forecast, then the actual weather. They were completely correct only 20% of the time. The problem is that they use words that are get-outs. They would say something like:
‘Most places will see a dry day, but there is the chance of an odd shower here and there’.
That isn’t a forecast. They can’t go wrong with saying that – so I discounted forecasts like that, and they were added to their ‘incorrect’ tally. Sometimes they would get it incredibly wrong, like:
‘A dry day everywhere’.
…And it would rain, maybe only for a few minutes, but it was still wrong.
Or:
‘A showery day’.
And we never had a single shower. When it was warm and sunny, they couldn’t go wrong, and anyone could have predicted it, even a chimp. All in all, I find the Met Office completely and utterly useless. ‘Maybe’, ‘Possibility of’, ‘Chance of’, ‘In places’, ‘Could be’, etc. are not forecasts. You can’t go wrong by saying those things, even if they’re true.

Reply to  QV
October 18, 2016 6:04 am

What does 37% accurate mean?
If you could predict the throw of a die 37% of the time it would be quite impressive,
if you could predict the lottery 37% of the time it would be extremely impressive.

MarkW
Reply to  QV
October 18, 2016 6:31 am

Are you arguing that until weather forecasts can predict exactly where each shower will form, they are useless?

Gerry, England
Reply to  QV
October 18, 2016 12:52 pm

Of course it helps that they keep changing the ‘forecast’ regularly by which time what they said 5 days earlier is long since gone. I borrow from Robin Page who in his book commented that the more sophisticated and technology driven the weather forecast has become the worse it has got. I recall a TMS classic when looking ahead to a day’s play, that the forecast they had now was completely different from the one they had been given an hour earlier.
Bolded but not commented on was the solar forcing on the stratospheric polar vortex. I thought their religion forbade any admission of solar effects?

Andrew Bennett
Reply to  John Edmondson
October 18, 2016 7:07 am

Statistically 62% accuracy is little better than guesswork

Geronimo
Reply to  Andrew Bennett
October 18, 2016 3:08 pm

62% is a lot better than guesswork when averaged over a long period.
If you could predict correctly whether a roulette wheel finished on black or white 62% of the time you would be rich. If you guessed you would be broke.

QV
Reply to  John Edmondson
October 18, 2016 7:17 am

“What does 37% accurate mean?”
It means precisely that the symbols were correct 37% of the time.
“If you could predict the throw of a die 37% of the time it would be quite impressive,
if you could predict the lottery 37% of the time it would be extremely impressive.”
But the throw of a die and the lottery is are random events, the weather isn’t.
16.6% would be chance for a die, the MO do twice as well as that.
If the MO can’t forecast the weather to the hour, then they shouldn’t pretend that they can.

Reply to  QV
October 18, 2016 10:25 am

“It means precisely that the symbols were correct 37% of the time.”
Which is quite impressive. The MO use a large number of symbols with subtle differences.
To get it right 37% of the time requires predicting the exact type of weather to a specific hour and getting it exactly right more than 1 in 3 times.
“If the MO can’t forecast the weather to the hour, then they shouldn’t pretend that they can.”
I doubt the MO ever claimed they can predict the weather to the hour with the sort of precision you require.
Their forecasts are a good indication of the likely weather patterns over the course of a day – not an exercise in precognition.

Richard Patton
Reply to  QV
October 18, 2016 8:27 pm

Obviously you never have been a forecaster. Because the forecasts (I’m talking about the 3-5 day forecasts) are issued for large geographic areas, they can’t catch the shower that falls on your house. If you need specific forecasts for your immediate are you need to pay a private forecaster who (for a price) can improve the accuracy of the forecast for your house.
I don’t know about the MET office but the NWS forecasters for over at **least** 40 years (that’s when I became an observer in the Navy) have issued forecast reasonings. They are transparent about their confidence in the forecasts, if the models are in disagreement, agreement, or not handling the situation well. If the MET issues forecast reasonings I’d suggest you check into why they forecast as they do.
As to long range >2 weeks, only a fool forecasts long range or the stock market; of which there are plenty in government.

QV
Reply to  John Edmondson
October 18, 2016 7:21 am

“Are you arguing that until weather forecasts can predict exactly where each shower will form, they are useless?”
I don’t think I am.
But it is the MO who are saying they can forecast weather to the hour, by producing the forecast.

bit chilly
Reply to  John Edmondson
October 18, 2016 9:33 am

5 days you say . they struggle with rain, wind strength and direction as close as 24 hours out in anything other than weather arriving from a westerly direction. in times of so called austerity that 90 odd million quid was a complete waste of money.

auto
Reply to  bit chilly
October 18, 2016 2:49 pm

bit chilly
Utter agreement. The MO has good days – and rather bad ones.
If the weather is from the west – and the jet stream isn’t on walkabout – they’re OK for a couple of days . . . .
If.
Good rule for the London commuter – carry a brolly 24/7/365 [366 this year!].
Auto

Peter Miller
October 18, 2016 12:02 am

As for accuracy, I think the UK Met Office can be congratulated for predicting it will be generally colder in winter than it will be in summer.
As for anything else, historical fact has not been kind to previous Met Office’s predictions. Mother Nature will probably take care to ensure it remains that way.

1saveenergy
October 18, 2016 12:04 am

Note how almost everything to do with climate/weather contains the magic number 97.

Eugene WR Gallun
October 18, 2016 12:19 am

I am sure the Met Office will be able to predict the arrival of each of the four seasons one year in advance — with 62% accuracy.
Eugene WR Gallun

rapscallion
Reply to  Eugene WR Gallun
October 18, 2016 5:13 am

Don’t bank on it Eugene. I found the Met Office offshore forecasts more wrong that right. One left me beating to windward going up The Channel in what was supposed to be a force 4. It was actually a force 7.
I’ve never trusted them since.

bit chilly
Reply to  rapscallion
October 18, 2016 9:35 am

not a nice experience at all . if only met office employees could experience the reality vs their forecasts that some of us do.

auto
Reply to  rapscallion
October 18, 2016 2:54 pm

Ah, now, like rapscallion and bit chilly, I would be content if the MO could get tomorrow right with – ohhh – say 97% accuracy.
Do a Vivaldi and get four seasons right – well, maybe, but they’ve had over 150 years practice.
Auto – long a weather fan.
And why not?
These Isles get their fair share of it.

NorwegianSceptic
October 18, 2016 12:19 am

I have predicted (with at least 97% accuracy) for many years, that here in Norway we will have winter sometime between october-april, followed by some kind of spring, then summer (june-august) and autumn next. I can even do this without a computer! (No grants for me, then….).

MarkW
Reply to  NorwegianSceptic
October 18, 2016 6:32 am

What happened in the 3% of years where spring did not follow winter?

rogerthesurf
October 18, 2016 12:34 am

“The Met Office has shown it can predict the weather one year in advance with its new £97 million supercomputer.”
Aah but can they tell me the weather on my day off next weekend?
Cheers
Roger
http://www.rogerfromnewzealand.wordpress.com

A C Osborn
Reply to  rogerthesurf
October 18, 2016 2:15 am

The straight answer is NO.
Where I live in South Wales, ie coastal they cannot “Accurately” predict the weather 24 hours before hand.
They get it generally correct, but there hourly timing is usually off and sometimes they get it completely wrong.
You also notice that they change their “5 days forecast” practically every day leading upt to your target day and sometime even on that day.

rogerthesurf
Reply to  A C Osborn
October 18, 2016 10:41 pm

Sounds very similar to the NZ Met Office:)

October 18, 2016 12:35 am

It will be interesting to see what odds if any the bookmakers will give on the predictions happening.
I suspect they won’t take bets on anything other than the traditional snow at Christmas.

Greg
Reply to  Oldseadog
October 18, 2016 4:10 am

bookmakers odds reveal bets placed, ie betting publics confidence in the predictions.
Means nothing except as an index of the beliefs of a superstitious sector of the public.

bit chilly
Reply to  Greg
October 18, 2016 9:38 am

yes, the odds i got on brexit were a good example of that .the “book” was top loaded with larger bets from the liberal islington set types and yuppies in and around the south east of england that wrongly thought they knew better than the rest of the country, as usual. still laughing now 😉

John
October 18, 2016 12:51 am

Ah, in fairness to the MetOffice, they made no such claim. They claimed that with a 62% accuracy they can predict the state of the NAO. Only time will tell if they can.
Of course, NAO can offer some clues to what weather can take hold, but no more than that.

Martin A
October 18, 2016 1:05 am

With ‘hindcasting’ to test a model, particularly a model that incorporates data from observations and one that is ‘tuned’ [aka fiddle-factors/parameterisations] it is very difficult to exclude the possibility that the results are biased in favour of ‘predicting’ what actually happened.
With the Met Office’s chaotic processes for model development, it is virtually certain that such bias is unintentionally built in to its models.

Admin
October 18, 2016 1:11 am

It is likely that the “Persistence Model” or “Naive” forecast will produce far better than 62% accuracy in forecasting a year in advance, i.e. the weather will be the same on this date a year from now as it is today.
The Met office using very generous, (loose?) metrics in assessing its own accuracy. For example, a temperature forecast is considered accurate if gets within plus or minus 2 degrees Celsius. Even a 3 hour forecast.
I think a Naive forecast would do quite well.

decnine
October 18, 2016 1:28 am

One for British readers.
From a Met Office press release dated 1 April 2018: “Our first 12 month forecast was a great success. We predicted all the right weather. Just not necessarily in the right order.”

richard verney
Reply to  decnine
October 18, 2016 3:17 am

Morecombe & Wise, Greig Piano concerto sketch

I am playing all the right notes, but not necessarily in the right order.

auto
Reply to  decnine
October 18, 2016 2:57 pm

decnine
Almost wet myself!!
Plus lots of shedloads!
Auto

mwhite
Reply to  mwhite
October 18, 2016 1:37 am

http://www.metoffice.gov.uk/images/prob_ensemble/20161001/2cat_20161001_mslp_months35_global_deter_public.png
I could be wrong but it seems to be suggesting an Atlantic driven (mild) winter??

stevekeohane
Reply to  mwhite
October 18, 2016 8:34 am

How do they get a 2 meter temperature where they don’t measure it, for example, where most of the red and orange is….

Ed Zuiderwijk
October 18, 2016 2:06 am

Actually, they had applied for a computer of £100 million but the financing body decided that it should allocate only 97% of that amount.

mwh
October 18, 2016 2:15 am

I think Piers Corbyn has been predicting the polar vortex with similar or better accuracy for several years, with the aid of his laptop which may or may not (delete as appropriate) have cost £97 million. It is interesting that they now believe that the sun does have an influence on the AMO and Polar Vortex having stated in the past, I believe, that the forcing is trivial.
I wonder also if they are including their version of CO2 forcing into predictions as they always have done – I am convinced this alone knocks their predictions out of whack relatively quickly. However notice that it isnt really mentioned – could this be because the percentage went up when it was removed – I wonder

bit chilly
Reply to  mwh
October 18, 2016 9:41 am

looks like they have been listening to ren regarding ozone levels.

Reply to  mwh
October 18, 2016 10:29 am

“I think Piers Corbyn has been predicting the polar vortex with similar or better accuracy for several years, with the aid of his laptop which may or may not (delete as appropriate) have cost £97 million.”
If only he would publish a paper demonstrating that success.

petermue
October 18, 2016 2:18 am

62% with 97% consensus?

Robin Hewitt
October 18, 2016 2:29 am

According to the old wives, cold British winters usually come in fours. Is that sufficient for a 62% prediction? I think it is.

Robert from oz
October 18, 2016 2:32 am

What an incredible waste of money , I have a 100% accurate weather instrument , it’s a bit of wool hanging from a small stand .
All you do is put it outside and somehow it starts working straight away .
If the wool is wet it’s raining .
If the wool is moving its windy .
If the wool is missing it’s a hurricane .
If the wool is warm it’s sunny , I have found it to be always accurate and beats the BOM as a matter of fact I think they might now use a similar system .

Richard Patton
Reply to  Robert from oz
October 18, 2016 8:31 pm

You forgot If it’s white on top it’s snowing, and If you can’t see it it’s foggy.

October 18, 2016 2:33 am

“Fitting a model to a limited data set is often not the same thing as accurately modelling the physics behind the data.”
Too true it isn’t. You can fit a polynomial to any data set but unless you understand the physics of why your polynomial fits you’d have to be insane to try using your polynomial for extrapolations into the future. Even if you do understand the physics – or think you do – these kinds of extrapolations are fraught with disaster until you’ve run and compared against real world data and refined over and over and over until it’s demonstrably accurate and reliable.

Greg
Reply to  cephus0
October 18, 2016 4:16 am

No, you would be insane to extrapolate you polynomial out to 2100 AD, but it may be a good guide for this winter.

Gary Pearse
October 18, 2016 2:35 am

Of course it was sceptical scientists who showed them that these ocean oscillations in both the Atlantic and Pacific were first order drivers that largely explained global weather without the need to evoke CO2 at all. Their pathetic models only 5yrs ago basically employed their much debunked CO2 control know equation which could only give a one way forecast of ever warming weather,even though they used the same 100M pound computer. Over the past 10yrs they would have better done with a coin toss.
Now with the reluctantly accepted knowledge of ocean oscillation elephants in the room, the forecasts can be done using a 300 GBP surface pro or maybe just an eyeballed guess with the status of these oscillations.

bertief
October 18, 2016 2:37 am

I’m grateful these morons are not designing aircraft or bridges: “ah yes, there is a 62% chance the bridge/plane will stay up”. Good grief.

Reply to  bertief
October 18, 2016 2:45 am

But people know much much more about aircraft and bridge designing and building than they do about the atmosphere and how it works.

bit chilly
Reply to  Oldseadog
October 18, 2016 9:42 am

i may be wrong osd, but i think that was part of the point bertief was making.

Reply to  Oldseadog
October 18, 2016 3:28 pm

Oh, of course…That’s why the debate is over.

Robert of Ottawa
October 18, 2016 2:50 am

I have just hacked the first UK Met Office 1 year forecast.
Here it is:
Sunny periods with scattered showers.

Alan the Brit
Reply to  Robert of Ottawa
October 18, 2016 3:41 am

Brilliant!

Gamecock
October 18, 2016 2:54 am

Who are their customers? Who is going to act on their predictions? What will they do differently?
It would seem to be a £97 million toy.

October 18, 2016 3:03 am

Warm and wet vs cold and dry. 62% chance of predicting correctly a year in advance. So even if this is true what possible use will it be? What specific behaviour should I change based on one prediction or the other. What specific behaviour would power suppliers, local councils or any other authority actually change based on such a prediction?

October 18, 2016 3:10 am

I have tracked the four day UK forecasts that are printed in the Daily Telegraph. These are very broad such as it will rain in north Scotland but it will be sunny in London. They are worthless. They fluctuate violently from day to day. To think it is my money these guys are wasting.

Robert of Ottawa
October 18, 2016 3:20 am

Well, there’s always a good chance tomorrow’s weather is going to be like todays; so maybe next year’s weather will be like this year’s.

John Silver
October 18, 2016 3:21 am

They didn’t mention what “62 per cent accuracy” actually means.

The Old Man
October 18, 2016 3:42 am

ahhh.. yes. The cursed recursive. Been to that dance in my arrogant youth, and suitably humbled, stood in awe at the elegance of it all, knowing I knew nothing.
https://notonmywatch.com/?p=679

October 18, 2016 3:49 am

To be fair, UKMET is one of the better models used to predict things like hurricane development, and its generally fairly accurate up to 4-5 days out.
Put that game – hurricane forecasting – shows the perils of simplified models. Often one or two models will generate completely different hurricane tracks. So its clear that subtle assumptions in models can make huge differences to the predictions.
Frankly we need a new way to deal with complex non linear phenomena. Treating them as linear in the limit ends up with huge maths and poor accuracy, or simplifying them into broader liner equations results in so much loss of accuracy that its pretty useless.
We need new mathematical tools.

Ian Macdonald
October 18, 2016 4:04 am

To be fair, forecasting UK weather is a difficult task. The prevailing conditions are determined by swirling high and low pressure regions that roll in across the Atlantic, and the paths of these regions can vary sufficiently that they sometimes miss us altogether. Thus, the best the forecasters can do is to say that there is a risk of high winds, heavy rain or whatever associated with a particular weatherfront approaching from the Atlantic. They cannot say with any certainty that it will hit, at least not until it’s a few hours away.
Add to that, the media presenters want the forecasters to give very simple and direct advice to the public, but it isn’t a simple situation.

Greg
Reply to  Ian Macdonald
October 18, 2016 4:43 am

It’s a chaotic system ! That is why it is pointless trying to do anything more than look at existing weather patterns 24h ahead.
A small niche group have manage to build a lucrative career path out of making spurious long term “predictions” using big computers. In past time they would have used sheep’s entrails and been equally held in awe.
Society is currently more aligned to believe in tech than sheep’s guts but it’s all a form of witchcrafft.

auto
Reply to  Greg
October 18, 2016 3:12 pm

Greg,
Tonight, my near neighbours, down the hill, had some lovely haggis.
That allows me the predict a general cooling here in London over the next two to three months.
Memo to HM Government:
I am about 20 % up to MO standard, already, so please send hardware cheque of 20% of 97 Million [incredible shrinking] pounds by return.
Auto

Toneb
October 18, 2016 4:06 am

“With enough effort your model could be coerced into accurately hindcasting past lottery results. But it is very unlikely your lottery model would be able to predict future draws. Fitting a model to a limited data set is often not the same thing as accurately modelling the physics behind the data.
No it couldn’t as picking numbered random balls, is just that – random.
Impossible to *predict* hindcast or otherwise.
A model is a tool to learn.
So as in early NWP, it has limited, but it seems some, skill, via applying “…two sources of skill for second-winter forecasts of the North Atlantic Oscillation: climate variability in the tropical Pacific region and predictable effects of solar forcing on the stratospheric polar vortex strength….”
Such that they learned of “model biases in Arctic sea ice that, if reduced, may further increase skill”.
That’s how it works.
And these *forecasts* may be useful to:
“the transport, energy, water management and insurance sectors.”
They are not “Push(ing) Inaccurate Forecasts out to 12 Months”.
They are reporting “modest but significant skill” in their endeavour. That is all.
http://www.metoffice.gov.uk/news/releases/2016/winter-forecast-skill

Greg
Reply to  Toneb
October 18, 2016 4:31 am

Thanks for the link. Curiously the omit any meaningful reference to the paper in question. No pre-print , no doi , not even the title of the paper and the authors.
PR puff piece they are scared to put up for inspection?
You go the their own web site to git it from the horse’s mouth and find they present you will the wrong end of the animal in question.
“Our latest research builds on that gain in skill in predicting the coming winter, but also reveals, for the first time, modest but significant skill in predicting the phase of the NAO one year ahead. This is an exciting first step in developing useful winter climate predictions on longer timescales.”
This discovery was made possible thanks to increased Met Office supercomputer capacity. This allowed the team to increase the resolution of the climate model and test the retrospective skill of their forecasts over the 35-year period from 1980 onwards.
What ? “test the retrospective skill of their forecasts ” That is one of their top scientists speaking?
If you test hind-casts through your calibration period you are testing your calibration not the predicitive capabilities of your model.

If they do not understand that subtly they should be being payed to do computer modelling. If they do they are being deliberately deceitful in order to justify continued funding.

Take your pick on which one it is.

Toneb
Reply to  Greg
October 18, 2016 9:09 am

“Curiously the omit any meaningful reference to the paper in question. No pre-print , no doi , not even the title of the paper and the authors”
It was in the post from Worrell my friend:
Again….
http://www.nature.com/ngeo/journal/vaop/ncurrent/full/ngeo2824.html
Oh, and seeing as it’s you…
Here’s the paper in full:
http://sci-hub.bz/10.1038/ngeo2824
Does it still meet you definition of puffiness?

Greg
Reply to  Toneb
October 18, 2016 4:36 am

In the absence of any link to their alleged paper it’s difficult to evaluate exactly what they are claiming.

Reply to  Toneb
October 18, 2016 7:04 am

There’s no mention of 62% accuracy in that release. I’d be surprised if the Met Office are making any such claim – it’s just the Telegraph are misunderstanding the r value of 0.62 quoted in the graph.

Toneb
Reply to  Bellman
October 19, 2016 12:58 am

“DePreSys3 hindcasts of the first winter (Fig. 1a) skilfully predict
the year-to-year variability in the observed winter NAO index with
a highly significant correlation skill of r =0.62 (p=0.001 using a
two-tailed Student’s t-test).”

Reply to  Bellman
October 19, 2016 4:29 am

“DePreSys3 hindcasts of the first winter (Fig. 1a) skilfully predict
the year-to-year variability in the observed winter NAO index with
a highly significant correlation skill of r =0.62 (p=0.001 using a
two-tailed Student’s t-test).”
Yes, that’s what I meant. The Telegraph reporter described the correlation coefficient of 0.62 as “62% accuracy”. This has obviously confused a lot of people here who are interpreting the 62% as meaning they are right 62% of the time and wrong 38% of the time.
I think the Telegraph are making another mistake as the 0.62 correlation is for the “first winter” forecast, that is the forecast made in November for the coming winter. The year ahead forecast had a (statistically significant) correlation of 0.42.

Omegaman
October 18, 2016 4:07 am

The Thames will freeze JAN – FEB 2017. Will their supercomputer confirm this before the freeze or after the fact?

Omegaman
October 18, 2016 4:09 am

The Thames will freeze Jan – Feb 2017. The NHS will have difficulty keeping up with the casualties.

DaveS
October 18, 2016 4:35 am

From The Daily Telegraph article on this story:
“After the 2007 prediction of a “barbecue summer” ended in a soggy washout, the Met Office pointed out that its seasonal forecast offered only a 65 per cent chance of being right.”
So spending £97 million has reduced the chance of being right from 65 to 62 per cent. Some progress!

P Wilson
Reply to  DaveS
October 19, 2016 5:11 pm

It would cost £167 million to reduce it further to 50% – that is the next step

biff
October 18, 2016 4:37 am

Will not be long before the start bleating about upgrades…

commieBob
October 18, 2016 4:37 am

I haven’t noticed much from Piers Corbyn lately. He claims to be accurate. He was able to pay the rent with a series of bets on the weather until the bookies cut him off.

P Wilson
Reply to  commieBob
October 19, 2016 5:13 pm

Piers Corbyn – the older brother of Jeremy Corbyn

commieBob
Reply to  P Wilson
October 19, 2016 6:50 pm

Yep, that’s the one.

Chris Schoneveld
October 18, 2016 4:46 am

Ha, ha, haha “But it is very unlikely your lottery model would be able to predict future draws.”
Not “very unlikely” but impossible.

schitzree
Reply to  Chris Schoneveld
October 18, 2016 5:32 am

Not impossible. A Lotto prediction program has the same odds of being right as the little old lady who goes down to the store each week to by Powerball tickets.
I wouldn’t bet on it. But hey, she does. ^¿^

Chris Schoneveld
Reply to  schitzree
October 18, 2016 6:56 am

I tell you that a simple laptop computer is as inadequate in predicting a lotto outcome as a super computer of a trillion dollars.

Chris Schoneveld
Reply to  schitzree
October 18, 2016 6:59 am

Sorry I initially missed the joke.

October 18, 2016 4:59 am

Now, if only they could tell us what ‘62% accuracy’ means in a weather forecast!
62% of a day is about 15 hrs. So if it rains sometime between 6am and 9pm a forecast of rain is 62% right?? Or does it mean that if rains more than an hour sometime between 6am and 9pm over 62% of the forecast area it’s counted as a correct fore cas?. Is a forecast of mostly cloudy today correct of it’s solid cloud between 6am and 9pm and totally clear overnight a correct one or is it correct if about 2/3 of the sky has clouds for 24hrs?
There is a huge difference in utility of a forecast when using such vague terms as 62% correct. Gusts of wind at 15 knots for 20 minutes every hour is not the same as 5knot breezes.

schitzree
October 18, 2016 5:09 am

The Met Office has shown it can predict the weather one year in advance with its new £97 million supercomputer.

Well no, they haven’t. They’ve shown their £97 million supercomputer can ‘predict’ the past with the data from a year earlier less then 2/3 of the time. To show they can predict the future a year in advance they need to make a prediction for a year from now and then wait to see if they were right. Then they need to do it another half dozen times at least to show it wasn’t a fluke.
Typical headline inflation.

October 18, 2016 5:57 am

How many watts does a £97 million supercomputer require these days ?

Reply to  Bob Armstrong
October 18, 2016 9:00 am

Looked it up for you. UKMet says 2.7MW.

Reply to  ristvan
October 19, 2016 8:42 am

That’s much better than I expected .
About 2000 horsepower 24 7 365 .

Bruce Cobb
October 18, 2016 6:00 am

Oh yeah? Well over here we’ve got the Old Farmer’s Almanac with an 80% accuracy rating. No government funding either.

David Wells
October 18, 2016 6:07 am

Having cocked up over trying to forecast what our climate will be in 100 years time they have now decided to spend even more of our money to do the job they were employed to do in first place, forecast our weather with some degree of accuracy though stretching that ideology to one year hence with only a 62% chance of being almost correct is somewhat stretching reality. What is more to the point is that they are making this gambit before in fact their new computer has even been installed which leads me to believe that their declaration today is more in hope than expectation. This is only what they believe they can do and the chimps currently in place at Westminster are falling in line as they always do with any trumped up nonsense that allows them to spend our money on anything other than what it should be spent on to curry favour and appease the sad halfwits whose beliefs continue to mitigate against what is left of the single electron that flits between one mutant alarmist and another looking for its final resting place but maybe not before what is left of our environment has been destroyed by solar panels and wind turbines. I wish I believed in God.

October 18, 2016 6:08 am

Global anthropogenic SO2 emissions are being tracked very closely, and it should be possible to make an accurate estimate of the amount of reductions expected for the coming year.
Give me that number, and I can predict the average global temperature at years end with an accuracy of greater than 90% with my $1.00 calculator.(after accounting for the temporary effects of El Ninos, Recessions, La Ninas, and volcanic eruptions)
(This claim is based upon temperature changes that occurred due to SO2 reductions 1975-2011)
This information could probably also be used to make regional predictions.

Gary
October 18, 2016 6:22 am

What happened to the 97% standard? Anything less just isn’t Climate Science.

October 18, 2016 6:46 am

So, are they actually going to predict something or waffle about with weasel words? I’ll bet you anything that 97M will produce nothing but weasel words and a whole list of excuses.

UK Sceptic
October 18, 2016 7:11 am

It’s only a matter of time before that improbably accurate12 months hence forecast will be pushed to an absolutely accurate fifty years hence we are all going to fry forecast.

John Boles
October 18, 2016 8:07 am

Just how is accuracy rated?

Logos_wrench
October 18, 2016 8:13 am

I would be happy with an accurate 5 day forecast. Let me know when that happens.

Toneb
Reply to  Logos_wrench
October 18, 2016 9:01 am

“Let me know when that happens”
http://www.cma.gov.cn/en2014/research/News/201509/W020150907602390847882.jpg
“Forecast skill is the correlation between the forecasts and the
verifying analysis of the height of the 500-hPa level, expressed as the anomaly
with respect to the climatological height. Values greater than 60% indicate
useful forecasts, while those greater than 80% represent a high degree of
accuracy. The convergence of the curves for Northern Hemisphere (NH) and
Southern Hemisphere (SH) after 1999 indicates the breakthrough in exploiting
satellite data through the use of variational data100.
http://www.nature.com/nature/journal/v525/n7567/abs/nature14956.html

Editor
October 18, 2016 10:35 am

The Met is attempting to predict the North Atlantic Oscillation — and in hindcasts, got it right 62% of the time.
Nothing wrong with that — and nothing wrong with the quite possible idea that the NAO is periodic, cyclical, and predictable from other knowable climate factors.
I bet that the best weatherman in England could do the same from experience alone — looking back at past winters and world weather patterns leading up to them. This kind of prediction is well within abilities of modern climate models — broad general patterns suggesting near-present future patterns.

Toneb
Reply to  Kip Hansen
October 19, 2016 12:42 am

“I bet that the best weatherman in England could do the same from experience alone — ”
Nope.
Nowhere near.

Editor
Reply to  Toneb
October 19, 2016 7:48 am

Toneb ==> Gee, the bar is pretty low….they only have to predict either “mild, rainy winter.” and “big freeze winter” and only need to get it right 62% of the time (a bit over the chance guess 50%).
In the US, some credit the Farmer’s Almanac with a near 80% success rate on similar predictions of winter conditions (mild or hard).

Resourceguy
October 18, 2016 11:36 am

Might as well if CO2 is their only variable.

Le Roy
October 18, 2016 12:43 pm

Who’s bothered about the £97 million anyway?……We’re still giving that much to the EU to squander every 2 days!

Candy
October 19, 2016 2:10 am

But 62% is no use at all if you plan to use this data to make any sort of decision.

ScienceABC123
October 19, 2016 6:46 am

I’ll bet that I can achieve better than 62% accuracy just by examining the ‘unadulterated’ historical records.

Brian Johnson uk
October 19, 2016 12:33 pm

My seaweed on a stick is better at forecasting than the 97 Million squids we taxpayers shelled out. Only accurate weather from the Met Office are the Aviation Actuals!