Finally, a climate forecast model that works?

Note: Short term predictions are relatively easy, it remains to be seen if this holds up over the long term. I have my doubts. – Anthony

Guest post by Frank Lemke

The Global Warming Prediction Project is an impartial, transparent, and independent project where no public, private or corporate funding is involved. It is about original concepts and results of  inductive self-organizing modeling and prediction of global warming and related problems.

In September 2011, we presented a medium-term (79 months) quantitative prediction of monthly global mean temperatures based on an interdependent system model of the atmosphere developed by KnowledgeMiner, which was also discussed at Climate Etc. in October 2011. This model describes a non-linear dynamic system of the atmosphere consisting of 5 major climate drivers: Ozone concentration, aerosols, radiative cloud fraction, and global mean temperature as endogenous variables and sun activity (sunspot numbers) as exogenous variable of the system. This system model was obtained from monthly observation data of the past 33 years (6 variables in total: the 5 variables the system is actually composed of (see above) plus CO2, which, however, has not been identified as relevant system variable), exclusively, by unique self-organizing knowledge extraction technologies.

Now, more than a year has passed, and we can verify what has been predicted relative to the temperatures, which have really been measured (fig. 1).


AGW_predictive_model

Fig. 1: Ex-ante forecast (most likely (red), high, low (pink); April 2011 – November 2017) of the system model as of March 2011 vs observed values (black and white square dots; HADCRUT3) from April 2011 to December 2012. These 21 months are used for verification of the out-of-sample predictive power of the system model.

Verifying the prediction skill of the system model from April 2011 to December 2012, the accuracy of the most likely forecast (solid red line) remains at a high level of 75%, and the accuracy relative to prediction uncertainty (pink area) is an exceptional 98%. Given the noise in the data (presumably incomplete set of system variables considered, noise added during measurement and preprocessing of raw observation data, or random events, for example), this clearly confirms the validity of the system model and its forecast.

In comparison, the IPCC AR4 A1B projection currently shows a prediction accuracy of 23% (September 2007 – December 2012, 64 months) and just 7% accuracy for the same forecast horizon as applied for the system model (April 2011 – December 2012, 21 months).

The two models, IPCC model and atmospheric system model, use two very different modeling approaches: theory-driven vs data-driven modeling. The IPCC model is based essentially on AGW theory by emission of greenhouse gases, namely CO2, the presented atmospheric system model on the other hand is a CO2-free prediction model. It is described by 5 other variables. The IPCC model shows a prediction accuracy of 7% and the atmospheric system model an accuracy of 75% for the same most recent 21 months of time…

The climate system is a complex system that consists of a number of variables, which are connected interdependently, nonlinearly and dynamically and where it is not clear, which are the causes and which are the effects. The simplistic linear cause-effect relationship “more atmospheric CO2 = higher temperatures” the IPCC model is based on is not an adequate tool to describe the complexity of the atmosphere sufficiently.

Read the complete post here:

http://climateprediction.eu/cc/Main/Entries/2013/1/21_What_Drives_Global_Warming_-_Update.html

Advertisements

  Subscribe  
newest oldest most voted
Notify of

How does it “predict” the response to El Nino in 1998? And I find this confusing:

In September 2011, we presented a medium-term (79 months) quantitative prediction of monthly global mean temperatures based on an interdependent system model of the atmosphere developed by KnowledgeMiner, which was also discussed at Climate Etc. in October 2011. This model describes a non-linear dynamic system of the atmosphere consisting of 5 major climate drivers: Ozone concentration, aerosols, radiative cloud fraction, and global mean temperature

So you use global mean temperature to predict global mean temperature? How does that work, exactly?

“no public, private or corporate funding is involved”
How do they manage that? Who pays the electric bills, etc.? I’d sure love to see how that works. 🙂

TRM

“is a CO2-free prediction model” – OMG. Really? Wow. And they got 75% instead of 7% …..
Go figure. It will be very interesting to see if this approach works over decades. I would love to see it expanded so others could re-weight variables and add their own to publicly make predictions.

Pamela Gray

Sunspot numbers? Why? And I am serious. Why? What algorithm do you use for sunspot numbers? And what is that algorithm based on mechanistically (not correlationally)?

Betting on weather= tomorrow will be mostly similar to today
Betting on climate= the present decade’s average anomaly will be next decade’s average anomaly.
/Bet
WAG= for the next 20 years the average anomaly will be 0.5C
/Sarc
/Fey
😉

Michael John Graham

bye-bye carbon di

YEP

crosspatsch says:
“So you use global mean temperature to predict global mean temperature? How does that work, exactly?”
Presumably lagged actual temperature as a partial vector autoregression (VAR). There’s bound to be persistence in the system.

Eric H.

It’s kind of like using parasitic drag as the main determinant of speed and ETs for a dragster. Though in theory it makes a difference, adding another layer of wax isn’t the solution to your low ETs.

Robinson

“unique self-organizing knowledge extraction technologies.”
What the hell? You mean a Neural Network, don’t you? Why not just say it? Why use this stupid jargon?

So you use global mean temperature to predict global mean temperature? How does that work, exactly?
The BUT (Business as Usual) theory to date is the most accurate way of predicting next year’s temperatures – same as last year.
However this makes me suspicious too. I once made a horrible mistake in a merchant banking model, in which I accidentally incorporated previous model results into the new run. This slipped through testing, because the inclusion of previous data masked problems with the rest of the model.
Once the mistake was corrected, the rest of the model went wild – very embarrassing.
So I’m very suspicious of any system which places a heavy reliance on previous values. Yes it might and probably is necessary when predicting global temperature, but my experience shows such inclusion could also easily mask problems with the model, at least in the short term.
At the very least I would expect inclusion of previous temperatures to lead to a cumulative error – any slight mistake in predicting this years temperature would create an even larger mistake in predicting next year’s temperature, which over a few iterations would render the model prediction worthless.

YEP says:
January 24, 2013 at 1:18 pm
Presumably lagged actual temperature as a partial vector autoregression (VAR). There’s bound to be persistence in the system.

How well does that work out in the last eight observations shown in the graphic? Prediction is for a downward trend, observations are an upward trend and then suddenly out of nowhere a massive reversal. I dunno, Color me skeptical.

Pamela Gray

IMHO. The temperature lag is probably a function of ENSO and creates the greater influence on prediction. It has been definitively demonstrated that sunspot numbers correlated with temperature is not robust, not reliable, and not valid. But the dang things have legs as much as CO2 does. Which is equally not robust, not reliable, and not valid.
The modeler that gets it right will use ENSO patterns of oceanic circulation and SST (a much slower lagged effect) with a variables related to other atmospheric circulation patterns that come and go (more immediate effects), that kick in after a certain value is reached (IE beyond neutral). Multiple scenarios will demonstrate these long and short term teleconnections, meaning that given an ENSO condition, depending on whether or not a shorter term atmospheric pattern kicks in, the temperatures will be thus.

Kasuha

Somehow I met that site a few months ago. Interesting approach indeed, but it’s not more than yet another extrapolated regression. Sophisticated and slightly obscure regression but still just a regression. There’s no guarantee the relations their self-organizing prediction machine established are real. They may be, or they may be just artifact of the method.

YEP

Data-driven models are good exercises to go through when analyzing a complex, dynamic, non-linear system. But nothing is theory-free, other than simple vector autoregression. Choosing the 6 variables, for example, had to be based on theory. And simple predictive skill doesn’t tell you much, except as something to compare the perfomance of a theory-based structural model with. The coefficients that emerge should be meaningful, and simulations should be used to test for things like stability and results that make sense given what we know about natural processes. What was it von Neumann said? “With four parameters I can fit an elephant and with five I can make him wiggle his trunk.”

DirkH

Eric Worrall says:
January 24, 2013 at 1:32 pm
“The BUT (Business as Usual) theory to date is the most accurate way of predicting next year’s temperatures – same as last year.
However this makes me suspicious too. I once made a horrible mistake in a merchant banking model, in which I accidentally incorporated previous model results into the new run. This slipped through testing, because the inclusion of previous data masked problems with the rest of the model.”
You had state leftover from a PREVIOUS run. (Forgot to clear all variables, I guess)
In the kind of time series extrapolation this model uses, you “look back” to the time series so far – of THIS model run. Which is legit.
Still, I’m not convinced it’ll have predictive skill for climate. Climate is the 30 year mean, in other words, low frequency component. The validation period is too short to tell us much about the low frequency component.
I’m using models of this kind for “next day” trading decisions, so my models have to guess the next day right, in a way. And are trained on a history of a thousand days ATM. I wouldn’t trust these models to look far into the future. It’s a probabilistic guess at best.
But if you need to… well I would say if you want to look 30 years into the future the smartest thing would be to train the model on a history of a thousand consecutive real 30 year intervals of climate.

Matthew R Marler

Knowledge Miner is nice software, but it would be nice to read a complete description of how it was implemented in this case. A bunch of us wrote the same thing about neural networks just a short time ago. Nothing is “self organizing” here: the modelers made choices such as what data to input to the algorithms.
It is good to see the model forecast tested against new data.

Steve C

“The Global Warming Prediction Project is an impartial, transparent, and independent project”? From the name of the project alone, you know that none of those adjectives applies.

The above prediction is quite similar (if you take off the fast fluctuation) to my prediction published in 2010 and later in 2012 papers.
e.g.
Scafetta N., 2012. Testing an astronomically based decadal-scale empirical harmonic climate model versus the IPCC (2007) general circulation climate models. Journal of Atmospheric and Solar-Terrestrial Physics 80, 124-137.
http://www.sciencedirect.com/science/article/pii/S1364682611003385
Scafetta N., 2010. Empirical evidence for a celestial origin of the climate oscillations and its implications. Journal of Atmospheric and Solar-Terrestrial Physics 72, 951-970.
http://www.sciencedirect.com/science/article/pii/S1364682610001495
see here for the latest update of the prediction (since 2000) which agrees great with the data:
http://people.duke.edu/~ns2002/#astronomical_model_1
for example the model predicts a peak in 2015 as mine.
The only problem with the above figure in the post is that it seems that the latest temperature dot for Dec/2012 is located at Jan/2012.

Ian

Has the model been used to make “hind-casts”? If so were they accurate? If not will hind-casting be attempted?

Steve Oregon

This climate forecast model is a real travesty.
Plus it got me thinking.
How will alarmists cope if warming never returns for the rest of their lives?
A few more years of the same will be bad for them. 6, 7 or 8 will be painful.
But 10, 20 or 30 years of a non-warming planet will be catastrophic for their funny little fictitious world.
I sure hope they cry us a river.

Truthseeker

Let us assume that they have correctly identified the most significant variables (which do not include CO2 – IPCC and alarmists please note) and can get a good correlation for past data. The problem is still predicting the values of those variables. Maybe they can use models to predict the variables they are using in the model to predict climate. Of course then they will have other variables to predict, which will mean other models to predict the variables they need for the models to predict the variables they need to predict the variables they need to predict the climate. Then they will need models … ad infinitim …

Rud Istvan

The neural net was fit to 33 years and 3 sunspot cycles. ‘Out of range’ accuracy was “good”, but only for 21 months, less than 2 years. Given the predictability of the seasons (winter is colder then summer), the time series autocorrelations, and the fact that climate changes very slowly, it is not surprising that a sophisticated data fit did better then first principle physics in GCMs– for a couple of years. But that is useless for multidecadal predictions for all of the known problems inherent in out of range forecasting from data fits. The Arts of Truth used a medically peer reviewed correlation between BMI and Miss America pagent winners to “prove” the winner would show up dead from starvation by 2020. You should believe that about as much as CAGW.
And even if CO2 wasn’t a predictor in this net, it is still “there” in the temperature side of the neural net fit. So says nothing about climate sensitivity, either. An interesting question is whether, had it been explicitly added, the neural net would have given better predictions? One suspects yes, but for the simple reason it’s another variable for the net to massage. I believe it was Von Neuman who said, “give me four variables and I can model an elephant. Give me five, and I can model its trunk.”
I agree with Anthony. Doubtful for the long run. Trivial for the short run.

AndyG55

What temperature sets did they use to calibrate to the past?
If they used GISS or HadCrud, they have serious problems matching any future reality.

bill

neural networks can be very good, the test is to keep adding historical strong data sets to keep validating things.
my suspicion is that it will not predict past the “momentum” of the current data or about 10 years.
love the concept of an agnostic neural network data mining, if that is what they did.
the work is in the data assembly not the processing anymore so funding could be quite modest if you value the required very smart people at enthusiast rates.

YEP says:
January 24, 2013 at 1:18 pm
crosspatsch says:
“So you use global mean temperature to predict global mean temperature? How does that work, exactly?”
Presumably lagged actual temperature as a partial vector autoregression (VAR). There’s bound to be persistence in the system.

But what is the source of that persistence?
I doubt there is much persistence from atmospheric temperatures, that is, the thermal energy in the air.
Otherwise, over the 21 month forecast period we have seen a large difference between summer and winter anomalies, which wasn’t the case for most of the prior period. To get such a good forecast it must be forecasting the summer/winter shift, which raises the question, How well does it hindcast the prior period when this shift was absent?

Disputin

Eric Worrall says:
January 24, 2013 at 1:32 pm
“At the very least I would expect inclusion of previous temperatures to lead to a cumulative error – any slight mistake in predicting this years temperature would create an even larger mistake in predicting next year’s temperature, which over a few iterations would render the model prediction worthless.”
Is that not the mark of chaos? And are not chaotic systems inherently unpredictable? My scepticism knows no bounds!

john robertson

Accuracy relative to prediction uncertainty.
I love that line, prediction is hard, we know dick all.
Only 1C error range,at least they’re making predictions and willing to pst them.

If I was looking at an electronic circuit I would say that the model is missing a damping factor (negative feedback) fwiw.
Any suggestions?

Not only are models wrong, they have to be wrong. This has to do with the fantastic scale, complexity and variability of the thing being modelled and the ludicrously narrow, simplistic and static nature of the models (the best ones, that is).
Apart from all that, models are great. Well, that chick who was married to Billy Joel, at least.

temp

To YEP
While I agree with the whole “anti-variable” argument. As long as these guys aren’t asking for money/holding us for ransom then i have no problem with them running this train of thought. If they can roughly predict long term weather then frankly doesn’t matter if they have a goat grinding variable…
Some of the issues dealing with short/long term weather and short/long term climate is that we don’t know all the variables currently. If we can roughly make predictions then they maybe about to reverse engineer the true variables from those predictions.

xham

Crosspatch says “So you use global mean temperature to predict global mean temperature? How does that work, exactly?”
Non-linear dynamic equations often take some fraction of the output as input. Its what makes them non-linear and usually chaotic. The real question for how well this model works is in the prediction of changes of state and how close to a cusp we are right now to one of those changes. Not that I want to be around for the next ice age but it would be nice to see if we can put a date range out there for us to worry about.

Joe Public

Finally, an inexpensive climate forecast model that works:-
http://www.cynicaltimes.co.uk/wp-content/uploads/2011/06/OverviewCoinFlipGreenonBlack.jpg

commieBob

… Short term predictions are relatively easy …

RELATIVELY … LOL Bench pressing 500 pounds is relatively easy compared to bench pressing 2000 pounds. I can’t do 500 pounds and nobody can do 2000 pounds.
In terms of climate, I’m not bad for five minute forecasts. I guess the definition of the word ‘short’ matters.

Jimbo

Verifying the prediction skill of the system model from April 2011 to December 2012, the accuracy of the most likely forecast (solid red line) remains at a high level of 75%, and the accuracy relative to prediction uncertainty (pink area) is an exceptional 98%………….
The climate system is a complex system that consists of a number of variables,

Is this weather forecasting or climate forecasting????????? Colder or warmer I am not impressed.

Jimbo

…..and the accuracy relative to prediction uncertainty (pink area) is an exceptional 98%.

Good noodnight all. Zzzzzzzzzzzzzzzzzzzzzzzzz.

Gunga Din

Mr. Layman here.
“Climate” is a study of chaos.
From what I’ve learned here and other places my impression is that the best we can do is identify past climate cycles we’ve gone through, theorize what may have caused them, try to see if what we think may have caused them is happening now and where that might lead us. The problem is that the past climate was as chaotic as our present “climate”. We just don’t know EVERYTHING long past to project in the present what will happen long term.
Think of all those drug commercials in the US. Lots of disclaimers. (Who knew an eye drop might kill you?8-) Sure, they have those disclaimers to legally CYA but they need to CYA because it could happen. Before a drug hits the market it’s been thoroughly tested. But they don’t know everything. The human body is incredibly complex.
“Climate” is incredbly complex.
Me, I’d be happy with an accurate weather forecast 4 days out. I don’t want to bet my kids’ future on a “climate forecast” 100 years out. (No disclaimers.)

Think of all those drug commercials in the US. Lots of disclaimers. (Who knew an eye drop might kill you?8-) Sure, they have those disclaimers to legally CYA but they need to CYA because it could happen. Before a drug hits the market it’s been thoroughly tested. But they don’t know everything. The human body is incredibly complex.
“Climate” is incredbly complex.
###############
yup. that is why it is weird that people think it is safe to geo engineer the planet by dumping C02 in the air.

Chuck Nolan

AndyG55 says:
January 24, 2013 at 2:02 pm
What temperature sets did they use to calibrate to the past?
If they used GISS or HadCrud, they have serious problems matching any future reality.
—————-
Using GISS might be why they can get to only 75%
A more accurate past may give a more accurate prediction.
A little Garbage In will still calculate some level of Garbage Out.
cn

Mean global temperature needs to be carefully defined. How can one be determined with reasonable accuracy when there are such limited and concentrated data stations that omit 70% of the planets surface. Extrapolation with such limited data is unreliable and subject to manipulation.

Leo Smith

“So you use global mean temperature to predict global mean temperature? How does that work, exactly?”
I think that what they mean is that global mean temperature is an OUTPUT variable. It sure aint a constant!

Joe Public

@ Steven Mosher says: January 24, 2013 at 3:36 pm
“Climate is incredbly complex ……. that is why it is weird that people think it is safe to geo engineer the planet by dumping C02 in the air.”
It’s only what we all exhale. Who should stop breeding then?

Bair Polaire

If earth’s climate is driven by chaotic processes – and I’m afraid it is to a large extent – than we will never be able to predict “climate” (= 30 years of future weather). Even if our model was a perfect fit and the model would perfectly mimic chaotic atmospheric behavior, the outcome would be correct only by chance.
Not even an identical second earth at the same place and time would have the same climate when chaotic processes are involved. We’re currently not even sure if major climatic events like ice ages are driven by predictable variables like orbital forcing. Even that coud be chaotic. How can we expect to be able to forecast any short term reversals in cooling or warming trends?
When modeling chaotic processes, you could select the best forecast model today and tomorrow it could still be wrong.
Color me unimpressed.

Ian W

Steven Mosher says:
January 24, 2013 at 3:36 pm
Think of all those drug commercials in the US. Lots of disclaimers. (Who knew an eye drop might kill you?8-) Sure, they have those disclaimers to legally CYA but they need to CYA because it could happen. Before a drug hits the market it’s been thoroughly tested. But they don’t know everything. The human body is incredibly complex.
“Climate” is incredbly complex.
###############
yup. that is why it is weird that people think it is safe to geo engineer the planet by dumping C02 in the air.

Large quantities of CO2 have been pumped into the atmosphere before by nature to levels far higher than those currently claimed to be tipping points to runaway warming – and the planet cooled. So you can forget your post-normal superstition.

The problem is that variations of ENSO can cause tremendous variation in atmospheric temperature at the surface and ENSO can not be predicted reliably in the future. It is also not cyclical in that you don’t always get an El Nino after a La Nina or vice versa. Having six consecutive El Nino events separated by neutral periods would be just as valid as any other outcome. Rather than using sunspot numbers, I might be more tempted to use something like F10.7 flux running average lagged by some period of time but that gives you some notion of general direction but nothing more than that. ENSO and volcanism can throw your predictions right out of the water and any kind of persistent change (such as a persistent negative NAO) can cause reinforcing feedbacks to those things and, again, make any prediction beyond a few months useless. The LIA onset was pretty quick and there is some evidence that there were some reinforcing feedbacks that kept it that way for a while. Trying to predict those, particularly when things like volcanism are involved can just be silly. The best you can do is something like “if overall conditions remain as they are, we expect the following, but if things change, then there will be changes”. And nobody likes to hear predictions like that.

Latitude

Steven Mosher says:
January 24, 2013 at 3:36 pm
yup. that is why it is weird that people think it is safe to geo engineer the planet by dumping C02 in the air.
========================
0.0003 to 0.00039 = 0.0001
0.01% of total

Gunga Din

Steven Mosher says:
January 24, 2013 at 3:36 pm
Think of all those drug commercials in the US. Lots of disclaimers. (Who knew an eye drop might kill you?8-) Sure, they have those disclaimers to legally CYA but they need to CYA because it could happen. Before a drug hits the market it’s been thoroughly tested. But they don’t know everything. The human body is incredibly complex.
“Climate” is incredbly complex.
###############
yup. that is why it is weird that people think it is safe to geo engineer the planet by dumping C02 in the air.
===========================================================
Or Olvatine in the ocean? 😎
Either way, we don’t know enough. If it wasn’t for the politics involved and the cost in $$ and freedoms that the policitians are after, let the honest scientist continue to … well … observe and apply the scientific method to any hypothsis that may arise from those observations. The Hansens and the Manns haven’t put out any ‘disclaimers’ that said they might be wrong or the consequences if they are. But the Gores and the Obamas are cashing in.
“People think it is safe to geo engineer the planet by dumping C02 in the air.”
People are just living. The only people I’ve heard of that are trying to “geo engineer” anything are those that have picked CO2 out of the chaos that makes “climate” and are trying to limit or remove it because they feel that what Man adds is somehow not “natural”.

oldfossil

I’ve spent a fair amount of time at climateprediction.net and generally I’m impressed. To quote from the home page:
Climateprediction.net is a distributed computing project to produce predictions of the Earth’s climate up to 2100 and to test the accuracy of climate models. To do this, we need people around the world to give us time on their computers – time when they have their computers switched on, but are not using them to their full capacity.
What do we ask you to do?
We need you to run a climate model on your computer. The model will run automatically as a background process on your computer whenever you switch your computer on and it should not affect any other tasks for which you use your computer. As the model runs, you can watch the weather patterns on your, unique, version of the world evolve. The results are sent back to us via the internet, and you will be able to see a summary of your results on this web site. Climateprediction.net uses the same underlying software, BOINC, as many other distributed computing projects and, if you like, you can participate in more than one project at a time.
Why do it?
Climate change, and our response to it, are issues of global importance, affecting food production, water resources, ecosystems, energy demand, insurance costs and much else. Current research suggests that the Earth will probably warm over the coming century; Climateprediction.net should, for the first time, tell us what is most likely to happen.

I don’t see a lot of hysteria going on here.

Bit off-topic – or maybe not?
http://hockeyschtick.blogspot.com.au/2013/01/new-paper-finds-why-weather-climate.html
Paper is released under Creative Commons:
http://www.hindawi.com/journals/ijgp/2012/863792/
I picked up on this as a result of a note I wrote years ago – “internal waves?” – as a result of observing hydraulic flume, high & low flow wind tunnel experiments.

It’s not safe to geo engineer the planet by dumping CO2 in the air. Nor is it unsafe. It’s just too bloody hard. That why there’s no CAGW or climate disruption or whatever we’re supposed to call it this week.
On the other hand, if we were foostering with windmills and solar panels when a Mount Tambora or Laki-style eruption occurred, then there’d be change! That change would involve climate and economics and all the stuff clever people like to “model” for the rest of us.
It’s odd that we don’t fuss much over the one trigger for radical climate change that is pretty inevitable and can come at any time. Behave, you Decade Volcanoes, till we can dismantle our neo-medieval piles of junk and get some nice new nukes and lots of fossil fuel power on stream. Governments! Make like Angela Merkel and talk green while digging brown.

pat

CO2 trading that isn’t working:
EU CO2 market fix hangs in balance after MEPs urge rejection
BRUSSELS, Jan 24 (Reuters Point Carbon) – A proposal to rescue the ailing EU carbon market hung in the balance on Thursday after a committee of European lawmakers urged the bloc’s parliament to reject efforts to rescue carbon prices, which crashed to record lows on the news…
http://www.pointcarbon.com/news/1.2152161
——————————————————————————–