Climate Models shown to be inaccurate less than 30 years out

From the University of Arizona (h/t to WUWT reader Miguel Rakiewicz):

A new study has found that climate-prediction models are good at predicting long-term climate patterns on a global scale but lose their edge when applied to time frames shorter than three decades and on sub-continental scales.

These maps show the observed (left) and model-predicted (right) air temperature trend from 1970 to 1999. The climate model developed by the National Center for Atmospheric Research (NCAR) is used here as an example. More than 50 such simulations were analyzed in the published study. (Illustration: Koichi Sakaguchi)

Climate-prediction models show skills in forecasting climate trends over time spans of greater than 30 years and at the geographical scale of continents, but they deteriorate when applied to shorter time frames and smaller geographical regions, a new study has found.

Published in the Journal of Geophysical Research-Atmospheres, the study is one of the first to systematically address a longstanding, fundamental question asked not only by climate scientists and weather forecasters, but the public as well: How good are Earth system models at predicting the surface air temperature trend at different geographical and time scales?

Xubin Zeng, a professor in the University of Arizona department of atmospheric sciences who leads a research group evaluating and developing climate models, said the goal of the study was to bridge the communities of climate scientists and weather forecasters, who sometimes disagree with respect to climate change.

According to Zeng, who directs the UA Climate Dynamics and Hydrometeorology Center, the weather forecasting community has demonstrated skill and progress in predicting the weather up to about two weeks into the future, whereas the track record has remained less clear in the climate science community tasked with identifying long-term trends for the global climate.

“Without such a track record, how can the community trust the climate projections we make for the future?” said Zeng, who serves on the Board on Atmospheric Sciences and Climate of the National Academies and the Executive Committee of the American Meteorological Society. “Our results show that actually both sides’ arguments are valid to a certain degree.”

“Climate scientists are correct because we do show that on the continental scale, and for time scales of three decades or more, climate models indeed show predictive skills. But when it comes to predicting the climate for a certain area over the next 10 or 20 years, our models can’t do it.”

To test how accurately various computer-based climate prediction models can turn data into predictions, Zeng’s group used the “hindcast” approach.

“Ideally, you would use the models to make predictions now, and then come back in say, 40 years and see how the predictions compare to the actual climate at that time,” said Zeng. “But obviously we can’t wait that long. Policymakers need information to make decisions now, which in turn will affect the climate 40 years from now.”

Zeng’s group evaluated seven computer simulation models used to compile the reports that the Intergovernmental Panel on Climate Change, or IPCC, issues every six years. The researchers fed them historical climate records and compared their results to the actual climate change observed between then and now.

“We wanted to know at what scales are the climate models the IPCC uses reliable,” said Koichi Sakaguchi, a doctoral student in Zeng’s group who led the study. “These models considered the interactions between the Earth’s surface and atmosphere in both hemispheres, across all continents and oceans and how they are coupled.”

Zeng said the study should help the community establish a track record whose accuracy in predicting future climate trends can be assessed as more comprehensive climate data become available.

“Our goal was to provide climate modeling centers across the world with a baseline they can use every year as they go forward,” Zeng added. “It is important to keep in mind that we talk about climate hindcast starting from 1880. Today, we have much more observational data. If you start your prediction from today for the next 30 years, you might have a higher prediction skill, even though that hasn’t been proven yet.”

The skill of a climate model depends on three criteria at a minimum, Zeng explained. The model has to use reliable data, its prediction must be better than a prediction based on chance, and its prediction must be closer to reality than a prediction that only considers the internal climate variability of the Earth system and ignores processes such as variations in solar activity, volcanic eruptions, greenhouse gas emissions from fossil fuel burning and land-use change, for example urbanization and deforestation.

“If a model doesn’t meet those three criteria, it can still predict something but it cannot claim to have skill,” Zeng said.

According to Zeng, global temperatures have increased in the past century by about 1.4 degrees Fahrenheit or 0.8 degrees Celsius on average. Barring any efforts to curb global warming from greenhouse gas emissions, the temperatures could further increase by about 4.5 degrees Fahrenheit (2.5 degrees Celsius) or more by the end of the 21st century based on these climate models.

“The scientific community is pushing policymakers to avoid the increase of temperatures by more than 2 degrees Celsius because we feel that once this threshold is crossed, global warming could be damaging to many regions,” he said.

Zeng said that climate models represent the current understanding of the factors influencing climate, and then translate those factors into computer code and integrate their interactions into the future.

“The models include most of the things we know,” he explained, “such as wind, solar radiation, turbulence mixing in the atmosphere, clouds, precipitation and aerosols, which are tiny particles suspended in the air, surface moisture and ocean currents.”

Zeng described how the group did the analysis: “With any given model, we evaluated climate predictions from 1900 into the future – 10 years, 20 years, 30 years, 40 years, 50 years. Then we did the same starting in 1901, then 1902 and so forth, and applied statistics to the results.”

Climate models divide the Earth into grid boxes whose size determines its spatial resolution. According to Zeng, state of the art is about one degree, equaling about 60 miles (100 kilometers).

“There has to be a simplification because if you look outside the window, you realize you don’t typically have a cloud cover that measures 60 miles by 60 miles. The models cannot reflect that kind of resolution. That’s why we have all those uncertainties in climate prediction.”

“Our analysis confirmed what we expected from last IPCC report in 2007,” said Sakaguchi. “Those climate models are believed to be of good skill on large scales, for example predicting temperature trends over several decades, and we confirmed that by showing that the models work well for time spans longer than 30 years and across geographical scales spanning 30 degrees or more.”

The scientists pointed out that although the IPCC issues a new report every six years, they didn’t see much change with regard to the prediction skill of the different models.

“The IPCC process is driven by international agreements and politics,” Zeng said. “But in science, we are not expected to make major progress in just six years. We have made a lot of progress in understanding certain processes, for example airborne dust and other small particles emitted from surface, either through human activity or through natural sources into the air. But climate and the Earth system still are extremely complex. Better understanding doesn’t necessarily translate into better skill in a short time.”

“Once you go into details, you realize that for some decades, models are doing a much better job than for some other decades. That is because our models are only as good as our understanding of the natural processes, and there is a lot we don’t understand.”

Michael Brunke, a graduate student in Zeng’s group who focused on ocean-atmosphere interactions, co-authored the study, which is titled “The Hindcast Skill of the CMIP Ensembles for the Surface Air Temperature Trend.”

Funding for this work was provided by NASA grant NNX09A021G, National Science Foundation grant AGS-0944101 and Department of Energy grant DE-SC0006773.

Advertisements

  Subscribe  
newest oldest most voted
Notify of
MarkW

If models have such a good track record predicting more than 30 years out, then why were the predictions made back in the 80’s, so far off?

Jack

Zheng is applying for more of that sweet AGW grant money.

Alvin

Come on guys, who mounts a Precision station in a server rack?

Alvin

So, the conclusion is that their current data is wrong and they know that, but if you will just believe them for 30 years then it will all clear up. Thirty years of global climate change based socialist policies. What could go wrong?

Myron Mesecke

This is just the latest excuse for why the models didn’t predict the current cooling.
“But it will warm back up! Just give it 40 years.”

“Climate-prediction models show skills in forecasting climate trends over time spans of greater than 30 years”
How do they know?

I love the way the ‘heat’ just kind of stays put.
As if someone turned off the surface winds, the jet-stream and the ocean currents then forgot to turn them back on again.
Don’t these people ever look at the animations from weather satellites?

PaulH

“Policymakers need information to make decisions now, which in turn will affect the climate 40 years from now.”
I am afraid this is wishful thinking on their part.

Louis

How can you possibly know that climate-prediction models “are good at predicting long-term climate patterns on a global scale” if they have gotten everything wrong so far? What principle of science allows you to assume that model predictions will be right in the future? (I can’t predict what the stock market will do over the next 30 years but I know exactly what the market will do a hundred years from now.)

You mean there is a climate model that is accurate minutes after the run(s)?

Stuck-Record

Brilliant. The get out of jail free card for modeller’s inaccuracy.
“Look. I know we’re totally wrong about everything that has happened, or will happen in the lifetimes of your career/industry/economy, but trust us anyway because we’ll be proved right after we’ve retired and are living on our untouchable pensions.”

JJ

“Xubin Zeng, a professor in the University of Arizona department of atmospheric sciences who leads a research group evaluating and developing climate models, said the goal of the study was to bridge the communities of climate scientists and weather forecasters, who sometimes disagree with respect to climate change.”
Huh. The goal of the study was political, not scientific.
Imagine that.

cd_uk

Hindcasts. Surely if you’re testing your models on the training set you used to fine tune them in the first place, then you’re likely to emulate the training set – it might also explain why they don’t do so well at a regional scale. This is retrospective fitting, and as far as I know you cannot use it to validate the predictive properties of a model – so I’m not sure how this got published unless they actually said something far more nuanced in the paper.
The other thing to note is that there is drift in the control data (the 20th century warming). This is a real problem as a basic stochastic model with a structural drift would probably be just as good. Therefore the test should have included such a baseline where the drift is an extrapolation of the late 19th century trend through the 20th century.
But then we’ve been through this all before and they’ll wheel another one out before another conference in a few years time.

So one what BASIS did they assure they were “accurate” for time periods more than 3 decades?
Sophistry, I think that’s the term here. A more gentle term than “pure fabrication”…
Max

“The scientific community is pushing policymakers to avoid the increase of temperatures by more than 2 degrees Celsius”
In other news, King Canute is going to stop the tide rising.

Bruckner8

Um, hourly updates [via model!] to Hurricane Isaac were incorrect…

H.R.

How can they know that the models are good longer than 30 years out? The earliest models are barely 30 years old and as MarkW points out, the track record isn’t so good. We’ll have to wait until 2040 to find out if the most recent models are any good. Sooo… I’ll let my heirs be the judge of how good the models are in 30-50-100 years.
Meanwhile, I’m totally on board that they’re not very good under 30 years. Darn proud they recognize that and they deserve an attaboy.

Roger Longstaff

This is complete nonsense. GCMs use numerical time step integration to calculate a “climate trajectory”. They lose contact with reality after just a few days of simulated elapsed time. In order to avoid violation of physical laws (conservation of momentum and energy) they employ low pass filters and pause/reset/restart techniques. The UK Metoffice models are completely useless at forecasting weather for the coming season, yet we are asked to believe that the same models can somehow recover numerical fidelity and proceed to make a meaningful prediction of the climate decades into the future, and that the more money we spend on models and supercomputers the more accurate the predictions will become.
It is mathematically impossible to model a complex, multivariate, non-linear and sometime chaotic system using simple deterministic equations. Hindcasting is NOT validation. The models are neither verified nor validated.

Disko Troop

The trouble is that with Hansen and Jones and the like in charge we can’t even predict the past anymore. They keep changing it. I’ve had to go back and change all my photographs.
Ivor Ward

Dave

Looks like more Climate Related Atmospheric Predictions (CRAP) from a recipient of the gravy train

JJ

““Climate scientists are correct because we do show that on the continental scale, and for time scales of three decades or more, climate models indeed show predictive skills. But when it comes to predicting the climate for a certain area over the next 10 or 20 years, our models can’t do it.”
Funny, because we have been told not only that they can do it, but that they have done it. Hansen is still running around claiming that his predictions from the 1980’s are demonstrably accurate. Maybe you fellahs should dedicate a little time to setting him straight. Not gonna hold my breath waiting for that to happen.
““Ideally, you would use the models to make predictions now, and then come back in say, 40 years and see how the predictions compare to the actual climate at that time,” said Zeng. “But obviously we can’t wait that long.”
Not at all obvious to me. Especially given the fact that we do have some prominent and dire predictions made 20 and 30 years ago that have turned out bust. Given that global temps are flat when you guys said that they would be rising precipitously, I say we wait and find out what else you are wrong about.
“Policymakers need information to make decisions now, …”
The information that the policy makers need now is the candid and forthright admission that we don’t have sufficient information right now to make science based decisions about the future.
Policymakers could then rely on adaptive management strategies that do not require extended predictions for decades or centuries out. For scientists to push that logical decision making process away from adaptive management toward untested predictions of doom and gloom scenarios is for scientists to act as policymakers.
“The scientific community is pushing policymakers to avoid the increase of temperatures by more than 2 degrees Celsius because we feel that once this threshold is crossed, global warming could be damaging to many regions,” he said.”
See, when you “push” you arent doing science. You are making policy.

wsbriggs

I’m with MarkW, given that they’ve been so wrong so far, how can they conclude that they’re accurate 100 years out? WUWT? To say they’re accurate in any meaning of the word, would mean that we could compare what they projected (not predicted), and what actually occurred. We’re a long way from that.

Gunga Din

Apparently the models miscounted the number of butterflies flapping their wings 40 years ago. Otherwise they’d be spot on.

Green Sand

Taking any forecast about any subject, just where would you expect the greatest degree of accuracy? Close to the inception of the forecast or at the end of the projected timescale?

“We wanted to know at what scales are the climate models the IPCC uses reliable,” said Koichi Sakaguchi, a doctoral student in Zeng’s group who led the study. “These models considered the interactions between the Earth’s surface and atmosphere in both hemispheres, across all continents and oceans and how they are coupled.”
  The boldface portion is utter nonsense.

DocMartyn

It’s like me with race horses, I can predict the winning horse in every horse race that will be run 100 years hence, but doing so just before the start causes my abilities to wilt.

Mickey Reno

In a crowd of 30 random people, the odds are about 50-50 that two of those people will have the same birthday. Hmmm, I wonder why I brought that up?

Ally E.

“Ideally, you would use the models to make predictions now, and then come back in say, 40 years and see how the predictions compare to the actual climate at that time,” said Zeng. “But obviously we can’t wait that long. Policymakers need information to make decisions now, which in turn will affect the climate 40 years from now.”
*
How convenient. Wasn’t the tipping point supposed to come by 2012? Why does this 30-40 year margin keep tracking into the future? It’s ALWAYS 30-40 years away, always the next generation when “trouble will show”. C’mon, even the warmest of the warmists must be wondering about that by now. We’ve HAD our 30-40 years. Pack it in already. We KNOW it’s for the money. Sheesh!

Richard M

I think we need a new acronym for climate modelling, MIGO … money in … garbage out.

TinyCO2

I suspect that Bernie Madoff might have said his investments would have come good if only they’d given him another thirty years.

LongCat

So models that are built to show warming no matter what can effectively hindcast a warming period.
Shocking.

richardscourtney

Friends:
At September 18, 2012 at 1:42 pm Roger Longstaff says all that needs to be said about the climate models.
But it needs to be said loudly, again and again and again and …
Richard

Richard of NZ

When a report does not even get its description of resolution correct one wonders what else they have wrong. One degree longitude is 60 nautical miles, about 111 kilometres, one degree latitude varies from 60 nautical miles at the equator to zero distance at the poles. They have managed to get their resolution out by 11% understated to infinitely overstated.

TinyCO2

This means that Hansen’s 1988 prediction should be accurate in another six years. Wow, that’s a big El Nino.

JJ

The skill of a climate model depends on three criteria at a minimum, Zeng explained. The model has to use reliable data, its prediction must be better than a prediction based on chance, and its prediction must be closer to reality than a prediction that only considers the internal climate variability of the Earth system and ignores processes such as variations in solar activity, volcanic eruptions, greenhouse gas emissions from fossil fuel burning and land-use change, for example urbanization and deforestation.
Given that what we think we know about the relationship between the internal climate variability of the Earth system and fossil fuel burning is determined by the choice of the assumptions designed into the models, such a comparison of predictive skill is not possible. You don’t know clouds, for example. If you don’t know clouds, you can’t make an “internal climate variability only” based prediction. And given that you arrive at the alleged fossil fuel effect by making assumptions about how clouds are acting when you calibrate the models, you can’t get the other one either.
Zeng described how the group did the analysis: “With any given model, we evaluated climate predictions from 1900 into the future – 10 years, 20 years, 30 years, 40 years, 50 years. Then we did the same starting in 1901, then 1902 and so forth, and applied statistics to the results.
Then what period of data were the models calibrated to? Fifty years into the future from 1900 does not get you into the period of alleged anthro global warming, so you aren’t testing how well those components of the model work. Once you get in to the CO2 era, you’re overrunning the data you used to come up with the anthro parameters. You need to be freezing your parameterizations, making falsifyable predictions about unseen (i.e. future) data, and seeing how well you do.

Andi Cockroft

Colour me stupid, but if you use a series of data to create a model, and create a good model, then its backcast capabilities should be spot-on – no matter which way you run your backcast.
What this doesn’t tell us at all is its validity as a forecast model.
If I have a model that is based on the sine-wave of my electricity supply over the past year, it might seem a viable forecast for the frequency next week – except if I forget to pay my bill !!!!
External factors that are unexpected or not well understood can put a wrench in the works of any forecast model – what if the Sun refuses to play ball with the climate models !!!
Andi

Martin A

Hindcasting.
In the 1970’s, it became apparent to researchers on pattern recognition that a fundamental error was to test a pattern recognition system’s accuracy on the same data used to train it. This inevitably leads to over optimistic assessments of a system’s capability. The same error is now seen with climate models, where they are assessed on their ability to reproduce the statistics of the data used to tune them.

Philip Finck

cd-uk
My thoughts exactly. The models are tuned to match past observations. So if you go into the past, apply data from that point back and make a prediction for the future…. well…. it sure as heck better have some skill. It is kind of like starting at a stop sign and walking back one kilometer. You turn around and predict that if you walk a kilometer forward you will end up at a stop sign. Sigh.
“The models include most of the things we know,” he explained, “such as wind, solar radiation, turbulence mixing in the atmosphere, clouds, precipitation and aerosols, which are tiny particles suspended in the air, surface moisture and ocean currents.”
Hmmm…… I didn’t know `we know’ clouds, aerosols, precipitation, etc all that well, certainly not to qualify as `we know’.

Hot under the collar

“Once you go into details, you realize that for some decades, models are doing a much better job than for some other decades”.
Try tossing a coin guys you may get the same (or more accurate) predictions!

Coach Springer

Climate models > 30 years = Accuracy? Now we know that the advancing ice age consensus of the 70s was accurate for 2012. Good thing the study cleared that up, because there’s been some confusion about those predictions up until now. Too soon to tell about the 80s models, so they’re batting 0-for-1?
I respect the U of A basketball team more than their climate government-is-the-basis-of-science team. At least basketball scores reflect something that actually happened.

Gunga Din

Ally E. says:
September 18, 2012 at 2:09 pm
“Ideally, you would use the models to make predictions now, and then come back in say, 40 years and see how the predictions compare to the actual climate at that time,” said Zeng. “But obviously we can’t wait that long. Policymakers need information to make decisions now, which in turn will affect the climate 40 years from now.”
*
How convenient. Wasn’t the tipping point supposed to come by 2012? Why does this 30-40 year margin keep tracking into the future? It’s ALWAYS 30-40 years away, always the next generation when “trouble will show”. C’mon, even the warmest of the warmists must be wondering about that by now. We’ve HAD our 30-40 years. Pack it in already. We KNOW it’s for the money. Sheesh!
================================================================
I wonder if just as “Global Warming” became “Climate Change” we’ll start to see fewer actual years mentioned and more “years from now” in the predictions?
Again I’m reminded of the seafood place that advertised on their buildings, “Free crabs tomorrow!”.

Tazilon

How does anyone know what the models’ accuracy is 30+ years out? They are not 30 years old!

Sean

This paper is just nonsense.

pat

Of course accuracy will only increase with time as the variables narrow. Oh wait. That is absurd.

Dave

Knowing how they’ve “corrected” old data to undoubtedly increase the fit during hindcasting, perhaps they’ve already made up future data that fits the models perfectly. I wouldn’t put it past them…

DocMartyn

” Philip Finck says:
My thoughts exactly. The models are tuned to match past observations.”
no, No, No, and No. They SWEAR on all they love that the models are not fitted to the past in any way what so ever, pinky promise.
We have to take their word for it that they chose their various constants, a prior, and didn’t fit them to the past at all.

cui bono

Zeng described how the group did the analysis: “With any given model, we evaluated climate predictions from 1900 into the future – 10 years, 20 years, 30 years, 40 years, 50 years. Then we did the same starting in 1901, then 1902 and so forth, and applied statistics to the results.”
——-
So from 1900 they could predict – what? Arctic and Antarctic ice extent in the 1930s and 1940s? Air temperatures? Adjusted or unadjusted? Changes in ocean currents? Cloud cover? Rainfall? The 1930s dustbowl (oh no, sorry, too regional).
This is moonshine! Worse than fantasy.
Let Judith Curry check their methods. She had a post recently about abject lack of model testing.
And give the “applied statistics” to Lucia!

JJ

“The models include most of the things we know,” he explained, “such as wind, solar radiation, turbulence mixing in the atmosphere, clouds, precipitation and aerosols, which are tiny particles suspended in the air, surface moisture and ocean currents.”
Most? Ring back when your models include all of the things you (think you) know.
And BTW, you don’t know clouds. Or precip. Or aerosols. And we have a very good inkling that you don’t know solar as well as you need to.
“Once you go into details, you realize that for some decades, models are doing a much better job than for some other decades. That is because our models are only as good as our understanding of the natural processes, and there is a lot we don’t understand.”
Go with that. And understand that a lot of what you don’t understand, you don’t yet know you don’t understand. Until you do understand, STFU with respect to policy. You shouldn’t be “pushing” on anything, when your feet are anchored in ingorance.

richardscourtney

DocMartyn:
At September 18, 2012 at 3:06 pm you say

We have to take their word for it that they chose their various constants, a prior, and didn’t fit them to the past at all.

Really?
I do not know of any modeler who claims “they chose their various constants, a prior, and didn’t fit them to the past”.
And we know for certain fact that they DID fit to the past by use of assumed aerosol cooling.
(ref. Courtney RS ‘An assessment of validation experiments conducted on computer models of global climate using the general circulation model of the UK’s Hadley Centre’, Energy & Environment, Volume 10, Number 5, pp. 491-502, September 1999).
We also know that each climate model uses a different aerosol ‘fudge factor’ to every other climate model.
(ref. Kiehl JT,Twentieth century climate model response and climate sensitivity. GRL vol.. 34, L22710, doi:10.1029/2007GL031383, 2007).
So, at most only one of the climate models emulates the climate system of the real Earth and there are good reasons to think none of them do.
Richard

son of mulder

Have they tested the models they ran on the Babbage Difference Engine 100 years ago to show how accurately they predicted todays climate? (;>)