Climate scientists can restart the climate change debate & win: test the models!

By Larry Kummer, from the Fabius Maximus website

Summary; Public policy about climate change has become politicized and gridlocked after 26 years of large-scale advocacy. We cannot even prepare for a repeat of past extreme weather. We can whine and bicker about who to blame. Or we can find ways to restart the debate. Here is the next of a series about the latter path, for anyone interested in walking it. Climate scientists can take an easy and potentially powerful step to build public confidence: re-run the climate models from the first 3 IPCC reports with actual data (from their future): how well did they predict global temperatures?

“Trust can trump Uncertainty.”

— Presentation by Leonard A Smith (Prof of Statistics, LSE), 6 February 2014.

The most important graph from the IPCC’s AR5

Figure 1.4 from p131 of AR5: the observed global surface temperature anomaly relative to 1961–1990 in °C compared with the range of projections from the previous IPCC assessments. Click to enlarge.

Why the most important graph doesn’t convince the public

Last week I posted What climate scientists did wrong and why the massive climate change campaign has failed. After 26 years, one of the largest longest campaigns to influence public policy has failed to gain the support of Americans, with climate change ranking near the bottom of people’s concerns. It described the obvious reason: they failed to meet the public’s expectations for behavior of scientists warning about a global threat (i.e., a basic public relations mistake).

Let’s discuss what scientists can do to restart the debate. Let’s start with the big step: show that climate models have successfully predicted future global temperatures with reasonable accuracy.

This spaghetti graph — probably the most-cited data from the IPCC’s reports — illustrates one reason for lack of sufficient public support in America. It shows the forecasts of models run in previous IPCC reports vs. actual subsequent temperatures, with the forecasts run under various scenarios of emissions and their baselines updated. First, Edward Tufte probably would laugh at this The Visual Display of Quantitative Information — too much packed into one graph, the equivalent of a PowerPoint slide with 15 bullet points.

But there’s a more important weakness. We want to know how well the models work. That is, how well each forecast if run with a correct scenario (i.e., actual future emissions, since we’re uninterested here in predicting emissions, just temperatures).

The big step: prove climate models have made successful predictions

“A genuine expert can always foretell a thing that is 500 years away easier than he can a thing that’s only 500 seconds off.”

— From Mark Twain’s A Connecticut Yankee in King Arthur’s Court.

A massive body of research describes how to validate climate models (see below), most stating that they must use “hindcasts” (predicting the past) because we do not know the temperature of future decades. Few sensible people trust hindcasts, with their ability to be (even inadvertently) tuned to work (that’s why scientists use double-blind testing for drugs where possible).

But now we know the future — the future of models run in past IPCC reports — and can test their predictive ability.

Karl Popper believed that predictions were the gold standard for testing scientific theories. The public also believes this. Countless films and TV shows focus on the moment in which scientists test their theory to see if the result matches their prediction. Climate scientists can run such tests today for global surface temperatures. This could be evidence on a scale greater than anything else they’ve done.

Testing the climate models used by the IPCC

“Probably {scientists’} most deeply held values concern predictions: they should be accurate; quantitative predictions are preferable to qualitative ones; whatever the margin of permissible error, it should be consistently satisfied in a given field; and so on.”

— Thomas Kuhn in The Structure of Scientific Revolutions (1962).

The IPCC’s scientists run projections. AR5 describes these as “the simulated response of the climate system to a scenario of future emission or concentration of greenhouse gases and aerosols … distinguished from climate predictions by their dependence on the emission/concentration/radiative forcing scenario used…”. The models don’t predict CO2 emissions, which are an input to the models.

So they should run the models as they were when originally run for the IPCC in the First Assessment Report (FAR, 1990), in the Second (SAR, 1995), and the Third (TAR, 2001). Run them using actual emissions as inputs and with no changes of the algorithms, baselines, etc. How accurately will the models’ output match the actual global average surface temperatures?

Of course, the results would not be a simple pass/fail. Such a test would provide the basis for more sophisticated tests. Judith Curry (Prof Atmospheric Science, GA Inst Tech) explains here:

Comparing the model temperature anomalies with observed temperature anomalies, particularly over relatively short periods, is complicated by the acknowledgement that climate models do not simulate the timing of ENSO and other modes of natural internal variability; further the underlying trends might be different. Hence, it is difficult to make an objective choice for matching up the observations and model simulations. Different strategies have been tried… matching the models and observations in different ways can give different spins on the comparison.

On the other hand, we now have respectably long histories since publication of the early IPCC reports: 25, 20, and 15 years. These are not short periods, even for climate change. Models that cannot successfully predict over such periods require more trust than many people have when it comes to spending trillions of dollars — or even making drastic revisions to our economic system (as Naomi Klein and the Pope advocate).

Conclusion

Re-run the models. Post the results. More recent models presumably will do better, but firm knowledge about performance of the older models will give us useful information for the public policy debate. No matter what the results.

As the Romans might have said when faced with a problem like climate change: “Fiat scientia, ruat caelum.” (Let science be done though the heavens may fall.)

“In an age of spreading pseudoscience and anti-rationalism, it behooves those of us who

believe in the good of science and engineering to be above reproach whenever possible.“

— P. J. Roach, Computing in Science and Engineering, Sept-Oct 2004 — Gated.

Other posts in this series

These posts sum up my 330 posts about climate change.

1. How we broke the climate change debates. Lessons learned for the future.
2. A new response to climate change that can help the GOP win in 2016.
3. The big step climate scientists can make to restart the climate change debate – & win.

For More Information

(a) Please like us on Facebook, follow us on Twitter, and post your comments — because we value your participation. For more information see The keys to understanding climate change and My posts about climate change. Also see these about models…

• About models, increasingly often the lens through which we see the world.
• Will a return of rising temperatures validate the IPCC’s climate models?
• We must rely on forecasts by computer models. Are they reliable?
• A frontier of climate science: the model-temperature divergence.
• Do models accurately predict climate change?

(b) I learned much, and got several of these quotes, from a 2014 presentations by Leonard A Smith (Prof of Statistics, LSE): the abridged version “The User Made Me Do It” and the full version “Distinguishing Uncertainty, Diversity and Insight“. Also see “Uncertainty in science and its role in climate policy“, Leonard A. Smith and Nicholas Stern, Phil Trans A, 31 October 2011.

(c)  Introductions to climate modeling

These provide an introduction to the subject, and a deeper review of this frontier in climate science.

• “A Model World” by Jon Turney in Aeon, 16 December 2013.
• “Climate Modeling 101: What are climate models and why are they important?” by the National Academy of Science.
• An introduction to climate models by the World Meteorological Society.
• “The Physics of Climate Modeling” by Gavin A. Schmidt in Physics Today, January 2007.

Judith Curry (Prof Atmospheric Science, GA Inst Tech) reviews the literature about the uses and limitation of climate models…

1. What can we learn from climate models?
2. Philosophical reflections on climate model projections.
3. Spinning the climate model – observation comparison — Part I.
4. Spinning the climate model – observation comparison: Part II.

(d)  Selections from the large literature about validation of climate models

• “How Well Do Coupled Models Simulate Today’s Climate?“, BAMS, March 2008 — Comparing models with the present, but defining “present” as the past (1979-1999).
• “Should we believe model predictions of future climate change?”, Reto Knutti, Philosophical Transactions A, December 2008.
• “Should we assess climate model predictions in light of severe tests?”, Joel Katzav, Eros, 7 June 2011.
• “Reliability of multi-model and structurally different single-model ensembles“, Tokuta Yokohata et al, Climate Dynamics, August 2012. Uses the rank histogram approach.
• “The Elusive Basis of Inferential Robustness“, James Justus, Philosophy of Science, December 2012. A creative look at a commonly given reason to trust GCMs.
• “Test of a decadal climate forecast“, Myles R. Allen et al, Nature Geoscience, April 2013 — Gated. Test of one model’s forecasts over subsequent 10 years. Doesn’t state what emissions data  used for validation (scenario or actual). The forecast was significantly below consensus, and so quite accurate. Which is why we hear about it.
• “Overestimated global warming over the past 20 years” by John C. Fyfe et al, Nature Climate Change, Sept 2013.
• “Can we trust climate models?” J. C. Hargreaves and J. D. Annan, Wiley Interdisciplinary Reviews: Climate Change, July/August 2013.
• “The Robustness of the Climate Modeling Paradigm“, Alexander Bakker, Ph.D. thesis, VU University (2015).
• “Uncertainties, Plurality, and Robustness in Climate Research and Modeling: On the Reliability of Climate Prognoses“, Anna Leuschner, Journal for General Philosophy of Science, in press. Typical cheerleading; proof by bold assertion.