Tipping point 10 years on: Who won the Armstrong-Gore 'bet' on the climate?

Kesten C. Green University of South Australia (kesten.green at unisa.edu.au)

The Challenge

In 2007, University of Pennsylvania Professor J. Scott Armstrong challenged former U.S. Vice President Albert Gore to a bet on what would happen to global average temperatures over the next 10 years. Professor Armstrong’s challenge was in response to Mr. Gore’s warning of a looming dangerous “tipping point” in temperatures. But when even scientists who are expert in a field make predictions about complex situation without using scientific forecasting methods, their forecasts have no value. The proposed $10,000 bet, then, was intended to draw attention to the need to assess the predictive validity of climate forecasts in an objective manner.

Emails to Mr. Gore were unproductive: after several attempts at engagement, his staff informed Professor Armstrong that Mr. Gore did not take bets. The important question of whether public policies should be based on the alarming projections had not gone away, however, and so Armstrong commissioned theclimatebet.com site to track how the bet would have turned out had Gore accepted.

At the time of the challenge, Mr. Gore had been warning that climate was warming at such a rate that large public expenditures were needed in great haste in order to prevent disaster. His book Assault on Reason—published in April 2007—stated on p. 204: “Many scientists are now warning that we are moving closer to several ‘tipping points’ that could – within as little as ten years – make it impossible for us to avoid irretrievable damage of the planet’s habitability for human civilization.”

Formulating The Climate Bet

Mr. Gore did not quantify his dangerous warming forecast, and so the “business as usual” projection provided by the U.N. Intergovernmental Panel on Climate Change’s Third Assessment Report in 2001 was used to represent his forecast. Using the IPCC projection of 3°C per-century warming favored Mr. Gore’s side of the bet because it was considerably less dramatic than the “tipping point” claims he was articulating and some of the IPCC’s own more extreme projections.

Professor Armstrong’s side of the bet was that the global average temperature would not change. The no-change forecast is consistent with a statement in the body of the aforementioned IPCC technical report. The report stated, “In climate research and modeling, we should recognize that we are dealing with a coupled non-linear chaotic system, and therefore that the long-term prediction of future climate states is not possible.” In other words, forecasting long-term trends in climate is impossible and, by implication, forecasting long-term changes in global mean temperatures is impossible.

The IPCC statement is also consistent with Green, Armstrong and Soon’s (2009) conclusion that the forecast of no-change in global temperatures over the long term would be hard to beat in terms of accuracy, even when applied to the U.K. Met Office Hadley Centre’s questionable annual average temperature data—based on adjusted thermometer readings from selected sites from 1850—that is used by the IPCC. Green, Armstrong and Soon found that no-change forecasts were so accurate for practical purposes—e.g., average errors of only ⁺/- 0.24°C for 50-year-ahead forecasts—that there would be no point in trying to do better.

The IPCC 3°C-per-century projection not only ignored their own authors’ conclusion about the inability to predict long-term trends, their procedures violated 72 of 89 relevant forecasting principles (Green and Armstrong 2007). As a consequence, there is no reason to expect the IPCC dangerous warming projection to be accurate over the long term, and thus no good reason for using it as the basis for policy.

Global temperatures have always varied on all time scales, however, so it was quite possible that Armstrong would lose a ten-year bet when temperatures have commonly drifted up or down by 0.3°C over ten-year periods in the past. A 150-year simulation of the bet suggested that his chance of winning was only about 70%.

Determining The Winner

In the end, the bet was offered, and monitored, on the basis of satellite temperature data from the University of Alabama at Huntsville (UAH). In contrast to surface data, the lower troposphere satellite data covers the whole Earth, is fully disclosed, and is not contaminated by poor maintenance and location of weather stations, changes from mercury to electronic measurement, and unexplained adjustments.

The cumulative absolute error (measure 1 in the table) was the key criterion for assessing accuracy. That measure has been tested and shown to be the best way to compare the accuracy of forecasts from different forecasting methods (Armstrong and Collopy 1992). By that measure, the no-change forecast reduced forecast errors by 12% compared to the IPCC dangerous warming projection. Forecasting models that cannot provide forecasts that are more accurate than the no-change forecast have no practical value.

The finding is consistent with Green and Armstrong’s 2014 analyses that compared the predictive validity of the no-change and IPCC forecasts over different time periods. The Loehle AD16 to 1935 temperature series was used to compare the accuracy of the global warming, global cooling, and no change forecasts over horizons from one to 100 years ahead; the no-change hypothesis was much more accurate than the global cooling hypothesis of 1°C-per-century cooling, which, in turn, was much more accurate than the global warming hypothesis of 3°C-per-century warming.

Alternative Measures of Winning

There are other ways that one might assess accuracy, especially since the bet was tracked each month, but the outcome is clear: there was no dangerous “tipping point” over the ten-year period. Global temperatures fell well within the range of natural variation. Seven alternative measures are presented in the table below so that readers can make their own assessments.

^1,2 Measures of forecast bias, with zero being totally unbiased and a positive number indicating a bias towards warming.

^3,4 The commonly used Ordinary Least Squares (OLS) procedure for estimating relationships violates the “Golden Rule of Forecasting” by failing to be sufficiently conservative. The lack of conservatism arises because the method’s use of squared errors gives extra weight to extreme values. The Least Absolute Deviation (LAD) procedure avoids that problem, and is consistent with our primary criterion for assessing accuracy: the relative cumulative absolute error. The OLS trend for the 120 months of the bet was 1.53°C per century, and the LAD trend 1.17°C per century. Note that the LAD trend over the last 10 years is somewhat lower than the trend for the whole 469 months of the UAH series to date, at 1.28°C per century: the opposite of a “tipping point”.

The second and third measures in the table provide measures of bias in the forecasts. By both measures, the no-change forecast is substantially (18% and 79%) less biased.

Another way to look at bias is shown in the chart: the blue shading indicates the difference between the Armstrong/no-change forecast and the actual temperature when the forecast was too warm, and the red shading indicates the difference between the Gore/IPCC projection and the actual temperature when the projection was too cold. A perfectly unbiased forecast would have 50% over- and 50% under-forecast errors. The sum of the Armstrong/no-change over-forecast-errors was 41% of the corresponding total absolute error, whereas the sum of the Gore/IPCC under-forecast-errors was only 14%. In other words, the Armstrong/no-change model produced forecasts that were close to unbiassed over the 10 year period, whereas the Gore/IPCC model projection was grossly biassed to forecast too warm, to the extent that the Gore/IPCC error from forecasting too warm was six times larger than the error from forecasting too cool.

How would planners who had relied on the official IPCC projection have fared? Looking again at the chart, planners would have expected “extra” warmth represented by the area of the triangle between the red IPCC line and the green no-change line (19.8). The extra warmth actually experienced over the period is represented by the area between the black actual temperature line and the green line: the area above the green no-change line, less the area below the no-change line (3.47). In other words, planners relying on the IPCC projection would have experienced less than 18% of the extra warmth that they had planned for.

Those who insist on looking for a trend in the decade of seesawing temperatures will find no support for either a “tipping point,” or the IPCC’s dangerous warming trend projection, in the best-fit line, which runs at a rate of little more than 1°C per-century[a]. As the footnote to the table explains, the fitted ordinary least squares (OLS) trend from the 2007 annual average base was 1.53°C per-century, and the least absolute deviation fitted (LAD) trend was 1.17°C per-century. When the bet forecasts are assessed against the OLS trend, the Gore/IPCC 3°C per-century projection is slightly closer (measure 5 in the table), but the Armstrong/no-change forecast is 44% closer to the more relevant LAD trend (measure 6). (Why would decision makers want to minimize squared errors?) Moreover, the trend line for the period of the bet was closer to no-change than was the trend over the entire UAH temperature anomaly series to the end of 2017.

The arbitrariness of fitting a trend—by whatever method—to such a series is reinforced by the fact that if the bet had been for five years, rather than 10, the fitted trend would have been negative: -1.13°C per century (LAD), or -1.61°C per century (OLS). Note also that on 10 February 2007, Sir Richard Branson was accompanied by Mr. Gore when he stated that the “world may already have crossed a ‘tipping point’”, and so one might ask whether the temperature at the end of the bet was dramatically higher that it was then, when the January 2007 UAH figure of 0.43°C had just been released. The answer is no: the December 2017 figure was lower at 0.41°C.

Global Temperatures from 2007 through 2017 v No-Change and 3°C/Century Warming

(Temperature deviation from 1981-2010 average, in degrees C*)

*www.nsstc.uah.edu/data/msu/v6.0/tlt/uahncdc_lt_6.0.txt (updated January 4 2018 from Version 6.0 as of April 2015)

The Future of The Climate Bet

Longer is better for assessing climate forecasts, and so theclimatebet.com site will monitor the “bet” in line with Scott Armstrong’s offer to extend the challenge for another ten years by sticking with the original 2007 annual average global temperature as the starting point. Extending the bet is intended to help further publicize the important role of scientific validation of forecasts that influence public policy. Policymakers should reject forecasts that fail to reduce errors compared to an appropriate no-change benchmark.

[a] Technical note to facilitate replication: The Climate Bet was framed in terms of what would happen to temperatures relative to the 2007 average, the year in which Mr. Gore warned of a “tipping point” and Professor Armstrong tried to get him to engage in a bet. Logically, then, if one insists on fitting a line through such a volatile time series, the starting point should represent the situation at the time the claim (tipping point) and challenge (offer of bet were made). The Gore tipping point claim was made in early 2007, so one could make a case for fitting trend lines with an origin at .43C (January 2007) but, given that discussions proceeded over the 2007 year and that monthly temperatures are so volatile, the 2007 average was chosen as the base for the bet; hence, also, as the value of Professor Armstrong’s no-change forecast, and the origin for trend line fitting. Given that the data are monthly and the origin was an annual average, lines are fitted with the origin located in mid-2007. A close look at the chart reveals that the red IPCC/Gore +3°C per century line is also projected on that basis: it is slightly above the green no-trend/2007-average line at the beginning of 2008.