Overshoot and Undershoot

Guest Post by Willis Eschenbach

Today I thought I’d discuss my research into what is put forward as one of the key pieces of evidence that GCMs (global climate models) are able to accurately reproduce the climate. This is the claim that the GCMs are able to reproduce the effects of volcanoes on the climate.

One of the most-cited papers in this regard is the Soden et al. study of the eruption of the Philippine volcano, Mt. Pinatubo. Their study is entitled Global Cooling After the Eruption of Mount Pinatubo: A Test of Climate Feedback by Water Vapor, available as a PDF. [hereinafter “Soden08”]

Figure 1. A NASA graphic showing satellite measurements of the spread of ash and aerosols from Mt. Pinatubo. In the first month (top right), the volcanic emissions had circled the earth (red area, Philippines on right). In six months, they were fairly evenly spread around the planet. Graphic Source NASA

The eruption of Mt. Pinatubo on 15 June 1991 injected aerosols and volcanic ash high into the atmosphere. This measurably changed the climate for a couple of years. It provided a wealth of observational data, as well as a good test for climate models.

Regarding the match between models and volcanic reality, the authors of Soden08 say (emphasis mine):

Because the transient response of the model depends on both its sensitivity and the external radiative forcing imposed on it, we first demonstrate the consistency between the model-simulated radiative forcing with that measured by satellites (Fig. 1 [of Soden08]). Both the observations and model simulations yield very similar reductions in the absorbed solar or shortwave (SW) radiation, which are nearly twice as large as the reduction in emitted LW radiation, a net loss of radiative energy that cools the surface and lower troposphere.

Parenthetically, when a scientist starts talking about “consistency” between observations and model results, I check my wallet. Consistency? What units are used to measure that? But I digress, back to my question.

How do the models actually stack up against the observations shown in their paper, in the Figure 1 they mention?

Before I get to the question and to their Figure 1, a bit of a diversion. For reasons that will become clear shortly, I want to make a distinction between simple negative feedback, and a governor. A governor is a system that keeps a heat engine running at a (relatively) constant speed. The most common example of a governor in daily life is the “cruise control” on a car. It keeps the car going at the same speed regardless of uphill and downhill grades.

Negative feedback is like increasing wind resistance on an accelerating car. Wind resistance slows the car down. Eventually at some speed wind resistance balances the energy pushing the car, and the car goes no faster. It balances out at a certain speed, where increasing feedback matches energy input. Fig. 2 shows how a negative feedback and a governor respond to increasing forcing.

Figure 2. A comparison of the actions of a governor and negative feedback. Qualitative only, numbers are nominal values.

Note that in response to changed forcings, the governor brings the value back to the desired equilibrium value. The governor does this by producing what is called “overshoot”. “Overshoot” is where the action of the governor drives the system past equilibrium (represented in Fig. 2 by the thick horizontal line at zero). This gives the governor the ability to recover quickly from perturbations.

Simple negative feedback, on the other hand, cannot maintain a specified speed. All it can do is reduce the size of a speed increase or a decrease . It cannot produce overshoot. As shown in the right side of the graph, simple negative feedback can create what appears to be a controlled equilibrium situation. This happens when feedback balances forcing so there is no change in speed.

However, this balance of negative feedback is not stable — any change in the forcing will lead to a new equilibrium speed. A governor, on the other hand, maintains the same speed despite changes in forcings.

Overshoot is necessary to control a “lagged” system such as the climate. This is a system where response to inputs is not instantaneous. I have argued elsewhere that the earth has at least one governor system incorporating overshoot which actively controls the temperature. For our current purposes, please take note of the very different shapes of the response curves of negative feedback and of a system with a governor.

With that as prologue, let us now look at the Soden08 Figure 1.

Soden08 Figure 1. ORIGINAL CAPTION. Comparison of the observed anomalies in absorbed SW (top) and emitted LW (bottom) radiative fluxes at the top of the atmosphere from Earth Radiation Budget Satellite observations (black) and three ensembles of GCM simulations (red). The observed anomalies are expressed relative to a 1984 to 1990 base climatology, and the linear trend is removed (30). The results are expressed relative to the pre-eruption (January to May 1991) value of the anomaly and smoothed with a 7-month running mean (thick line). The GCM anomalies are computed as the difference between the control and Mount Pinatubo simulations for each ensemble member. Both the model and observed global averages are from 60N-60S due to the restriction of observed data to these latitudes.

This looks good at first blush, and the authors say that the GCM results (red) are “consistent” with the observations. However, closer examination reveals issues. What struck me immediately about their results is that the actual observations of both the shortwave and longwave anomalies show clear signs of overshoot. After being knocked down by the volcano, after 1994 they both come back higher than pre-eruption. This worked to quickly restore the pre-disturbance state.

None of the GCMs show this sign of overshoot. Instead, the GCMs gradually drift back to the pre-eruption anomaly value of zero. This is similar to the negative feedback balance shown in Fig. 2. Unlike the overshoot in the observations, the GCM results flatline after 1994. While this doesn’t prove anything, it is another piece of evidence that the GCMs are missing some basic climate mechanisms.

How Much Total Difference did Pinatubo Make?

There is a second problem with the Soden08 model results, one which is less theoretical and more mathematically demonstrable. This has to do with the cumulative energy deficit from the volcanic eruption.

As you can see in the Soden08 Fig. 1 above, after the eruption the amount of incoming solar energy (SW, or shortwave radiation) dropped about twice as much as outgoing energy (LW, or longwave radiation). As a result, after the volcano there was a global net energy deficit. There was less energy entering the system than there was leaving the system. This deficit continued for some months.

The total magnitude of this deficit is an important indicator of the overall impact of the Pinatubo eruption on the climate system. It is a basic measurement of the phenomenon, answering the fundamental first question everyone asks — how big is it? We can investigate the total magnitude of the volcanic disturbance by looking at the cumulative energy deficit created by the eruption.

To do that, I first digitized the data in the Soden08 Figure 1. The data is available here as an Excel worksheet. Results in the worksheet for the GCMs are the average of the three runs shown in their Figure 1.

I then calculated the net energy balance (solar energy absorbed minus longwave energy emitted to space) for each month. I then started a cumulative total at zero, and added each month’s net energy balance to the cumulative total. This cumulative total shows how far out of balance the system was each month.

The resulting curve shows the size of the total disturbance caused by the volcano, as well as showing the path of recovery. The units are Watt-months/metre^2 (for convenience, since the data is monthly). For example, a two-Watt/m2 deficit that continued for three months would give a cumulative deficit of six Watt-months/m2. Fig. 3 illustrates the problem with the GCM results.

Figure 3. Cumulative effect of the Pinatubo eruption. Red line shows data from the average of the models (GCMs) shown in Soden08 Figure 1 above. Blue line shows data from the ERBS observations shown in Soden08 Figure 1 above.

So what does this result mean? Well, among other things it means that in this case the general “first glance” similarity of observations and models is misleading. A closer examination shows that the models did a very poor job at being consistent with the observations.

• The models greatly underestimated the magnitude of the peak impact of the eruption.

• They showed recovery starting much sooner than the observations show.

• They greatly underestimated the speed of the recovery once it started.

• They greatly underestimated the total impact of the eruption.

To put some numbers on those statements:

• The peak energy deficit in the observations was -42 Watt-months/m^2. This is more than twice the -18 Watt-months/m2 in the model results.

• The models show recovery starting about a year and a half after the eruption. The observations show about two and a half years before things turn around.

• The observed speed of the recovery is more than five times that of the modelled speed of recovery (post-1994 linear trends).

• The total impact of the eruption on the global energy balance is given by the area underneath the curves. Alternatively, it can be expressed as the average value of the energy deficit over the time period of the curves (1991-1995). The observed average energy deficit over that period is -21 Watt-months/m2. Again, this is more than twice the models’ estimate of -10 Watt-months/m2.

My conclusion? These models are not doing a credible job of representing Pinatubo’s effect on the global energy balance. The total size of the disturbance is a fundamental, basic measure of the accuracy of a simulation. Both the observed peak energy deficit and the observed total impact of the volcano were more than double the model results.

An error where the raw observed size of the phenomenon is more than double the model estimate? That sounds like a government project. Bad model, no cookies. Clearly, there is some fundamental problem with their simulation — they are showing the eruption of Mt. Minitubo, the half-size model.

When models show that kind of error in the raw size, peak size, and timing of the effects of a volcanic eruption … are the models “consistent” with the observations? Would you pay good money for a model that gave that size of error?

In addition, model results do not show the observed “overshoot” that leads to a speedy recovery from a disturbance. The results of this observed overshoot are seen in the difference between modelled and observed recovery rates. Driven by overshoot, the observed recovery rate is five times that shown by the models.

In short, I see no support for their implied claim that the models are an accurate representation of reality. Far from that, I don’t even see support for their vague claim of “consistency”. Their idea of consistency reminds me of the line from the old song, “She could easily pass for forty-threeeeee … in the dusk … with the light behind her.”

Their model results are inconsistent with observations. I do not see the Soden08 study as support for the idea that GCMs can successfully model the changes from volcanic eruptions.

Regards to all,

w.

PS – A final note for clarification. As the title suggests, the main thrust of Soden08 is concerned with whether the models perform better if they include a water vapor positive feedback. It finds that the GCMs do in fact perform better if there is positive feedback from a warming.

My analysis of the model results and observational data above is completely separate from the Soden08 analysis. We are simply using the same results and data. I make no claim that their analysis is right or wrong. In fact, I strongly suspect they are right, that the models do perform better if there is positive feedback from warming, although I have my own ideas what that demonstrates. But that is distinct from my analysis above.