Guest Post by Willis Eschenbach
Today I thought I’d discuss my research into what is put forward as one of the key pieces of evidence that GCMs (global climate models) are able to accurately reproduce the climate. This is the claim that the GCMs are able to reproduce the effects of volcanoes on the climate.
One of the most-cited papers in this regard is the Soden et al. study of the eruption of the Philippine volcano, Mt. Pinatubo. Their study is entitled Global Cooling After the Eruption of Mount Pinatubo: A Test of Climate Feedback by Water Vapor, available as a PDF. [hereinafter “Soden08”]
Figure 1. A NASA graphic showing satellite measurements of the spread of ash and aerosols from Mt. Pinatubo. In the first month (top right), the volcanic emissions had circled the earth (red area, Philippines on right). In six months, they were fairly evenly spread around the planet. Graphic Source NASA
The eruption of Mt. Pinatubo on 15 June 1991 injected aerosols and volcanic ash high into the atmosphere. This measurably changed the climate for a couple of years. It provided a wealth of observational data, as well as a good test for climate models.
Regarding the match between models and volcanic reality, the authors of Soden08 say (emphasis mine):
Because the transient response of the model depends on both its sensitivity and the external radiative forcing imposed on it, we first demonstrate the consistency between the model-simulated radiative forcing with that measured by satellites (Fig. 1 [of Soden08]). Both the observations and model simulations yield very similar reductions in the absorbed solar or shortwave (SW) radiation, which are nearly twice as large as the reduction in emitted LW radiation, a net loss of radiative energy that cools the surface and lower troposphere.
Parenthetically, when a scientist starts talking about “consistency” between observations and model results, I check my wallet. Consistency? What units are used to measure that? But I digress, back to my question.
How do the models actually stack up against the observations shown in their paper, in the Figure 1 they mention?
Before I get to the question and to their Figure 1, a bit of a diversion. For reasons that will become clear shortly, I want to make a distinction between simple negative feedback, and a governor. A governor is a system that keeps a heat engine running at a (relatively) constant speed. The most common example of a governor in daily life is the “cruise control” on a car. It keeps the car going at the same speed regardless of uphill and downhill grades.
Negative feedback is like increasing wind resistance on an accelerating car. Wind resistance slows the car down. Eventually at some speed wind resistance balances the energy pushing the car, and the car goes no faster. It balances out at a certain speed, where increasing feedback matches energy input. Fig. 2 shows how a negative feedback and a governor respond to increasing forcing.
Figure 2. A comparison of the actions of a governor and negative feedback. Qualitative only, numbers are nominal values.
Note that in response to changed forcings, the governor brings the value back to the desired equilibrium value. The governor does this by producing what is called “overshoot”. “Overshoot” is where the action of the governor drives the system past equilibrium (represented in Fig. 2 by the thick horizontal line at zero). This gives the governor the ability to recover quickly from perturbations.
Simple negative feedback, on the other hand, cannot maintain a specified speed. All it can do is reduce the size of a speed increase or a decrease . It cannot produce overshoot. As shown in the right side of the graph, simple negative feedback can create what appears to be a controlled equilibrium situation. This happens when feedback balances forcing so there is no change in speed.
However, this balance of negative feedback is not stable — any change in the forcing will lead to a new equilibrium speed. A governor, on the other hand, maintains the same speed despite changes in forcings.
Overshoot is necessary to control a “lagged” system such as the climate. This is a system where response to inputs is not instantaneous. I have argued elsewhere that the earth has at least one governor system incorporating overshoot which actively controls the temperature. For our current purposes, please take note of the very different shapes of the response curves of negative feedback and of a system with a governor.
With that as prologue, let us now look at the Soden08 Figure 1.
Soden08 Figure 1. ORIGINAL CAPTION. Comparison of the observed anomalies in absorbed SW (top) and emitted LW (bottom) radiative fluxes at the top of the atmosphere from Earth Radiation Budget Satellite observations (black) and three ensembles of GCM simulations (red). The observed anomalies are expressed relative to a 1984 to 1990 base climatology, and the linear trend is removed (30). The results are expressed relative to the pre-eruption (January to May 1991) value of the anomaly and smoothed with a 7-month running mean (thick line). The GCM anomalies are computed as the difference between the control and Mount Pinatubo simulations for each ensemble member. Both the model and observed global averages are from 60N-60S due to the restriction of observed data to these latitudes.
This looks good at first blush, and the authors say that the GCM results (red) are “consistent” with the observations. However, closer examination reveals issues. What struck me immediately about their results is that the actual observations of both the shortwave and longwave anomalies show clear signs of overshoot. After being knocked down by the volcano, after 1994 they both come back higher than pre-eruption. This worked to quickly restore the pre-disturbance state.
None of the GCMs show this sign of overshoot. Instead, the GCMs gradually drift back to the pre-eruption anomaly value of zero. This is similar to the negative feedback balance shown in Fig. 2. Unlike the overshoot in the observations, the GCM results flatline after 1994. While this doesn’t prove anything, it is another piece of evidence that the GCMs are missing some basic climate mechanisms.
How Much Total Difference did Pinatubo Make?
There is a second problem with the Soden08 model results, one which is less theoretical and more mathematically demonstrable. This has to do with the cumulative energy deficit from the volcanic eruption.
As you can see in the Soden08 Fig. 1 above, after the eruption the amount of incoming solar energy (SW, or shortwave radiation) dropped about twice as much as outgoing energy (LW, or longwave radiation). As a result, after the volcano there was a global net energy deficit. There was less energy entering the system than there was leaving the system. This deficit continued for some months.
The total magnitude of this deficit is an important indicator of the overall impact of the Pinatubo eruption on the climate system. It is a basic measurement of the phenomenon, answering the fundamental first question everyone asks — how big is it? We can investigate the total magnitude of the volcanic disturbance by looking at the cumulative energy deficit created by the eruption.
To do that, I first digitized the data in the Soden08 Figure 1. The data is available here as an Excel worksheet. Results in the worksheet for the GCMs are the average of the three runs shown in their Figure 1.
I then calculated the net energy balance (solar energy absorbed minus longwave energy emitted to space) for each month. I then started a cumulative total at zero, and added each month’s net energy balance to the cumulative total. This cumulative total shows how far out of balance the system was each month.
The resulting curve shows the size of the total disturbance caused by the volcano, as well as showing the path of recovery. The units are Watt-months/metre^2 (for convenience, since the data is monthly). For example, a two-Watt/m2 deficit that continued for three months would give a cumulative deficit of six Watt-months/m2. Fig. 3 illustrates the problem with the GCM results.
Figure 3. Cumulative effect of the Pinatubo eruption. Red line shows data from the average of the models (GCMs) shown in Soden08 Figure 1 above. Blue line shows data from the ERBS observations shown in Soden08 Figure 1 above.
So what does this result mean? Well, among other things it means that in this case the general “first glance” similarity of observations and models is misleading. A closer examination shows that the models did a very poor job at being consistent with the observations.
• The models greatly underestimated the magnitude of the peak impact of the eruption.
• They showed recovery starting much sooner than the observations show.
• They greatly underestimated the speed of the recovery once it started.
• They greatly underestimated the total impact of the eruption.
To put some numbers on those statements:
• The peak energy deficit in the observations was -42 Watt-months/m^2. This is more than twice the -18 Watt-months/m2 in the model results.
• The models show recovery starting about a year and a half after the eruption. The observations show about two and a half years before things turn around.
• The observed speed of the recovery is more than five times that of the modelled speed of recovery (post-1994 linear trends).
• The total impact of the eruption on the global energy balance is given by the area underneath the curves. Alternatively, it can be expressed as the average value of the energy deficit over the time period of the curves (1991-1995). The observed average energy deficit over that period is -21 Watt-months/m2. Again, this is more than twice the models’ estimate of -10 Watt-months/m2.
My conclusion? These models are not doing a credible job of representing Pinatubo’s effect on the global energy balance. The total size of the disturbance is a fundamental, basic measure of the accuracy of a simulation. Both the observed peak energy deficit and the observed total impact of the volcano were more than double the model results.
An error where the raw observed size of the phenomenon is more than double the model estimate? That sounds like a government project. Bad model, no cookies. Clearly, there is some fundamental problem with their simulation — they are showing the eruption of Mt. Minitubo, the half-size model.
When models show that kind of error in the raw size, peak size, and timing of the effects of a volcanic eruption … are the models “consistent” with the observations? Would you pay good money for a model that gave that size of error?
In addition, model results do not show the observed “overshoot” that leads to a speedy recovery from a disturbance. The results of this observed overshoot are seen in the difference between modelled and observed recovery rates. Driven by overshoot, the observed recovery rate is five times that shown by the models.
In short, I see no support for their implied claim that the models are an accurate representation of reality. Far from that, I don’t even see support for their vague claim of “consistency”. Their idea of consistency reminds me of the line from the old song, “She could easily pass for forty-threeeeee … in the dusk … with the light behind her.”
Their model results are inconsistent with observations. I do not see the Soden08 study as support for the idea that GCMs can successfully model the changes from volcanic eruptions.
Regards to all,
w.
PS – A final note for clarification. As the title suggests, the main thrust of Soden08 is concerned with whether the models perform better if they include a water vapor positive feedback. It finds that the GCMs do in fact perform better if there is positive feedback from a warming.
My analysis of the model results and observational data above is completely separate from the Soden08 analysis. We are simply using the same results and data. I make no claim that their analysis is right or wrong. In fact, I strongly suspect they are right, that the models do perform better if there is positive feedback from warming, although I have my own ideas what that demonstrates. But that is distinct from my analysis above.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
kse says:
November 30, 2010 at 5:56 pm
I love the idea that because you did majors in systems processing and microsystems, that therefore something you haven’t heard of must be bogus …
The “governor” is a fundamental part of many heat engines. It has a noteworthy history as well, because the first one (a “flyball” governor) was invented by James Watt of steam engine fame.
But hey, kse hasn’t heard of it, so it must be trivial or meaningless …
Frank says:
December 1, 2010 at 12:49 pm
Although my cumulative energy deficit may have problems, the setting of the zero point is not one of them. According to the paper:
However, even if we use a very different anomaly zero value, the results are the same. The models still only give values that are about half of what the observations show. Try it by setting the anomaly zero value of each dataset (GCMs and observations) to the average of that dataset. You’ll see that it makes little difference.
George — You mistake my (and others’) point about air resistance as a feedback, which is that its effect on the dynamics of the system is every bit as real as if it were an engineered feedback effort. In the cruise control system, it is no less real because the engineered feedback system does not sense it directly.
I think your confusion stems from not understanding where in the block diagram the air resistance force gets fed back to. It is not to the summing node with the commanded velocity setpoint. Instead, it is to the summing node of forces on the car, of which the output of the engine governed by the cruise control system is one of the force inputs.
If I had a way of easily posting a block diagram, my point would be obvious. I will try to explain the block diagram in words, using the traditional orientation of these diagrams:
Into the top left is a signal representing the commanded velocity (the setpoint). It goes into a “summing node” where it is compared to a signal representing the measured velocity, coming from below and the right. The output of the node going to the right is the velocity error. This enters the transfer function for the “control law”, most likely with a proportional and integral term. The output of this block, continuing to the right, is the resulting motive force on the car. (If you want to get more detailed this would go through another block representing the dynamics of the engine and drive train that produce the force.)
This “control force” is an input to another summing node, this one combining all of the forward and reverse forces on the car, including gravity (if there is an incline) and air resistance. The output of this summing node, again going to the right, is the net forward/reverse force on the car. This enters the transfer function for the car’s physical “plant”, which is basically a 1/[M*s] integrator from force (through acceleration) to actual velocity, which is output to the right.
This actual velocity value “goes” two places in this simple model. First, it goes through a transfer function block that produces the retarding air resistance force as a function of the velocity. This force is one of the inputs into the force summing node mentioned above. Second, it goes through a sensor block to the velocity summing node first mentioned, where it is subtracted from the setpoint velocity value.
The fact that the first feedback is inherent in the physics of the plant, and the second feedback is an engineered effect does not matter — both are feedbacks.
Well Curt, my intent was not to make you jump through hoops; and actually your word description of the system gave me a crystal clear picture of the block diagram. And I’m impressed that people do design such elaborate control loops, and I can see how many of your factors would become applicable to controlling a plane flight.
Where I differ in use of the term “feedback”, is that to me feedback comes directly from the controlled out put variable that the loop is supposed to be controlling, by adjusting the input signal (or set point) using whatever processing algorithm one may choose; such as three mode controllers would do.
I agree completely that other parameters or variables may exist that affect the desired output, such as wind resistance, friction, gravity, power supply noise or changes; water in the gasoline; etc. And it is clear to me that any of those things which can change the desired output, could if they can be “sensed” be used to create what I believe are properly “feedforward” terms; well I would call them “guesser circuits”, in that they anticipate things before they happen.
A change in headwind for example, is going to place an increased drag on an aeroplane, and absent a control mechanism, that would eventually slow the plane through F= ma. Now the velocity feedback will correct for this but only after the change has already started to happen. Directly sensing the perturbing force, such as an increased wind resistance allows one to deduce that the plane will be slowing; and the controller can figure out that relationship, to apply a corrective (engine thrust) to immediately oppose the grag increase, and actually stop the speed decline from even starting. Those sort of pre-emptive strikes are in my view feed forward examples; not feedbacks; they react to something that is expected to change the controlled variable (output) eventually.
And note that I said if you can “Sense” the perturbing influence such as wind resistance. That is not the same as calculating some presumed change in wind resistance that would accompany a change in speed, and using that as an input control signal, because that can’t pre-empt the ouput velocity change.
One of the best ways in chemical processing control loops to make a bomb, is to sense something other than the specific variable you want to control, and then generate a control feedback command that is based on some theoretical presumption of a specific relationship between the sensed variable, and the desired controlled variable. If any other perturbing influences can change that assumed relationship, then you don’t really have control of the process at all.
Having designed and built reactors, and process controllers for growing crystals and epitaxial layers in materials like Gallium Arsenide, and Gallium Arsenide Phosphide, involving quantities of Arsenic sufficient to poison all life in the universe; with lots of Hydogen carrier gas flowing around; I’ve had plenty of opportunity so contemplate what could happen if a loop got out of control.
Through about 12 years of operating such environments; we never once had a single employee ever test positive on a monthly Arsenic test. That is not the same as saying we never had any Hydrogen escape incidents. Well hydrogen being super light quickly floats up, and can form pockets in ceiling cavities, which can make for some interesting sparkles; but those are a lot more spectacle, than danger.
But honestly; if in your field it is the practice to consider these output perturbing variables as feedbacks and refer to them as such; then I certainly wouldn’t want to create confusions.
As it relates to the climate situation, thermal re-radiation from a warmed atmosphere back to the surface; which can release more CO2, AND more H2O into the atmosphere; i s absolutely independent of how that atmospheric warming occurred; so whether it resulted from H2O GHG effect, or CO2 GHG effect, of from H2O solar energy capture; the re-emision from the atmosphere is totally oblivious to what caused the atmospheric heating; and it is disingenuous for climatists to say that one (CO2) is a GHG “forcing”, but the other (H2O) is merely a “feedback amplification”. That is just plain silly.
George — I think we’re getting a little closer. I’ll flog this almost-dead horse one more time. It is common terminology in many fields to use “feedback” to refer to effects that feed back within the physical system, and not just to the engineered controller. Of course, there is no engineered controller (yet!) for the climate system.
As far as sensors are concerned, a very large fraction of modern control design practice concerns the decision as to what states of the system should be measured, and which should be estimated. In any real-world system, it would be cost-prohibitive to sense every state and possible input. I doubt that there is any cruise-control system on the market that directly senses windspeed or inclination. Could you get better control if you did? Sure. But would it be worth the extra cost? Very doubtful. People are willing to tolerate a temporary speed perturbation when the wind or inclination changes. But they do expect the cruise-control system to reasonably quickly return the car to the setpoint speed. (And to do this, the controller must temporarily “overshoot” its response, a key point of Willis’ post.)
The control system I described to you in my last post was a “toy” system, the simplest possible system I could think of that could make my point. Many real-world control systems are several orders of magnitude more complex. In modern “state-space” control practice, one of your first steps is to create a matrix of relationships between each degree of freedom (“state”) of the physical system and every other one. Most of these are “feedbacks” inherent in the physics of the system. Then you figure out which of these states you can/want to sense, and which you will estimate. Next you write your algorithms for estimating the unmeasured states (the “observer”), based on your model of the system. Finally, you design your controller to act on the measured and estimated states as compared with any desired values for the states (the setpoints), again based on your model of the dynamics of the system. As I said before, your life often depends on the success of these systems.
In this formal system analysis, you must define which effects on a state are internal to the system as you have defined it (including feedbacks), and which are external to the system (called “disturbances”). In my cruise-control model, air resistance due to ground speed is a feedback to the force node. Air resistance due to wind is an external disturbance to this node, as are gravitational effects due to inclination. This is completely standard terminology and practice.
With this in mind, I do not have trouble with climate scientists calling CO2 radiative effects (primarily) a forcing and H2O radiative effects (primarily) a feedback. I don’t see any evidence that any internal effect could cause the CO2 concentration changes we have seen in the last century. Yes, there is a feedback effect from temperature that increases CO2 as a function of warming, but that is too slow and weak to explain what we have seen. Conversely, I don’t see any way human activity (external to the climate “system” as usually defined) could directly change the global water vapor concentration to a significant degree, especially given the atmosphere’s capability to quickly “rain out” excess vapor.
By the way, for 3 years in the 1980s, my office was on the floor directly above a III/V GaAs etc. fab. I developed a healthy respect for the issues there. I also noticed that no one was lobbying for on-site daycare…
Sorry about the blowing the gasket and posting a rather stupid comment. And yes – I have checked the dictionary for the meaning of “governor” and now I believe that I understand what it means. It just does not happen to be a common definition in signal processing (or I just have forgot it). But try to bear with me – after all, English is something like 3rd language for me.
Anyhow, I still what to claim that Willis’ concept about differencies between “governor” and “feedback” systems are bogus – mainly because (as I now understand) governor is just one type of feedback system. Furthermore, I do not see any need for taking back my words about the true nature of the likely reason for Willis’ misconception in this topic – and besides that – well, all your lengthy speculations about cruise-control, boilers and so on are not exactly very helpful in making this issue more understandable.
So, lets’ get back to basics – first of all there are two types of feedback systems: positive and negative ones. The former tends to multiply the effect of the input signal and in this case we are quite clearly not interested in that case. In the latter case, the feedback system dampens the effect of the input signal and tries to direct the system response towards a certain system dependent “normal state”.
If we forgot the badly designed feedback systems (poles of the transfer function are in veeery bad places) that can have chaotic behavior, we have basically two possible basic behaviors in our negative feedback system: one system that tends to overdampen the input (that is there won’t be any overshot) and another a system that allows the system response to oscillate around the “normal state” (there will be some overshooting).
If we now look at the “Soden08 figure 1” in the Willis’ blog post, we can quite easily make a claim that the GCM response is overdampened while the observed system behavior is not, that is, there is some overshoot.
I assume that the latter alternative is the one that Willis and the most of you commenters understand as a “governor” system, right? But that is just one kind of a negative feedback system and you really cannot claim that “feedback” would be bogus because the system is clearly a “governor”. Such claim is just an oxymoron.
If that did not make my point clear, please, take “Signal processing 101” or “Control theory 101” – or ask anyone with signal processing masters.