There’s a long, long way still to go…

Ben Flurie

I kinda figured you knew the difference (about forecast error), but just couldn’t be sure from my uncertainty about your question. (I.e., my bad, not yours.) And it seems to me that we’ve been computing CI’s based on the error of the slope rather than the error of a forecast, which is what we ought to be doing. And then further, if one could figure out how to do a CI for the IPCC “forecast” and if it was done properly, including bootstrapping, that by the time you get out 100 years the CI would be so huge as to be pretty useless. Which would actually be an intellectually honest acknowledgment that any “forecast” out 100 years must be taken with a grain of salt, and anyone who insists it be taken seriously is full of it.

Basil

Or are you asking me how I would generate error bars for the IPCC’s actual forecast?

I’m asking this one.

My interest is comparing information and projections the IPCC actually makes. To the extent they are vague, it is difficult to do. It also means that later one, the working group for the next report has to decide, to some extent, what precisely the projections really was.

That means, one can pick interpretations to make the data comparison seem better or worse– depending on how one is inclined to make the projections look.

I understand how you get your forecasts projected from time series and the difference between the error in the slope and error in the forecast.

You asked “There is a difficulty though: How do you figure out the green error bars for weather noise in the IPCC’s forecast?”

Are you asking how I generated the kind of green error bars in the graph I did?

Or are you asking me how I would generate error bars for the IPCC’s actual forecast?

Let’s take the second possibility first. The short answer is I don’t know how. Actually, if I understand correctly, the IPCC’s “forecast” is more in the nature of a model run using GCM’s, and not strictly a forecast based on historical temperature data and the noise in that data. I imagine that the models have some notion of the uncertainty in them, but I don’t think that’s really what you are asking. I think you want to compare the model projections to what can be independently analyzed using actual data. The only way I know how to do that is to fit a model of some kind to the historical time series data and generate projections with error bars, and then see if the IPCC “forecast” is consistent with that.

Which brings me back to the first form of the question: Are you asking how I generated the kind of green error bars in the graph I did? I use the econometric package “gretl” which has built in routines for forecasting and developing confidence intervals (error bars) for the forecasts. A general introduction can be found here:

http://www.duke.edu/~rnau/three.htm

I call your attention to the following:

“Most forecasting software is capable of performing this kind of extrapolation automatically and also calculating confidence intervals for the forecasts. (The 95% confidence interval is roughly equal to the forecast plus-or-minus two times the estimated standard deviation of the forecast error at each period.)”

The sd of a forecast error is not the same as the sd (or standard error) of slope coefficient. Take a look at the final formula shown in this wikipedia article:

http://en.wikipedia.org/wiki/Prediction_interval

Or, if the link is valid, the image here:

http://upload.wikimedia.org/math/0/e/6/0e612bd396c703d236d84d69baa78a3e.png

See the “1″ in the equation? This substantially widens the confidence interval (what wikipedia calls a prediction inverval). This is because we are predicting a single event, not the mean of repeated events. This means that confidence intervals for forecasts are going to be much wider than what you would imagine from a confidence interval for the slope of a trend line (which is a mean estimator, not an estimate of a single event).

All of which is to say that you’ll have an order of magnitude greater difficulty rejecting the 2C/century forecast than you may realize.

Basil

http://sealevel.colorado.edu/current/sl_ib_ns_global.jpg

“But if you’re dealing with the science, then at some point you have to take good faith that the scientists are doing impartial work. There are always exceptions, but in general, science is an impartial, impersonal process.”

Wrong. This is science: show me your data, your method, your code, etc., and I’ll try to reproduce your results. There is absolutely no “faith” involved in the process. What you’re describing is a climate science religion, which won’t pass muster around here.

Furthermore, you’re working with a decade which most people agree exhibits an anomalous temperature signal, possibly even a short-term cooling one.

I sort of missed that at first. You think most people agree with this? Hmmm….

So, in reality you are agreeing with my conclusion: The current trend is not consistent with 2C/century warming. Your criticism appeara to be that I am wasting my time showing that the inconsistency with 2C/century is born out by a statistical test because everyone already knows the data are inconsistent with 2C/century.

Is that what you are saying? Because if you are, I suggest you go visit Eli, Tamino, RC, Annan, Stoat etc. and tell them that everyone concedes the current period is inconsistent with predicted 2C/century warming. :)

You are correct– the OLS is because several bloggers are insisting that OLS is the “only way”. Of course, this idea in not universally believed. . . However, the fact is, we get rejections if we do it their way or if we use Cochrane Orcutt, which is specifically valid for data that has AR(1) residuals. Other methods would be better, but I don’t know them.

I do have a couple alternate ways to look at this stuff– one will be shown sometimes next week. You’ll see it looks a little like this

“Now what lucia should do is at the beginning of the forecast period, plot a linear trend with a slope of 2C/century and see if it is outside the green error bars of the forecast. I can tell you that it isn’t.”

There is a difficulty though: How do you figure out the green error bars for weather noise in the IPCC’s forecast?

The IPCC doesn’t describe their “weather noise”. So, they don’t make a forecast in that sense. They do show spagetti diagrams, but it’s not clear they as a body consider those to describe “weather”. They don’t give any statistics for what would be “weather noise”, and only project the central tendencies for the slope. It’s not at all clear the WG1 has any agreement on how the scatter in the runs related to “weather noise”.

They leave the definition of “mean temperature in 2000″ ambiguous. In my graph, I assume they will define it the way they defined “mean temperature in 1990″.

So, I’m not sure how it’s possible to do what you suggest, while still testing what the IPCC AR4 really, honestly told people in any unambiguous way.

Counters–
There is a reason for testing the subset of data. The IPCC has made a specific projection. Their projection is for GMST to display a mean trend of 2C/century during the first 3 decades of this century. Weather noise, whose characteristics are not specified, result in excursions from mean behavior.

So, the idea to test that projections. As far as I am aware, that specific hypothesis can’t be tested using data from 1900-2000. I haven’t failed to say this is the thing I’m testing: I’ve said it over and over!

If you have other questions one might ask or test using statistics, I think that’s great. It might be useful for you to illustrate what you mean, and post it somewhere. Blogs are free. In my opinion, the more interesting different analyses done to answer a range of question people might have, the better. I’d encourage you to go ahead and see if you can discover the answer to the question you are telling you is more interesting than the one I’m tesing, and communicate it to us.

Gary–
Over the long haul, the GISS data isn’t really an outlier. Also, even if you compare now, you’ll see in the table of individual trends, that while it’s got the highest trend, it’s not ridiculously high. There are five measurement groups. One is highest. One is lowest. There is scatter. GISS is right in there.

Mike
No– I’m not saying the IPCC got the climb wrong. I’m not saying they got it right. But, for all we know, Roger doesn’t have access to any and all relevant data. Mostly, those are examples of the type of thing they might, hypothetically have right or wrong. I’m trying to capture a qualitative issue: There could be things wrong that have nothing to do with the nuts and bolts of calculating general circulations.

I suspect we may not even know some of it after say, 2003. How quickly do they know aerosols? There are other questions: How much does soot on snow matter? Is the amount of soot knows? Is that coded in the models?

There could be “unknown-unknowns”. But in that case, I can’t add them to my list. So, instead, I list the types of known errors that could hypothetically happen.

Basil

You state: “For example:
a) The IPCC could over predict the amount of GHGs in their SRES. That is, they could think CO2 or methane would rise faster than it really did.”

I know you are familiar with Roger Pielke, Jr.’s work earlier this year on “falsification” and he presented a graph showing the IPCC pretty much got the climb in GHGs right. Are you saying you disagree with Roger/IPCC on the rate of increase in GHG’s?

If you (more or less) agree that they got the GHG’s right, could you clarify what you are trying to say?

Mike

Friend, you are seriously deluded; this mythology went out with Glenn Seaborg and his tweed jacket. Are you an RC spinmeister by trade or avocation?

Statistics for the win!

I seem to have missed the argument for GISS data. Is it simply that it is consistent with the others under the suite of tests chosen?

Red noise: I take this to be with respect to OLS where the data are not by some measure random. Could you expatiate a bit on the analogy and its importance.

Here’s the problem: We lay skeptics don’t hear much about anything but doom & gloom from the climate crowd. This is utterly frustrating to us, the received wisdom comes across as sparse, overstated & manipulative because it does not explain the breadth or depth of the scenarios. I find this all very frustrating, a little science presentation of negative vs. positive forcings would go a long way into so many misapprehensions. These folk aren’t climate “denialists,” most of them understand that CO2 is important to the Earth’s climate, but the are certainly rejectionist toward claims of serious warming trend. There are those who say CO2 has zero effect, but that’s not what Bob Carter, Roger Pielke and other climate moderates are saying. They’re not rejectionists or denialists, not in the least, but they doubt the worst-case scenarios.

Recent entrants into the skeptic side of the discussion come from unlikely sources:

+ V. Ramanathan & Charlie Zender, on the broader window of opportunity that lays in the offing from soot mitigation (Ramanathan & Zender are not skeptics by any measure). From the dearth of public advocacy regarding soot mitigation, it appears that soot is the carbon that must not be named. So the question of an imminent “tipping point” seems overstated. And the role of soot in the decimation of Greenland’s peripheral ice sheets is not being mentioned even though the pictures of black icebergs pretty much say it all.

+ Kevin Ternberth of NCAR, on the Argo data showing a great deal of expected heat is missing & may have radiated back out into space. Many suffer misapprehensions on the latent ocean heat content b/c all we hear about are air temperatures. We have some confusing trends here WRT sea level, boreal ice sheet loss (soot?) and ocean temperatures. But if the seas can offload heat in ways heretofore unanticipated it might provide some bigger insights into the thermodynamic capacities of Earth systems.

+ Other researchers, finding more modest water vapor levels in the middle troposphere and over Antarctica. As for the latter I must wonder if this doesn’t reflect on the interglacial record as well, that the ice ages may have in fact been much drier and the interglacials being driven by unlocked water vapor, well past the effects of CO2. If Aqua data aren’t show water vapor piling up in the atmosphere then what’s going on? ***

+ Aerosols: The big mystery bus. If it’s really conceivable that worldwide aerosols could mask as much GHG warming then everyone would be delighted to know what offset we’re looking at. I’ve read that global aerosols might be masking GHG effects by some margin, around -1.1 degr. C, but seasonal soot might be countering that masking effect within regional brown clouds with a net warming of its own of +0.3 degrC (0.9 w/m-2). The def’n of “regional” gets weird here, since an area encompassing the vast expanses of the E. Indian Ocean, E. Asia, the boreal region & the vast Pacific are truly global in scope with soot adding 40% warming to the air.

+ And finally, the sun. So far average TSI has slackened to the order of -0.1 degr/C since the early 1990’s, and is apparently due to dim even more, to the order of another -0.1 to -0.2 degrC. Understand that Drew Shindell modeled a -0.3 to -0.4 degrC effect in Maunder Minimum/LIA due to the decrease of sunspot facular UV warming the upper troposphere (stratosphere), so his model and our current solar trend are consistent in decreased TSI.

So:

If the boreal ice loss is 90 percent due to soot deposition, why can’t this be reversed? I would think a full restoration effort would stem sea level rise.

If the oceans are not heating up at a marked rate then where’s the dangerous pipeline of latent heat that’ll blindside us 50 years from now?

If the middle troposphere (around 300 to 400mb) isn’t as humid as had been modeled, then how will the middle and upper atmosphere enter into the surface warming problem? (I believe the UAH MSU data show a cooling trend in the middle troposphere, not the stratosphere).

If the dimming sun is going to net offset -0.3 degrC by 2020, won’t this only help?

These are the questions climate moderates are asking be considered.

***I’ve noted that a casual comparison of the paleo record appears to show water vapor (light oxygen availability) tracking more consistently with temperature than CO2. Would you like to comment on this? Here’s a Q&D graphic: http://i30.tinypic.com/izon5h.jpg

I agree with the general sense of what I think you are saying. I chimed in on the 11 year hiatus discussion, not because I thought 11 years was the right period to be using, but because it was a good example to illustrate the use of loess regression.

If I were trying to do what I understand lucia to be doing, which is to test the falsifiability of a hypothesis made in 2001, I’d look into using data up to 2001, and then forecast from that point, determine the confidence interval of the forecast, and see whether a 2C/per century projection is falsified.

For example, here is a linear fit through 30 years of GISS (using robust errors) data through 2000, and then projected to the present (in this case, actually 2008:04):

http://s3.tinypic.com/vyp8c5.jpg

Now what lucia should do is at the beginning of the forecast period, plot a linear trend with a slope of 2C/century and see if it is outside the green error bars of the forecast. I can tell you that it isn’t. By the end of the forecast period, the projected anomaly would be ~0.55, and we can see that the upper limit of the confidence interval is well above this at the end of the forecast period.

So, have we verified the IPCC 2C/century projection? No, we’ve simply failed to reject it. And I don’t know if lucia appreciates it or not, but when we start working with forecasts, as opposed to sample means, the confidence intervals are always considerably greater.

Note, as well, though, that the forecast confidence intervals allow for the possibility of substantial negative growth, and despite that, the recent negative downturn is below the bottom of the error band. Now it is too soon to read much into this, since 5% of the time we’ll have observations falling outside of the error bands.

This has got me wondering, now, what a similar forecast would look like using one of the other metrics. More on that later.

Basil