There’s a long, long way still to go…

Ben Flurie

I kinda figured you knew the difference (about forecast error), but just couldn’t be sure from my uncertainty about your question. (I.e., my bad, not yours.) And it seems to me that we’ve been computing CI’s based on the error of the slope rather than the error of a forecast, which is what we ought to be doing. And then further, if one could figure out how to do a CI for the IPCC “forecast” and if it was done properly, including bootstrapping, that by the time you get out 100 years the CI would be so huge as to be pretty useless. Which would actually be an intellectually honest acknowledgment that any “forecast” out 100 years must be taken with a grain of salt, and anyone who insists it be taken seriously is full of it.

Basil

Or are you asking me how I would generate error bars for the IPCC’s actual forecast?

I’m asking this one.

My interest is comparing information and projections the IPCC actually makes. To the extent they are vague, it is difficult to do. It also means that later one, the working group for the next report has to decide, to some extent, what precisely the projections really was.

That means, one can pick interpretations to make the data comparison seem better or worse– depending on how one is inclined to make the projections look.

I understand how you get your forecasts projected from time series and the difference between the error in the slope and error in the forecast.

You asked “There is a difficulty though: How do you figure out the green error bars for weather noise in the IPCC’s forecast?”

Are you asking how I generated the kind of green error bars in the graph I did?

Or are you asking me how I would generate error bars for the IPCC’s actual forecast?

Let’s take the second possibility first. The short answer is I don’t know how. Actually, if I understand correctly, the IPCC’s “forecast” is more in the nature of a model run using GCM’s, and not strictly a forecast based on historical temperature data and the noise in that data. I imagine that the models have some notion of the uncertainty in them, but I don’t think that’s really what you are asking. I think you want to compare the model projections to what can be independently analyzed using actual data. The only way I know how to do that is to fit a model of some kind to the historical time series data and generate projections with error bars, and then see if the IPCC “forecast” is consistent with that.

Which brings me back to the first form of the question: Are you asking how I generated the kind of green error bars in the graph I did? I use the econometric package “gretl” which has built in routines for forecasting and developing confidence intervals (error bars) for the forecasts. A general introduction can be found here:

http://www.duke.edu/~rnau/three.htm

I call your attention to the following:

“Most forecasting software is capable of performing this kind of extrapolation automatically and also calculating confidence intervals for the forecasts. (The 95% confidence interval is roughly equal to the forecast plus-or-minus two times the estimated standard deviation of the forecast error at each period.)”

The sd of a forecast error is not the same as the sd (or standard error) of slope coefficient. Take a look at the final formula shown in this wikipedia article:

http://en.wikipedia.org/wiki/Prediction_interval

Or, if the link is valid, the image here:

http://upload.wikimedia.org/math/0/e/6/0e612bd396c703d236d84d69baa78a3e.png

See the “1″ in the equation? This substantially widens the confidence interval (what wikipedia calls a prediction inverval). This is because we are predicting a single event, not the mean of repeated events. This means that confidence intervals for forecasts are going to be much wider than what you would imagine from a confidence interval for the slope of a trend line (which is a mean estimator, not an estimate of a single event).

All of which is to say that you’ll have an order of magnitude greater difficulty rejecting the 2C/century forecast than you may realize.

Basil

http://sealevel.colorado.edu/current/sl_ib_ns_global.jpg

“But if you’re dealing with the science, then at some point you have to take good faith that the scientists are doing impartial work. There are always exceptions, but in general, science is an impartial, impersonal process.”

Wrong. This is science: show me your data, your method, your code, etc., and I’ll try to reproduce your results. There is absolutely no “faith” involved in the process. What you’re describing is a climate science religion, which won’t pass muster around here.

Furthermore, you’re working with a decade which most people agree exhibits an anomalous temperature signal, possibly even a short-term cooling one.

I sort of missed that at first. You think most people agree with this? Hmmm….

So, in reality you are agreeing with my conclusion: The current trend is not consistent with 2C/century warming. Your criticism appeara to be that I am wasting my time showing that the inconsistency with 2C/century is born out by a statistical test because everyone already knows the data are inconsistent with 2C/century.

Is that what you are saying? Because if you are, I suggest you go visit Eli, Tamino, RC, Annan, Stoat etc. and tell them that everyone concedes the current period is inconsistent with predicted 2C/century warming. :)

You are correct– the OLS is because several bloggers are insisting that OLS is the “only way”. Of course, this idea in not universally believed. . . However, the fact is, we get rejections if we do it their way or if we use Cochrane Orcutt, which is specifically valid for data that has AR(1) residuals. Other methods would be better, but I don’t know them.

I do have a couple alternate ways to look at this stuff– one will be shown sometimes next week. You’ll see it looks a little like this

“Now what lucia should do is at the beginning of the forecast period, plot a linear trend with a slope of 2C/century and see if it is outside the green error bars of the forecast. I can tell you that it isn’t.”

There is a difficulty though: How do you figure out the green error bars for weather noise in the IPCC’s forecast?

The IPCC doesn’t describe their “weather noise”. So, they don’t make a forecast in that sense. They do show spagetti diagrams, but it’s not clear they as a body consider those to describe “weather”. They don’t give any statistics for what would be “weather noise”, and only project the central tendencies for the slope. It’s not at all clear the WG1 has any agreement on how the scatter in the runs related to “weather noise”.

They leave the definition of “mean temperature in 2000″ ambiguous. In my graph, I assume they will define it the way they defined “mean temperature in 1990″.

So, I’m not sure how it’s possible to do what you suggest, while still testing what the IPCC AR4 really, honestly told people in any unambiguous way.

Counters–
There is a reason for testing the subset of data. The IPCC has made a specific projection. Their projection is for GMST to display a mean trend of 2C/century during the first 3 decades of this century. Weather noise, whose characteristics are not specified, result in excursions from mean behavior.

So, the idea to test that projections. As far as I am aware, that specific hypothesis can’t be tested using data from 1900-2000. I haven’t failed to say this is the thing I’m testing: I’ve said it over and over!

If you have other questions one might ask or test using statistics, I think that’s great. It might be useful for you to illustrate what you mean, and post it somewhere. Blogs are free. In my opinion, the more interesting different analyses done to answer a range of question people might have, the better. I’d encourage you to go ahead and see if you can discover the answer to the question you are telling you is more interesting than the one I’m tesing, and communicate it to us.

Gary–
Over the long haul, the GISS data isn’t really an outlier. Also, even if you compare now, you’ll see in the table of individual trends, that while it’s got the highest trend, it’s not ridiculously high. There are five measurement groups. One is highest. One is lowest. There is scatter. GISS is right in there.

Mike
No– I’m not saying the IPCC got the climb wrong. I’m not saying they got it right. But, for all we know, Roger doesn’t have access to any and all relevant data. Mostly, those are examples of the type of thing they might, hypothetically have right or wrong. I’m trying to capture a qualitative issue: There could be things wrong that have nothing to do with the nuts and bolts of calculating general circulations.

I suspect we may not even know some of it after say, 2003. How quickly do they know aerosols? There are other questions: How much does soot on snow matter? Is the amount of soot knows? Is that coded in the models?

There could be “unknown-unknowns”. But in that case, I can’t add them to my list. So, instead, I list the types of known errors that could hypothetically happen.

Basil

You state: “For example:
a) The IPCC could over predict the amount of GHGs in their SRES. That is, they could think CO2 or methane would rise faster than it really did.”

I know you are familiar with Roger Pielke, Jr.’s work earlier this year on “falsification” and he presented a graph showing the IPCC pretty much got the climb in GHGs right. Are you saying you disagree with Roger/IPCC on the rate of increase in GHG’s?

If you (more or less) agree that they got the GHG’s right, could you clarify what you are trying to say?

Mike

Friend, you are seriously deluded; this mythology went out with Glenn Seaborg and his tweed jacket. Are you an RC spinmeister by trade or avocation?