To Tell the Truth: Will the Real Global Average Temperature Trend Please Rise?
Part II
A guest post by Basil Copeland
Before proceeding, I want to thank Anthony for allowing me to guest blog at Watt’s Up With That? Anthony is doing some remarkable work in trying to insure the integrity and quality of the surface record, and it is an honor to be able to use his blog for my modest contribution to the debate over climate change and global warming.
In Part I we looked at seasonal differences in the four global average temperature metrics Anthony has recently been blogging about, and demonstrated that since around the end of 2001 there has been no “net” global warming, that positive seasonal differences have been offset by negative seasonal differences. More recently, negative seasonal differences have dominated, suggesting the possibility of a recent negative trend in global average temperatures.
Reader comments to Part I were interesting. It was obvious from many that they were struggling to understand what I was getting at, and that this was a different perspective on the data than usual. Others quickly raised the specter of cherry picking the data, or suggesting a hidden agenda of some kind. That some would jump to such conclusions without giving me the courtesy of waiting until I was finished is a sad commentary on what’s happening to the field of climate science. Science is supposed to be all about the freedom to engage in critical inquiry without being impugned with false motives, the freedom to hold scientific consensus up to the critical scrutiny of falsifiable hypotheses. When voices immediately seek to shut off avenues of inquiry, or impugn motives for questioning scientific consensus, I don’t know what that is, but I know that it is not science..
Resuming where we left off with Part I, if there is evidence of a recent negative trend in global average temperature, is it “statistically significant,” and if so, in what sense? That’s the question I left hanging at the end of Part I, and is the question we will address in Part II. There are various ways we might go about investigating the matter. I chose one that comes from my particular field of experience and expertise (economics, though it is perhaps worth noting that my training was in environmental and resource economics): the Chow test. The Chow test is used to test for “structural breaks” in time series data. Just as correlation does not prove causation, a “structural break” doesn’t necessarily prove anything. It merely suggests that things were different in some way before the “break” than afterward. It doesn’t answer the question of “why” things changed. Or, given the venue, we might say that it doesn’t answer the question Watts Up With That? But it does answer the question of whether the change is “statistically significant.” And if it is, then perhaps inquiring minds might want to know about it, and consider whether it makes any difference to matter under discussion.
The Chow test involves fitting a regression to the sub parts, and comparing the sum of the mean square error (MSE) of the sub parts to the mean square error of a regression fitted to the entire time period. If the sub parts come from sufficiently different regimes or circumstances, splitting the time series into two parts will reduce the total MSE, compared to the MSE of a single regression fitted to the entire time period. The Chow test follows the F distribution, and is a test of the null hypothesis of no change, or difference.
Table 1 summarizes the Chow test for each of the four metrics under consideration, for a structural break at 2002:01. The Chow test was statistically significant in all four cases, though in varying degree. In Table 1 I describe the level of statistical significance using the same likelihood terminology used by IPCC. Evidence for a structural break is “very likely” from the UAH satellite dataset, “extremely likely” from the GISS and RSS datasets, and “virtually certain” from the HadCRUT land-sea dataset.
I cannot say that, though, with remarking about how silly it is. I do not know of any other field where statistical significance is interpreted this way. In my field, anything less than a 95% level of confidence is considered weak support of a tested hypothesis. Instead of “very likely,” for support at the 90% level of confidence I’d say “probably.” Instead of “extremely likely” at the 95% level of confidence, I’d say “likely.” And instead of “virtually certain” at the 99% level of significance, I’d say “very likely.” In other words, to my way of thinking, the IPCC likelihood terminology is shifted about two orders of magnitude in the direction of overstating the likelihood of something. But even with my more cautious approach to characterizing the results, the evidence is somewhere between “probably” and “very likely” that a structural break occurs in the data after 2002:01.
However we choose to put it, there is statistical support for modeling the trends with a break at 2002:01. This is done, statistically, with dummy slope and constant variables, and the results are shown graphically in Figures 1, 2, 3, and 4. In each figure, there are three “trends” noted. The first, to the left and above the data, is the trend for 1979-2001. The third, to the right and below the data, is the trend for 2002 through 2008:01. In the middle, labeled “dT” is a trend for the entire period derived from the delta, or difference, in the end points of the the trend lines, with a number in parentheses representing the decadal rate of change from fitting a single trend line to the data. This overall trend, based on the difference in end points of the trend lines, is a “best estimate” of the overall trend using all 29 years of data (thus refuting any notion of cherry picking).
Figure 1 – click for larger image
Figure 2 – click for larger image
Figure 3 – click for larger image
Figure 4 – click for larger image
Many readers will probably be familiar with the use of 30 years as a basis for a “climatological norm.” While we do not have 30 years of data here, we’re close, close enough to refer to the overall trends as a climatological normal for the past three decades. As I look at the results shown in the four figures, two things stand out.
First, the dT of the final “best estimate” is 0.025C/decade (UAH_MSU) to 0.047C/decade (HadCRUT) lower than what we’d expect from fitting a straight trend line through the data. That is perhaps the major point I’m trying to make in all this: that over the period for which we have satellite data to compare to land-sea data, the rise in global average temperature is not quite as great as one would think from fitting straight trend lines through the data.
Incidentally, this not entirely owing to fitting a downward trend through the data since 2001. Separate slope and constant dummy variables are also included for the 1998 El Nino, and this accounts for some of the difference. In fact, somewhat surprisingly, when a constant dummy is added for the 1998 El Nino, it reduces the slope (trend) for the non-El Nino part of the time series through 2001. We usually expect a constant dummy to affect the model constant term, not the slope. But in every case here it reduces the slope in a significant way as well, so some of the difference in the “dT” and the result we’d get from a straight trend line owes to the effect of controlling for the 1998 El Nino.
The second thing that stands out, of course, is the downturn since 2001. Whether this downturn will continue or not, only time will tell. But if it continues, then the “dT” will likely decline further.
Other things may stand out to other observers. The differences within the two types of metrics are notable. GISS implies more warming than HadCRUT, and RSS_MSU implies more warming than UAH_MSU, with the latter showing quite a bit less warming in the period up to 2001 (given the way we’ve modeled the data). In the case of GISS vs. HadCRUT, the trends are actually quite similar in the period up to 2001; it is after that that the difference emerges, making one wonder if something has changed in recent years in the way one or the other is taking its measure of the earth’s temperature.
Just a final comment, as a way of putting this all in some perspective. In AR4 IPCC projects warming of 0.2C per decade for the next two decades in a variety of its climate change scenarios. That will take a lot more warming than we’ve seen in recent decades. And with the leveling off of the trend in recent years, even if an upward trend resumes, at present it seems highly unlikely that we will see a rise of 0.4C over the next two decades. Of course, the future has a way of humbling all forecasts. But perhaps the apocalypse is not as near at hand as some fear.
I use CUSUM at times to try and answer the question “did something change, if so, when?”
Basil, have you had a look at menne’s change point analysis
for temps. ( err.. google Menne change point )
Basil,
You’ve lost me here. The purpose of least squares fitting, including regression, is to test whether your data could have been produced by some process that you have a scientific reason for believing may be at work, plus other effects that can be modelled as noise. A linear temperature rise, say, would qualify; even, perhaps, a rise with a continuous change in gradient. But the models you are now producing have no plausibility – particularly discontinuous temperature. You might as well fit the original data. If you want to test whether 2001 is a pivot point, regression with two continuous line segments would be more interesting.
Basil, you talked about adding a “constant dummy”, tammy may take a fence to that.
Sorry Rev, delete at your pleasure
I meant “discontinuous change in gradient”
GISS uses the poles where HadCRUT does not, so that could be the difference. Other than that, good posts Basil. It will definately be interesting to see what the next couple years brings.
Basil,
I’m not an expert on statistical analysis. When I look at the trendlines on the graphs, the discontinuities jump out at me. I expect the trendlines to meet at the endpoints.
Also when I look at the trendline from ~1979 to 2001, if it is extended it appears from my eyeball that it would be close to intersecting the 2001-2008 trendline in 2008 for HadCRUT and RSS. For GISS and UAH_MSU it appears the intersection would be in a later year.
Could you comment on the meaning of the discontinuities and the intersection of the trendlines if they are extended, if there is some meaning? Thanks.
The figures aren’t showing on my two browsers. Is there an html glitch?
REPLY: Not that I know of, all apears fine and comments from others seem to indicate they can view them. Could be a routing problem. Try reset of your DSL or cable modem box to get a new IP address, clear cache and try again.
TSI peaked in 2000. A 2 year lag would explain a break point in the 2002 range.
Prior to that TSI peaked in 1991 and 1980.
The graphs seem to show similar breaks around those times too but it is obscured by the volcanic eruptions in 1984 and 1991.
Bottom line: there is a physical basis for choosing 2002 as a structural break.
You remember when Tammy selected 1975 as a “good place” to start a linear regression? Well, basil is actually using
a METHOD to determine places in a time series where things change as opposed to eyeballing things.
Change point analysis. Not cherry picking. We might nit pick his method
but I did not see you get bothered by taminos ‘selection’ of 1975. ( even though
I suggested mennes change point analysis to him a few times)
here is and example of a similiar approach. Used by NOAA. before you attack this be aware. This method is used to adjust USHCNv2 data . and we all know who uses that. and the change point is 1964, not 1975 or 1977 as tammy thought.
http://ams.confex.com/ams/pdfpapers/100694.pdf
Interesting comments and questions so far. I don’t have a time to answer them all right now. I’ll come back when I do have more time, but I can answer a couple of Lee’s questions pretty quickly.
The criterion for choosing a break point at 2002 has been explained serveral times now. Cumulative seasonal differences turn negative, and stay negative, after that point. In other words, there are more negative seasonal differences than positive seasonal differences. That’s rationale enough to justify looking at 2002 as a break point. In truth, one could pick a break point at random, and investigate whether it constitutes a significant break point, and there would be nothing wrong with that other than being a likely waste of time and resources. In any event, the proof is in the pudding, as they say, and that’s in the significant Chow statistics.
As for autocorrelation, the regressions were all estimated with Cochrane-Orcutt, so that’s not an issue. In fact, had I used OLS, the autocorrelation would have biased the variances downward, overstating the Chow test.
As to allegedly introducing a 0.2C step change, and then testing for it, you don’t appear to understand the process. I just introduce the hypothesis of a step change, without quantifying it; the 0.2C is what comes out as an estimate of what the step change was. I.e., I didn’t make up the 0.2C, that’s what comes out of the regression analysis. And it is in fact different in each case, being the product of the regression, not an a priori input on my part.
More later.
Basil
“The criterion for choosing a break point at 2002 has been explained serveral times now. Cumulative seasonal differences turn negative, and stay negative, after that point. In other words,”
Basil, this is why I was nagging you in the last thread about your plots being just annually smoothed temperatures, rescaled. I showed it with algebra, numbers, I’d show the plot if I could. All this quoted statement means is that the annually smoothed temp was the same in 2002 and 2008, and was higher in between. That doesn’t make 2002 any kind of special event.
coaldust,
Good questions. Your eyes are sharp, and it is true that extrapolating the 1979-2001 trend ends up meeting the downward trend from 2001 at some point. As for the discontinuities, I don’t necessarily have an explanation for them, as I’m not a climate scientist. I cannot imagine an a priori reason, though, why a complex system like the climate might not exhibit discontinuities like that shown in the trend lines. I’m just reporting what I see in the data. Maybe others can suggest plausible explanations (like Raven does with his comment about TSI).
Steve Sadlov/Steven Mosher,
As I said, “There are various ways we might go about investigating the matter. I chose one that comes from my particular field of experience …” As for Menne’s changepoint analysis, I was unfamiliar with it, but having taken a quick look, it appears to be based on the same principle as the Chow test, but designed to track more complex patterns of change than the Chow test.
Looking back, I see I haven’t answered all your questions. The 98 El Nino is controlled for by simple dummy variables (1’s for months where it appears that the El Nino was affecting global temperature, and 1 x time for the same months to capture the change in slope). As for not controlling for the current La Nina, there is not yet enough data to model it meaningfully with a dummy variable. In time, there well may be. I wish you were less argumentative, because I’d very much like to be straightforward with you. But statements like “Your trace shows multiple breaks, introduced by you, pre-2002 What did you actually do?” imply that I’ve taken liberties with the data to make things appear that are not really there. What I did was not terribly sophisticated, and I cannot believe that you are really having that much trouble understanding what I did.
Basil,
Your description of confidence levels is interesting in light of a 10-year-old Iowa State University paper I stumbled across recently in which they say:
It would be interesting to take these one by one and see how many of them have been born out.
PS. sorry about the Flea comment. I kid.
Steve, IIRC, back in the 60’s and 70’s the loyal fans who followed professional golfer, Lee Trevino, around the course during tournaments were fondly called Lee’s fleas. I’m just sayin’ — historical perspective?
Nick Stokes,
I would love to see a plot of what you are describing. Is there some reason why that’s not possible? You have access to the same data I do — Anthony’s. Mind you, I’m not saying that what you are saying is not true. But even so, regardless of WHY the decision was made to test a break at 2002, the reality is that the trend IS negative after that point, and that the Chow tests provide ad hoc justification for treating the period as somehow different than the period before. Is your point that that is merely serendipity, and that any six year period chosen at random might well show a similar kind of discontinuity? If so, be my guest and demonstrate this to us.
The long term “climate” trend is almost certainly continuous, not discontinuous.
We see daily temperature changes 2 orders of magnitude larger than 0.2C.
We see much larger than 0.2C month to month changes (in the anomaly) here in Australia, that cannot possibly be caused by heat transport to/from somewhere else (SST don’t vary that much around Australia and there are no nearby landmasses except PNG).
That the Earth is always in thermal equilibrium (absent changes in forcings) is untested dogma.
The alternate theory is that step changes occur over short time periods due to say the PDO, etc, and this is interpreted as a trend due the noise in the data.
Would it not have been easier and at the same time save everyone a lot of work by just putting a $ sign on the y axis of these graphs and sending them to a stock analyst? Tell the analyst that you are thinking of investing in this particular stock (ticker symbol GW). Ask the analyst if the technical trends suggest that the stock will go higher or lower. Guys, Wall St. have been doing this type of analysis for decades. Why re-invent the wheel?
My brother, an astronomer who studies climate on Mars, sent me today an article from the March Issue of Physics Today. In it, the authors from Duke University and Research Triangle Park describe a new way of modeling the impact of the sun on the earth’s climate. If I recall correctly (I left the article at the office), it claims global cooling since 2002 and the sun has contributed up to 65% (roughly two-thirds) of the warming seen since 1900. I’m surprised that no one has mentioned the article yet.
I was very surprised to see that article show up in the same magazine that recently (over the last 18 months or so) gave Gavin Schmidt of NASA GISS and Kerry Emanuel of MIT space to educate the physics community on the successes of GCM’s in predicting global temperature variations, and Carnot cycle models of hurricanes.
The article is by Scafetta and West, and reviews some of their recent work looking for power-law fluctuations in solar TSI and global surface temperatures, implying that even weak sol-earth coupling has a significant effect on earth’s climate. Pretty interesting read. They highlight the dramatic differences in TSI satellite reconstructions by ACRIM and PMOD, and argue that as much as 69% of the current warming trend can be attributed to solar influences.
Is ozone actually being depleted? I thought the trend was flat since the “hole” was discovered in the mid-50s.
Raven,
I read Menne’s paper, at least this one. He does the analysis more completely, allowing the uncertainty in the break location to be part of the F-statistic. And he couldn’t even find a significant change point even in the 70’s, although there was one about 1964. That’s using decades of data.
The case against sudden temperature changes is simple physics – thermal inertia. The atmosphere is quite well mixed on a scale of weeks or months, so you need a substantial heat flux over a long period to change the temperature, even ignoring that heat would be flowing into the ocean as well. By my calc 2 W/m2 (a ball-park AGW figure) will take about three years to heat the atmosphere 1 deg C.
On the significance of 2002 – I was specifically arguing against Basil’s contention that the zero of the CSD was an event. But a max in TSI isn’t an event either; it doesn’t change any physics, any more than noon does.
Basil,
It’s not that I can’t do a plot of the running average, but I don’t know how, as a commenter, I could post it.
On 2002 and all that – yes, the trend from 2002 to 2008 is, well, zero really. But as far as concluding that something happened in 2002, that is post hoc reasoning. In 2006, you would have said the trend from 2002 is positive, and probably said the break was in 1999. In 2010 the trend from 2002 will be something else.
I’ve mentioned this paper by Menne. I think he’s doing the analysis more correctly, in that he allows the break point to be determined by the analysis. But the F-test then shows much less isgnificance – even with decades of data, he couldn’t find a significant breakpoint in the mid 1970’s
By an odd coincidence, just published is an interesting new paper that links non-gaussian variability of solar activity to non-gaussian variability of global temperature anomalies:
http://www.fel.duke.edu/~scafetta/pdf/opinion0308.pdf
The paper suggests that both time series show Levy statistics, and that the two time series are closely linked.
The randomness of the solar fluctuations shows a short time scale of roughly 7.5 years (linking to El Ninho / La Ninha??), on top of the 11 and 22 year known solar cycles.
Oddly enough, the paper suggests that global cooling started in 2002…..
Well, I’m pretty grateful there was a breakpoint six years ago. Had there not been, and had the ‘Post Hoc, Ergo Propter Hoc’ logical fallacy of temperature rise co-inciding with CO2 rise run successfully for a few more years, then we would have been tragically bound to the irrelevant solutions of carbon capping. As it is, the biofuel delusion has raised the price of food worldwide, utterly unnecessarily.
When the general public understands that they are kicking in an extra dollar a gallon every time they buy milk because of the ‘madness of crowds’ and the dishonesty of a few scientists, there will be Hell to pay.
‘Buy a gallon of ethanol, starve a dozen children’
==============================
fwiw. from 1880 to 2007 there were 48 7 year peroids with a negative trend.
22 of them stayed negative until 10 years
Nick Stokes,
On Menne’s paper, it is worth noting that he is analyzing annual data, and that the time frame is only through 2003. There’s nothing wrong with that per se — using annual data — but because of the smoothing performed by annualizing the data, the results are obviously going to be different than if monthly data are used. For instance, if you look at the charts in the link to Menne’s paper you posted, you can still see the effect of the 98 El Nino (if you know to look for it), but it doesn’t stand out in the same degree that it does when looking at monthly data over a shorter period of time. As for Menne not finding a break in the mid 1970’s, just eyeballing Menne’s data, I wouldn’t expect to find one either. That doesn’t mean one wasn’t there, were one to look at monthly data. In other words, applying the technique to monthly data would one expect to find more change points, with smaller orders of magnitude, than one is going to find in annualized data.
As for Menne’s technique being superior because it infers the change points from the data, rather than postulating them a priori, I would argue that from a philosophical point of view (the philosophy being the philosophy of science) that’s nonsense. There’s no disputing that Menne’s technique is more sophisticated (for want of a better way to say it) than the Chow test. But from the standpoint of science, it matters not how we come up with our hypotheses, it matters only that they be falsifiable. I could throw darts at the data, and say “let’s see if there was a change point” where the dart hit, and that would be perfectly valid.
If, after successive dart throws, and continually NOT rejecting the null hypothesis of no change, the dart landed on 2002, and I found that I COULD reject the null hypothesis of no change, would it matter how I found the change point? In truth, Menne’s approach isn’t much more than that — mining the data, looking for change points. I just mined the data differently. Having done so, the hypothesis is out there for anybody to disprove. If you want to run Menne’s methodology on the data sets we are looking at, go for it!
As for not knowing how to post a plot of the running average, just upload it to http://www.tinypic.com, and post the URL here, and we can see it that way.
Basil,
While I am sympathetic with your conclusion that appears to be a down turn, or at least a leveling off since 2001, there is something about your analysis technique that bothers me. You say:
It seem inevitable that the more subparts you choose to divide the data into, the smaller the total MSE will be. Take the absurd case of dividing n points into n-1 subparts, then the total MSE will be zero.
Best Regards,
Tom Moriarty
ClimateSanity
Hi, Athony. One of the temperature plots I watch has to do with the Indo-Pacific Warm Pool (“IPWP”). That’s the giant region of very warm tropical water which serves as Earth’s boiler room, supplying heat to the rest of the planet.
It is rather well-correlated with global temperature, as one might imagine. A map which shows the correlation is this one
http://davidsmith1.files.wordpress.com/2008/03/0313081.jpg
(Readers who aren’t familiar with these maps should simply note that the orange and red colors indicate that, as sea surface temperature changes there, so does the global air temperature.)
How has the IPWP behaved recently? I use a somewhat unusual plot to get a feel for the answer
http://davidsmith1.files.wordpress.com/2008/03/0314081.jpg
This plot shows the IPWP temperature anomaly and how it compares with the same month in prior years (1948 to the present). For example, the final month (February 2008) is about the 22’nd warmest (or 38’th coolest) of the last 60 years.
What it shows is that the IPWP has been undergoing a general cooling trend since about 2001. That’s important, because this is Earth’s boiler room.
(So, until you point to the article, I’m just going to assume that, as far as we know, Tamino pulled 1975 out of a hat.)
I would bet that you could get 1975 by plotting 2nd derivatives of temps looking for the part where the spikes are more up than down and then go +/- 5 years or so on either side of that, curve fit it, and wind up with 1975. This seems the easy way to do this since you go from cooling (40’s-70’s) to warming (70’s – present) somewhere in that 70’s data range anyway. All this does is narrow down where you can pick the transition change point.
While I’ll bet that the method used was probably more involved than that and used fancier math, I’m borrowing from Dr. Pielke’s example and doing this the simple way. But that’s my guess as to how that date was reached, and it’s not pulling it out of a hat.
Lucia — does this sound all that farfetched to you?
Regardless of how this was reached, my problem with this is that this method of hinges etc seems to be more of a low freq detector of PDO or some other cycle (avg 30 years or so) and this seems disingenuous at best to detect the cycle start point and claim warming.
Tom,
The Chow test only envisions subdividing into two parts. I would imagine that were it extended to the kind of case you are considering, that the degrees of freedom would rapidly decline with each successive partitioning of the data, and that in your reductio ad absurdum case the degrees of freedom would be zero, which is just another way of saying that the confidence limits would plus or minus infinity! IAC, the Chow test, limited as it is, is well established as a valid hypothesis test.
Basil
steven mosher says:
“fwiw. from 1880 to 2007 there were 48 7 year peroids with a negative trend.
22 of them stayed negative until 10 years”
And we did not have 2 degC/century warming either.
The issue is not: ‘can we have decadal declines in temps as part of a long term rising trend?’ (answer: of course).
The issue is: ‘how much weight should be placed on IPCC projections of >2 degC/century warming given a 10 year decline while CO2 levels continued to rise?’ (answer: not much).
That said, we are still dealing with probabilities and there is a 1 in 20 chance that the 2 degC/trend is real even it is outside the 95% confidence limits given the data since 2002.
Why does everyone use linear methods to analyze the climate system? Its natural component is almost surely cyclical, which would make a Fourier-like analysis more relevant (this is not a knock against this blog post per-se — it’s done everywhere in climate studies that I can see). What I mean is, if you take, e.g. from 0 to pi/2 (1/4 period) from a sin wave, it has a strong (and highly significant) linear trend but no one would argue that you’ve found something meaningful simply because your linear fit was at the 99.999% confidence level.
I’m not savvy enough in statistics to know what this leads to, but I do know that in general you can’t simply take data that is fundamentally non-linear and fit a line to it. All the statistical tools that measure significance, etc. are only valid on fundamentally linear data. Part of the problem is that there aren’t good techniques (or at least not any that they teach in undergrad stats) that test significance of e.g. a sinusoidal fit. This is probably why no one does it, but still, I can’t help but wonder whether this sort of thing doesn’t make a HUGE difference when, e.g. the IPCC says they are 95% confident that the recent trend is not natural.
A gentleman would have quietly made the effort to get the link, even asking Tammy directly for it.
Instead we’ve got someone who has time for post after post…….almost all with a huge chip on the shoulder.
compare Tammy’s method ( unpublished, not peer reviewed, I checked his publication history and he has nothing published on this ) with any other method for change point analysis. I think you’ll find tammy wanting.
Essentially he fit two lines to the series to mimimize the regression error.
Why two? Put another way, choosing to find ONE hinge point in the temp series
will guide you to 1970s. A more robust method would find all the “hinge” points
All the change points. So in a way the undocumented TAMMY method just
data mines for the exact year in the 70’s that minimizes the error of two lines
of regression.
I searched in vain to find the tamino method of change point analysis
in consensus time series statistical science. I find he has no published papers
here. The Chow test is akin to the tammy test excect it is more general and
not restricted as his was to finding one break point. It and other methods
find multiple break points. Thats the point! Anyway.
Be that as it may. Perhaps somebody should publish the tammy test as a new method of change point analysis in a statistics journal.
Basil’s Selection of 2001 is fair enough. The real issue is what to make of a
seven year period of negative trend?
Not Much. just deal with the fact that you have a 7 year
negative trend. WEATHER HAPPENS. Like I said, if you randomly selected
a start year since 1880 you would have a 38% chance of picking a start year
that had a seven year negative slope. So, it’s not at all rare. neither is it likely
to last.
Now, given that you pick a 7 year period with a negative slope you have
a roughly 50% chance it will last 10 years, and a 30% chance it will last 14
years and almost no chance it will last 30 years.SO BUCK UP, the cold will
end. In fact I think the coldists are overplaying this, and just tweaking
the noses of the warmists who also overplay short term fluckuations
Off Topic, but please enjoy:
The winner for Most Ironic Comment is…
Steve Mosher is most certainly correct that we cannot “make [here I would add ‘too’] much” out of the downturn since 2001. So let’s be clear about what I made of it. All I made of it is that it is part of the 29 year period of history since satellite measurements began, and that if ignored, as in the case of fitting a straight line regression through the entire period to measure the trend, it will lead us to somewhat overstate the “best estimate” of what the “average” trend was during the entire period.
Let’s take RSS_MSU as an example to clear this up. Nowhere do I say anything about the likelihood of the -0.336 since 2001 continuing indefinitely. Rather, what I’m saying is if we do not properly model the in sample data, and simply fit a straight line through the data, we end up concluding that the average trend over 29 years was 0.169C/decade, when in truth a better estimate of the average trend over the past 29 years was on the order of 0.131C/decade.
In other words, if we were to use the last 29 years as climatological norm to predict what will happen over the next 29 years, what I’m saying is that 0.131C/decade is a better estimate than 0.169C/decade.
And nothing anybody has said, in comments, materially undermines any of this. In fact, most of it completely misses the point.
Basil
I doubt that anything I could say at this point will change the minds of those bothered most by what I’ve done, and I need to get busy on Part III (which will be more of a wrap up, than anything new or exciting), but if it will help in the least little bit, what I’m doing is something analogous to this:
Imagine a 30 year period in which a number is constant at a value of 20 for the first 15 years, and is constant at a value of 10 for the last 15 years. Who here thinks they can compute the “best estimate” of what the average will be for the following 30 years by fitting a trend line through the data? If this pattern repeats every 30 years, the trend is zero! Fitting a straight line regression through the data leads us to imagine that the trend is negative. The only way to accurately determine the “trend” would be to separate the series into two parts, where we would find that in the first part the “regression” is a constant 20, with a slope of zero; the “regression” in the second part is a constant of 10, with a slope of zero; and the best estimate for the next 30 years is a constant of 15 and a slope of zero.
Is that too hard to fathom?
Fitting straight lines through cyclical data creates all sorts of problems, of which correcting for serial correlation is only one. Controlling for discontinuities is another.
Yet supposed experts or authorities use straight line regression all the time, like here:
http://www.remss.com/data/msu/graphics/plots/sc_Rss_compare_TS_channel_tlt.png
Well, excuse me for having the temerity to suggest that there might be other, and better, ways than this, to estimate what the real trend has been for the past 29 years.
Interesting that noone seems to take the step-change proposition seriously. Given a 10 degree local daily variance, and a 0.5 annual global average variance, the thermal inertial argument seems flawed (oceans have massive capacity, but limited flux. For the atmosphere, I have my doubts.) For me, step changes suggest positive feedback, short term perturbations resulting in transition from one regime to another. Who is claiming that there are not localised climate systems which do not sit close to a tipping point in their behaviour?
Lucia,
Menne specifically leaves open that the newer model may not be more accurate than previous descriptions – if you read the very start of the conclusions in the paper, it says so.
“Yes. Someone, somewhere, once found a hinge point at 1975.”
Not someone somewhere, but several people referenced in that paper. Is there a reason to make them sound like random clowns on the Internet when they are published statisticians (that support Lee’s claim).
By the way the support for Lee’s claim is easy to find – just search for 1975 in that paper. Or ask Lee to do it for you.
“But Menne specifically shows that disappears on further analysis! ”
He showed that it might disappear on further analysis.
Basil
Could you comment on the seemingly 30 year periodicity that both GISS and HADCRUT appear to show all the way back to 1880 and 1850. Start at 2004 (I use this asBriggs wavelet analysis indicates a cooling started then and it fits pretty good :] -cherry picking!! Dell commented on this in part one, but no one picked up on it.
During the period 1944 to 1974 Hadcrut shows a small warming and GISS a small cooling. This was interesting because Dell indicated (in the latest thread) that “the infamous 1975 Newsweek article on “The Cooling World’ shows a temp drop of about .65 degrees during that period of time.”
We could be just naturally going into another cool period if this periodicity has any legs.
Obsessive,
Regardless of what statistical test is used to assign a “hinge point”, any analysis of 20th century temperature trends, IMHO, has to start with three little letters: PDO. Here’s a comment and graph I posted at CA. I’m sure there’s a more thorough treatment someplace, I just haven’t found it yet.
http://www.climateaudit.org/?p=2223#comment-222189
David Smith has also done some extensive looking at other Oceanic trends that can impact global temperature trends.
Chris: you said
“Ask the analyst if the technical trends suggest that the stock will go higher or lower. Guys, Wall St. have been doing this type of analysis for decades. Why re-invent the wheel?”
If you would go back to the GW’s IPO (about10,000 years ago), I’ve would have sold this loser a long time. The only way to play this stock would be to short sell it about every 1500 or 500 or 60 years.
Basil,
OK, I followed your advice re tinypic. Here is the HADCRUT3 monthly data since 1997, plotted raw, and with it’s 12-mth moving average.
here
And here is the 12-mth curve, from the previous plot, inverted. Since I can’t preview here, I’m just hoping it works. You’ll see it looks exactly the same as the second plot in your part 1 thread.
On the general philosophy of curve fitting, I refer back to what I said early in this thread. You have a set of data, and want to see if it can be explained by some model that makes sense to you. So you check the differences, and see if they look like noise. If so, you say the model fits. It won’t be the only one that does.
You can always find something that is closer to the original data that fits better. It’s a compromise between fit and meaningfulness. And I think you have gone too far away from meaningfulness, especially when you have abrupt temperature changes, which imply infinite heat fluxes.
Basil,
No, that tinypic didn’t work, though I did what they said. I’m hoping this will:
Here is the HADCRUT3 monthly data since 1997, plotted raw, and with it’s 12-mth moving average.
here
And here is the 12-mth curve, from the previous plot, inverted.
Basil,
OK, nearly there. The plots are actually linked in the last post, but it isn’t clear. Wish I could preview. Hope this works:
Here is the HADCRUT3 monthly data since 1997, plotted raw, and with it’s 12-mth moving average.
And here is the 12-mth curve, from the previous plot, inverted.
Correction:
in describing the Chow test, where I said “mean square error,” I should have said “sum of the squared residuals.” The Chow test calculations are correct, just not my representation of them.
John M
Thanks for that. I have seen graphs like that before. Are there any records or way of knowing what the PDO were back to 1850? The more I look at the data and the correlations of the sun, PDO, and other parameters (and I realize that correlation is no definitive of anything) with temperature the less likely I am to believe that increased CO2 has any overriding effect on climate and the more concerned I get about the headlong rush to “control climate change”.
The Gorites can’t lose this one. While we all get on here a discuss the fine points in the science of global warming vs global cooling, they left are now calling for protection against any and all climate change. Got that “Climate Change”.
This debate and movement is not about the that actual climate, it’s about the take down of the democratic free enterprise societies on planet earth. They mean to replace it with a Marxist-Leninist brand of socialism in a world of serfs under the auspices of the United Nations.
I don’t meant to belittle the work being done to disprove the loons who are selling us this load of crap, but it simply is not the issue. The issue is global slavery vs freedom.
We need to get a lot more political about this war with the Climatites and elect people who will not fall for it. Canada has Stephen Harper who has stated that he knows it’s a lot a crap and that it will destroy our economy if we do what the Kyoto Accord demands. He will have to do something to appease the zealots who insist we ‘save the planet’ or he will lost too many votes. My own provincial government in British Colombia has just levied a seven cent per liter on gas to “save the planet”. That would amount to about thirty cents per gallon.
Add to that the diversion of food stuff in favor of growing fuel for cars and trucks and we will have a world wide famine soon. All food will go up exponentially . The poor will starve and the more well off will go broke trying to keep up.
This is a global extortion the likes of which we have never seen before.
This is a global extortion the likes of which we have never seen before. John West (14 March).
Harsh words in a teeth-gritting post, John; a post I wish I could dismiss as a silly rant, if only for my own comfort… but I can’t, because it seems the unbelievable is real and something malignant has taken hold amongst us. The money hunger I can understand; money has always trumped morals amongst some. I can even understand a quest for personal power. I cannot understand the missionary zeal, or the slack-jawed awe, amongst the wider population which is feeding both.
I herewith endorse your concerns; and note to our host I believe your post, John, and my brief response here, is very much on topic and a necessary mix and caution to give urgency to the vital knowledge being generated here, and its dissemination through Watts Up With That?.
Nick,
A preview feature would be nice. I’ve wished for that myself too, and time or two.
Well, like they say, a picture is worth a thousand words. And now that I can see, visually, what you’ve been trying to say, I see your point.
Now try to see if you can see mine. If we start with your plot, the uninverted 12 month moving average, and start at the most recent point, and work backwards, at a point somewhere in late 2001 (yours is not labeled, but we can work it out from the tic marks, around #70), the 12 month MAV drops back to a point equal to where it is at the end of the graph (i.e. January 2008). Between those two points, there is no “net” cooling or warming. While the anomaly rises and falls between #70 and #144, at #144 it has dropped back to where it was at #70, indicating no net change over that time frame. That’s six years of no “global warming” even by your way of representing the data.
That is all I need to justify what I’ve done. My “scientific curiosity” makes me wonder if the most recent six years are somehow different than the 23 that proceeded. And so I expose my curiosity to the possibility of being wrong by performing a Chow test.
There is no philosophical or scientific basis for contending that what I’ve done is illegitimate. If your point is that the last six years are NOT different, in any meaningful way, than the proceeding 23 years, then state your point in the form of a falsifiable hypothesis and tell us how you would propose testing it.
Basil
for those interested here is another change point approach
http://www.beringclimate.noaa.gov/regimes/Red_noise_paper_v3_with_figures.pdf
the author also has an excell plug in which is fun to play with.
I’ll see if I can dig it up.
Also, I’m glad that folks finally got to the end of Menne’s paper. his method
folks will be used to do adjustments for the temperature series of ushcnv2.
so, who knows what that will produce.
Basil keep up the good work i enjoyed your post
http://www.beringclimate.noaa.gov/regimes/help3.html
works in excell
FrankD-
Sure. Menne leaves open that his analysis may not be better than the previous one. Lee opened with the first paragraph on part I with a brief comment to Basil accusing him of cherry picking (heaven knows what) and linking to a graph that supposedly picks it’s endpoint for Basil’s as yet unposted analysis “the right way”.
Lee has been vague, refused to link to articles explaining how or why 1975 was selected in the graph he posted. Finally, when pressed, suggests we read the paper SteveMoscher selected. That paper may or may not explain any method used to pick the 1975 in the graph Lee linked. Moreover, it suggests that, yes, there is a way to get 1975 as a hinge, but that vanishes if we use the method proposed in the only specific paper Lee proposes. Lee hasn’t explained how this new analysis will affect Basil’s as yet unpublished post.
So what, precisely are we to make of Lee’s immediate accusation of cherry picking?
If he wants to debate whether or not hinge points exist, that’s fine. But, we can’t know what Basil is going to say in part III. Until Basil uses these hinge points to make some sort of conclusions, how can anyone accuse him of any such thing.
OP
Here’s an opinion piece by Joseph D’Aleo who takes it one step above what I did (includes a correlation with both PDO and AMO). I think this is what Anthony was referring to recently. It’d be nice to see an exposition on the same thing in a refereed journal.
http://www.energytribune.com/articles.cfm?aid=544
As far as going back further, Michaels had a post referencing a peer reviewed paper. The figure looks kind of cartoonish, so it may be for demonstration purposes only. I haven’t read the original paper.
http://www.worldclimatereport.com/index.php/2007/02/13/more-bad-news-about-el-nino/
Obviously, the farther back you go, the less certain the data, so finding a distinct link between PDO and global temperature pre-1900 may be tough.
This is all fascinating stuff. I’m not sure how any card-carrying scientist could let the phrase “settled science” slip through his or her lips when it comes to the global temperature trend and its causes.
Basil – thank you for your insight and statistical analysis. I find them helpful and thought provoking.
John West – could not agree more
Anthony – many, many thanks for publishing and maintaining such an informative website. Your’s is one I visit very regularly and take the time to read thoroughly.
As others have pointed out – Sol remains very quiet.
REPLY: Thanks for the kind words. See latest post on “sol” at home page of website
There’s an interesting “discontinuity” in the Ap index graph in this latest post on the solar activity.
http://wattsupwiththat.wordpress.com/2008/03/15/sun-still-blank-no-sign-of-cycle-24/
Busy today, moderation may be slow. Don’t fret if comments don’t show up for a few hours.
I like ‘climatites’. That resonates.
========================
Basil, having worked all my life ( I am now 67 years old, ) as a engineer in the petroleum industry, I have seen more of the world than most, the only thing that is sure in this life, is the weather, it’s going to change, I do not believe in global warming as portrayed by the MSM, Gore-al. In my work, having to deal with multi cultures, idiots, theoreticians, ignorance, and heavy on the ignorance, what has, when I don’t know, powered my decisions is logic. Having followed you and the full spectrum of comments that you have generated, and not knowing night from a hinge, logic tells me that you have the answer.
Lee, I bet your Mother was glad to get you out of the house.
You have to laugh. People keep telling us to use statistical techniques to find the trends because simply eyeballing isn’t good enough. Then we find there’s at least a half dozen techniques, none of which give the same answer. Shall we just go back to eyeballing it because it’s quite clear that modern statistics is about deciding what you want to show and then finding a mathematical method to wrap up your bias in? And such a quaint, if rather ridiculous, idea that a few straight lines can represent a highly non-linear, chaotic system – please! Likely Scaffeta and West were the closest with their curvy fits. Then theres another chap telling us to ask those geniuses on Wall Street to do the analysis for us. Words fail me!
Basil,
Yes, I see that modified interpretation. Here is a summary of my objections:
1. It’s post hoc. You postulate a change in 2002 because the temperature then (smoothed) is equal to what it is now in 2008. But suppose, as some assure us, temperatures for the next few years go down. Similar analyses would find break dates right through the 1990’s. But if something really did happen, the timing can’t depend on when future people choose to do their analyses.
2. Discontinuous temperature. Economic analogy – OK you can have a discontinuous interest rate or share price. But suppose you are studying the number of houses. The rate of increase may change, but the actual number of houses can’t suddenly change. Heat is like that. You’ve postulated that in a month in 2002, the temperature rose suddenly by 0.2C. All other months were normal. Now 0.2C is what about a decade of AGW heating is supposed to achieve. Some people find that hard to believe. To expect that such heat would accumulate in a month, then go back to normal, is not plausible. And OK, you could say that the model is idealised, and the heating may have taken longer. But still, it just isn’t physical.
3. Too many degrees of freedom. You’ve compounded this by trying to model the 1998 El Nino rise separately. This creates a justified accusation of bias. But I think it also probably doesn’t make much difference, and just confuses the issue. Just two line segments would do.
4. More on degrees of freedom. With linear regression, you just fit two parameters, and the stat analysis tells you whether the slope is well determined, or whether other slopes might have fitted nearly as well. Only in the former case are you getting a good answer. Here you are introducing a wider group of functions – multiple line segments. You should really test whether your fit is uniquely the best of that class – that other perhaps very different looking models from the same set would not have done almost as well. That is the point of letting the model try to determine the breakpoint, as Menne has done. All your Chow test does is test whether a member of an arbitrary subset the large class is better than a member of another subset (the single line segment).
Kim. you know there is no such thing as the climate. Some day we should
dereify it.
Basil,
I agree with Nick Stokes here – you want to do more to justify your selection of change points. For a casual blog post, what you did is probably fine. But really to be persuasive, there’s more needed (perhaps this is coming in part 3?).
In your post above, you mentioned that the major point of your Chow analysis is that the global temperature trend is less than what’s given by a simple linear fit. You may well be correct. But you haven’t yet shown it. Is your significant result special among all the potential change points you could have “Chowed” down on (so to speak)? You seemed to dismiss this in a comment above as “a likely waste of time and resources.” But it’s crucial to establish that the change points you analyzed fit the data especially well – better than some number of other possible sets of change points, particularly if you want to conclude that it’s a better representation of trend than a simple linear fit.
Step Change?
The selected base years of NINO3.4 anomaly data dictate the relationship of El Nino and La Nina data. Do they reflect their impact on global temperature? Probably not.
The term step change was used approximately 29 times in this thread. How could the 97-98 El Nino have caused a step change in global temperature if it was followed by a La Nina that lasted longer? Following that line of thought, pick an ENSO SST index (ONI, NCDC NINO3.4). I chose the NCDC data. Multiply the average 97-98 El Nino data by the number of months the anomaly was above 0 deg C. Then multiply the subsequent average La Nina data by the number of months the anomaly was below. The La Nina “deg months” (-33) are greater than the El Nino (25), so if there had been a step change it should have been toward cooling, not a warming.
For example, now shift the base line down 0.6 deg C, making the El Nino data more prominent, and run through the calculations again. The beginning and end shifts forward, too. The “degree months” of the El Nino (39) now exceed the La Nina (10). That would have caused a major positive step change.
Does this prove anything other than anomaly data is impacted by its base? No.
Refer to Trenberth et al (2000) “The Evolution of ENSO and Global Atmospheric Temperatures” if you question the role of the ENSO. Based on his linear trend, El Nino dominance raised global temperature 0.06 deg C from 1950-1998. Add in the immediate effects of the 1950 La Nina and the 97-98 El Nino, and ENSO is responsible for over 0.4 deg C of the rise in global temperature between 1950 and 1998.
Using 1950 as opposed to the mid-to-late 70s for the starting point significantly reduces the positive linear trend, since from 1950 to the 70s, the trend was negative. How much does it reduce it? If I had the statistical capabilities of your guest posters, I’d quantify it. Since I don’t, I won’t.
When I see this kind of pompousness from our side it scares me. First the guy thinks that he can put up posts separately over several days, but that we should wait to criticize? why not him wait to post!?
And then the whole overdone “Chow test” when the guy can look at his data and see the random walk in it, yet he insists on looking at 10 year patterns for breaks in a long term pattern. And he doesn’t understand the point about human decision for where to test break points as being a decision point.
It just scares me, when I see my side being so simple-minded.
(No offense.)
A Step Change Around 1998?
Here’s MSU Global, Northern Hemisphere, Northern Extratropics, and North Pole temperature anomalies from 1978 to 2007. What diverges and what causes it?
http://tinypic.com/fullsize.php?pic=zswazc&s=3&capwidth=false
Here’s MSU Northern Hemisphere and North Pole temperature anomalies and trends from 1999 to 2007. The North Pole trend is more than double that of the Northern Hemisphere.
http://tinypic.com/fullsize.php?pic=mcb5md&s=3&capwidth=false
Here’s MSU Northern Hemisphere and North Pole temperature anomalies and trends from 1978 to 1997. Prior to 97-98, the North Pole and Northern Hemisphere trends are just about equal.
http://tinypic.com/fullsize.php?pic=2qjvwqw&s=3&capwidth=false
Hi,
could Basil describe the way he finds the linear regression for his different segments, as I find very different value if I try a linear fit (least square) on the Hadley center since 2002 (up to 2008). I am using monthly means.
Regards.
I second Gal’s request. A blogger is having problems in replicating the -0.4C/decade from figure 2. Could Copeland or Watts please elaborate? – thanks
http://jhubert.livejournal.com/181274.html?view=844058#t844058