A guest post by Basil Copeland
Lucia, at rankexploits.com, has been musing over Tilo Reber’s posting of a graph showing flat 11 year trends in the HadCRUT land-ocean global temperature anomaly and the two MSU satellite data sets, UAH and RSS. In answer to the question whether global warming is on an 11 year hiatus, “not quite,” says Lucia. She challenges Tilo’s omission of the GISS data set, because notwithstanding questions about the reliability of GISS, it still shows a positive trend over the 11 year period in question. Unless all the measures show a flat trend, Lucia’s not ready to conclude that global warming has been on an 11 year hiatus.
I understand the desire to look at as many metrics as possible in trying to divine what is going on with globally averaged temperature. I also understand the reasons for questioning the reliability of GISS. What I don’t understand is why the only measure of trend that seems to count is a trend derived from linear regression. William Briggs recently had an interesting post to his blog on the relationship between trends in CO2 and temperature in which he introduced the use of loess lines to track trends that are not represented well by linear regression. Loess refers to a type of locally weighted regression that in effect fits a piecewise linear or quadratic trend through the data, showing how the trend is changing over time. Especially in an environment where the charge of cherry-picking the data — choosing starting and ending points to produce a particular result – is routinely made, loess lines are a relatively robust alternative to simple trend lines from linear regression.
Click for a larger image
Figure 1 fits a loess line through the data for GISS using the same 11 year period used by Tilo Reber (except that I’ve normalized all anomalies in this discussion relative to their 11 year mean to facilitate comparison to a common baseline). The red line is the GISS anomaly for this period, about its mean, and the blue line is the loess line. While it varies up and down over the period in question, I would argue that the overall trend is essentially flat, or even slightly negative: the value of line at the end of the period is slightly lower than at the beginning of the period. What this loess line shows is that a linear regression trend is not a particularly good way to represent the actual trend in the data. Without actually fitting a linear trend line, we can reasonably guess that it will trend upwards, because of the way the loess line is lower in the first half of the period in question, and higher in the second half. Linear regression will fit a positive, but misleading, slope through the data, implying that at the end of the period the GISS is on an upward trend when in fact the trend peaked around 2006 and has since declined.
Click for a larger image
Figure 2 is rainbow of colors comparing all four of the metrics we tend to follow here on WUWT. Not surprisingly, the loess lines of HadCRUT, UAH and RSS all track closely together, while GISS is the odd duck of the lot. So what does this kaleidoscope of colors tell us about whether global warming is has gone on an 11 year hiatus? I think it tells us rather more than even Tilo was claiming. All of the loess lines show a net decline in the trend over the 11 year period in question. It is relatively minor in the case of GISS, but rather pronounced in the case of the other three. Of the other three, the median anomaly at the beginning of the period, as represented by the loess lines, was 0.125; at the end of the period, the median anomaly had dropped to -0.071, for a total decline of 0.196, or almost 0.2C.
Global warming on hiatus? It looks to me like more evidence of global cooling. Will it continue? Neither linear regression nor loess lines can answer that question. But the loess lines certainly warn us to be cautious in naively extrapolating historical trends derived by simple linear regression.
Not even GISS can support the conclusion from the last 11 years of data that global warming continues to march upward in unrelenting fashion.
What strikes me about that graphic is that you can see what I have heard talked about over at Climate Audit. Hansen’s GISS diverges cold in the early years and warm in the later years. You can see the impact of adjusting the past colder and the present warmer around some break point. Now since the break point for each data stream (each recording station) isn’t at the same point, there is a long period of convergence with the other data sets in the center but as you go back farther in time, you get on the “cold” side of more and more of the break points and the negative adjustments accumulate. In the center you have some adjustments going positive but some are still negative and as you get closer to the present you eventually get to a “tipping point” where so many are adjusted positive that it begins to become more obvious in the divergence from the others in the warm direction.
Gary P: the cosmic ray intensity at solar minima returns to the same value [alternating a bit between a ‘peaked’ and a ’rounded’ top – for well-understood reasons] and the current minimum is not special. See e.g. http://www.leif.org/research/CosmicRayFlux.png so if cosmic rays are important then the temperature now is no different from what it was in 1954.
Mark,
I have to disagree with your point about cherry picking. There were seven El Nino/La Nina events over that time period. All of them influenced the slope. The largest influence was not even the 98 El Nino, but rather the La Nina that immediately followed it. I took a stab at quantifying the effects of the 7 El Ninos and La Ninas during that period, and it appears to me that there is no justification at all for the claim that the flat trend is caused by a selection of end points. Here is what I did. The period is slightly different, since it begins with Jan 98, but I think that the results would be similar regardless.
http://reallyrealclimate.blogspot.com/2008/05/ten-year-hadcrut3-enso-effects.html
lucia,
Since 1988, a loess line through GISS looks like this:
http://i32.tinypic.com/14bj7ef.jpg
At the end, the trend is still positive, at a rate of 0.056C per decade (well below the current IPCC scenarios/forecasts).
Basil
Basil:
While I respect Lucia’s opinion on which data sets to use, in my mind one has to draw a quality standard somewhere. On top of all of the quality problems that Anthony has documented with the data, I think that there have been adjustment choices made with GISS that are designed to give warming results that increase the trend. I think that this paper does a good job of explaining some of those choices.
http://www.friendsofscience.org/assets/documents/CorrectCorrections.pdf
crosspatch also has a good take on it above.
And of course the divergence over the last decade should also tell you something. The decadal divergence is .13C. That is well over half of the .2C decadal trend that is blamed on CO2 by the warmers.
Once we do a better analysis on HadCru data we may find out that it has many problems of its own. But for right now I have to draw a line in the sand somewhere, and my personal choice is not to accept GISS for any purpose. I believe that it would be excusing agenda oriented work to use it. Just my opinion.
Regarding the use of a linear regression trend line, I don’t wish to claim that this is the best method. But it seems to be a commonly used method, even among the warmers themselves. So in order to have a common basis of comparison and to avoid arguments about methods, I will probably continue to use it. On top of that, I’m lazy and Excel will do the calculations and fit the line for me.
Sorry about taking so long to respond, but I’ve been motorcycling through the Rockies – trying to get those poor trees a little more CO2 to grow with. 😉
Leif Svalgaard (20:21:43)
I believe we were in a cooling phase in 1954. So if it is the same as 1954, I would say there would be some more cooling on the way. Maybe 20 more years worth.
Gary Plyler,
Sorry that I so poorly stated my point. By referencing the realclimate.org, I was not endorsing the position. I agree with you that the arguments at that site are simplistic. After reading items in “Whatsupwiththat”, “”realclimate.org” argument in ‘The temperature record is simply unreliable’ is silly.
I was merely trying to point that there should a website similar to that site with a structured ouline of arguments, but presenting the counter AGW position. From what I seen, this website has many of the elements, but not in a structured outline format.
Leif Svalgaard writes:
The relevant measure is ionization near ground level – since low level clouds produce cooling but high level clouds do not make much difference.
Our records and data for ionization near ground level are fragmentary and incomplete. For those years for which we have good data, we get a pretty good correlation – more ionization, cooler weather, less ionization, warmer weather.
The data that supposedly shows a poor fit relies on various proxies that one might think might reflect ionization at seal level – but, of course, if one takes this approach, then a poor fit is pretty much guaranteed regardless of the truth of the hypothesis, since proxies are proxies. And if one *wants* a poor fit, one is bound to find proxies that will deliver.
Basil,
Thanks for that 1979-present graph. As you said, we will all see in a few years how this whole thing plays out. The one take-away seems to be that we shouldn’t place too much weight on the anomalous first few months in 2008 unless they continue. If 2008 as a whole ends up being considerably below the trend (say, below 0.15 relative to 1979-1998 baseline), it might serve as a bit of a wakeup call to the climate science community that we need to reassess some assumptions of the magnitude of natural forcings.
Mike Dubrasich,
Not to beat a dead horse too much, but the IPCC projections are far from monotonic. See http://www.realclimate.org/images/runs.jpg for a reasonable assembly of specific model runs.
Bill Illis,
While I admire the aversion to cherrypicking, I’d be a bit wary of picking a single month for your endpoint given the scope of monthly variability. Comparing the mean 1979 temperature to the mean temperature over the last 12 months would be less noisy and yield 0.28 degrees warming in RSS and 0.19 warming in UAH.
Zeke,
Consider this: there is no other 10 or 11 year flat period from 1978 on, except for 1982-83 to 1992-93 because of the Pinatubo eruption. So, no matter what method you want to use, the flat/downward trend from 1997 or 1998 to 2008 is unique. No doubt you will say this is because of the extreme nature of the 1998 El Nino…but then also consider that when it comes to ENSO, the 2008 La Nina has cooled temperatures to the same point that the 1999 La Nina did 9 years earlier – and they were of comparable strength. No other La Nina during this 30 year period was as cool as the previous one…until 2008.
Basil,
Smoothing the curve to see the longer term effects is fine. And LOESS is a fine smoother, one of many. All smoothers work by replacing values by some average of values over a nearby interval. Generally, the bigger the interval, the smoother, but then you lose detail over a timescale that you really might want to see.
But it isn’t a magic way of getting a better estimate of a slope over a longer interval. To see this, consider why you don’t just work out the slope using the first and last value. That actually gives you an unbiased estimator. You’d be over as often as under. But it has high sensitivity to noise.
By taking the difference between smoothed values, in the right way, you preserve the unbiasedness, while reducing the noise sensitivity. And the more smoothing, the more you reduce it. Any good smoother works like this.
If you keep increasing the smoothing, you get to linear regression. It is the unbiased estimator which is (wrt sum of squares) least sensitive to noise.
Gary Plyler:
Thanks for the CR website.
Am I reading the following correctly? CR intensity is about 15% more than during the time around 2002?
http://cosmicrays.oulu.fi/Request.dll?Y1=2000&M1=Jan&D1=01&h1=00&m1=00&Y2=2008&M2=May&D2=01&h2=00&m2=00&YR=00&MR=00&DR=00&hR=00&mR=00&PD=1
Leif Svalgaard: Off topic. Have you posted, or do you know the wherabouts of, monthly TSI reconstruction values since 1850 or earlier? If so, please provide a link. I can only find annual figures.
Hey, did you catch the latest environmentalist hoax? Reported on none other than the Huffington Post…HA!
http://www.huffingtonpost.com/2008/06/22/hoax-the-lost-amazonian-t_n_108541.html
Make up whatever lies you like, after all, you are trying to save the world.
What’s the point of this “rebuttal” here or at realclimate?
No one has ever claimed the IPCC says the weather is monotonic. The IPCC graphs show the underying climate trend (ensemble average of weather) is expected to be monotonically increasing. That’s all anyone claims: climate is predicted to be monotonic.
The authors at realclimate need to stop rebutting all these strawmen that someone is claiming weather is monotonic. No one claims that. So, yes, if you keep repeating that rebuttal, you are not only beating a dead horse, but you are beating a dead straw horse that was concocted for the express purpose of making it easy to beat.
Meanwhile, the live kicking horse is still there snorting at you.
With regard to that link: The RC people also need stop trying to test models using model data and ignoring real honest to goodness earth data.
In that post, Gavin wants to claim the spread of model predictions for 8 year trend is due to “internal variablity (weather!)”
From a fundamental view point, that spread is not wheather noise. Weather noise conctributes to the spread– but so do the different parmeterizations in different models, and different treatments of forcing in the past (half with volcanic eruptions, half without). It is not even clear any individual model gives correct weather noise, and this could be affected by ocean treatments, parameterizations, etc.
So, clearly, one should at least do a quick reality check of the spread of those 8 year old trends to data. It’s easy to see it’s way to big to be due to weather only. If we are to believe Gavin, those weather noise during a period without volcanos produced a variability of 8 year trends that exceeds the measured variability in the full temperature record. That is to say: If we calculate the variability of all 8 year trends seen over all time, including contributions to volcanic eruptions, measurement noise (including things like the errors due to bucket-jet inlet transitions etc.) that “measured” variablity in 8 year OLS is distinctly smaller than his claimed “weather noise”!
Taking out the bucket noise and the periods with volcano eruptions, the actual variability of OLS trends during periods without volcanic eruptions is noticably smaller than the variability I estimate from residuals of an OLS fit– and between 1/2 and 1/3rd the side of the variabilty Gavin concocts by assembling all his model predictions for a periods without “weather noise”.
It seems to me that the colling observed recently follows the solar flare levels
http://en.wikipedia.org/wiki/Image:Solar-cycle-data.png
This would just be a coincidence, I guess, if it weren’t for how well it jibes with the cosmic ray idea, since solar flares clear the heliosphere of cosmic rays, and less of them mean more cosmic rays.
As the peaceful 99.98% of human history is not widely known yet (Raymond Kelly: Warless societies, Douglas Fry:Beyond War-The Human Potential for Peace), because of the cortisol addicted aggressives’ propaganda, the reconciliating decentralized Autarky of our peaceable ancestors is usually not proposed as the best, tested solution for ANY climate change.
The hushed-up Swiss system of collective control of fear and guided overpopulation through mutual-aid traditions, reforestations, autarky farms and unified list for elections, may help rescuers prepare kids for the future.
What this excercise really shows is that our climate is too complex to represent by any single measure. An El Nino here, a volcano there causes any trend to be called into question. When we look only at temperature, I think there is a tendency for we skeptics to be disappointed by any upward trend or encouraged by any flat or downward trend. Since we aren’t in a period of glaciation we should expect an upward or flat trend. The rise in CO2 fits a linear positive trend much more easily than temperature. So how do temperatures prove CO2 driven warming?
This is obviously a flawed perspective. Given that Loess Curves will appear different whenever the period is different, then the only acceptable period is the longest possible one, and we cannot really appreciate the end slope seriously, because it is constantly changing between the graphs presented.
This is cherry picking at its peak, and I expect more seriousness from you.
REPLY: I grow so tired of this argument about “cherry picking”, particularly since its becoming the standard complaint for ANYTHING that somebody doesn’t like about a time series analysis.
The period is a period of interest. The near present is far more interesting than 50 years ago because it gives clues to the near future. We analyse it, we present the results, we talk about the results. It is an opinion with supporting evidence, just as James Hansen’s GISTEMP work or Roy Spencer’s UAH is an opinion with supporting evidence.
Everybody, everyday, in every endeavor “cherry picks” something. Choosing things that interest you and not engaging in things you don’t. Which is probably why you read THIS blog as opposed to blogs on Paris fashion trends or fantasy football.
I expect more insight from you. -Anthony
xanthippa: I’ve heard of Thatcher’s AGW work, but haven’t found documentation. What study was the one you mention?
Nick,
That’s a very good exposition which, in sense, brings us back to the problem of choosing when and what data we look at. My overall preference is to look at all the data and I’ve grown fond of Hodrick-Prescott for smoothing because it reveals periodic oscillations that can be corroborated with spectrum analysis. Since we’re focusing in here on GISS, and I happen to have this data handy, here is a chart that reinforces my belief that we’re in a period of natural cooling, and that even GISS can be used to show this:
http://s3.tinypic.com/2efozys.jpg
The grey lines are the raw GISS anomalies, the blue line is a smoothing with HP, and the red lines are the first differences of the blue line. The first differences reveal decadal and bidecadal changes in the rate of warming. We’re in a particularly strong cooling phase right, and even GISS shows a downturn since 2006 in this depiction. Now it may be nothing more than a downturn like that you can see around 1990 — brief, and then followed by a steady march upward. Or, it may be like the downturn we see in the early 1940’s, which was followed by a long period of cooling.
I honestly do not know which is more likely. I do know that the current downturn is inconsistent with the power frequently attributed to AGW. I also know that a big chunk of the warming trend in recent decades owes increases in reported temperatures over the northern hemispheric land masses, and that the trend is much less in the tropics, something else hard to reconcile with CO2 induced AGW (but not so hard to reconcile with UHI induced AGW). Then we have the matter of whether any of the decadal or bidecadal variation can be attributed to solar, and if so how much.
In the end, the trick is to not be so vested in a particular way of looking at things that you cannot see other things that matter.
Thanks for the observations. I thought you described well the benefits and pitfalls of smoothing, and especially the issue of how if you smooth over too long of a period, you may miss something you want to see. On that, with HP, I’m using a level of smoothing that basically smooths out cycles of less than 6 or 7 years, leaving the decadal and bidecadal oscillations, because these are what I want to see. Based on the pattern I see in the image linked to above, I’d say there is slightly greater chance that in the decades ahead we’re in for a repeat of the mid 20th century than the late 20th century.
We shall see. 🙂
Basil
So, I finally got around to plotting trend lines from 1997 to present and, low and behold, all the series show a slight warming based on a linear regression over the entire period.
http://i81.photobucket.com/albums/j237/hausfath/1997-2008temps.jpg
Now, can anyone spot the difference between my graph and Tilo’s (other than the normalization of the data to a common baseline, which has no effect on the trend)? I’m rather perplexed how we could obtain different results from the same dataset, unless he is starting from some point other than January 1997.
Great post.
I have a somewhat off-topic question for everyone/anyone..
Does anyone know how I can get my hands on a plans and guidelines for building a weather station? Dimensions, requirements? Any help would be great and keep up the good work!!
Zeke,
You wrote “The one take-away seems to be that we shouldn’t place too much weight on the anomalous first few months in 2008 unless they continue.” If you look at the chart I just linked to in my reply to Nick, the first few months of 2008 are not particularly anomalous when looking at the longer pattern. Downturns like this occur with roughly bidecadal frequency. This has already proven, though, to be a deeper downturn than any experienced since 1964 (look at the troughs of the red line in the following figure), and so if this continues, we’re getting into territory not seen since the mid 20th century cooling.
Here’s the link again, for convenience:
http://s3.tinypic.com/2efozys.jpg
Basil
james: And if one *wants* a poor fit, one is bound to find proxies that will deliver. I guess that works the other way too: And if one *wants* a good fit, one is bound to find proxies that will deliver.