A guest post by Basil Copeland
Lucia, at rankexploits.com, has been musing over Tilo Reber’s posting of a graph showing flat 11 year trends in the HadCRUT land-ocean global temperature anomaly and the two MSU satellite data sets, UAH and RSS. In answer to the question whether global warming is on an 11 year hiatus, “not quite,” says Lucia. She challenges Tilo’s omission of the GISS data set, because notwithstanding questions about the reliability of GISS, it still shows a positive trend over the 11 year period in question. Unless all the measures show a flat trend, Lucia’s not ready to conclude that global warming has been on an 11 year hiatus.
I understand the desire to look at as many metrics as possible in trying to divine what is going on with globally averaged temperature. I also understand the reasons for questioning the reliability of GISS. What I don’t understand is why the only measure of trend that seems to count is a trend derived from linear regression. William Briggs recently had an interesting post to his blog on the relationship between trends in CO2 and temperature in which he introduced the use of loess lines to track trends that are not represented well by linear regression. Loess refers to a type of locally weighted regression that in effect fits a piecewise linear or quadratic trend through the data, showing how the trend is changing over time. Especially in an environment where the charge of cherry-picking the data — choosing starting and ending points to produce a particular result – is routinely made, loess lines are a relatively robust alternative to simple trend lines from linear regression.
Click for a larger image
Figure 1 fits a loess line through the data for GISS using the same 11 year period used by Tilo Reber (except that I’ve normalized all anomalies in this discussion relative to their 11 year mean to facilitate comparison to a common baseline). The red line is the GISS anomaly for this period, about its mean, and the blue line is the loess line. While it varies up and down over the period in question, I would argue that the overall trend is essentially flat, or even slightly negative: the value of line at the end of the period is slightly lower than at the beginning of the period. What this loess line shows is that a linear regression trend is not a particularly good way to represent the actual trend in the data. Without actually fitting a linear trend line, we can reasonably guess that it will trend upwards, because of the way the loess line is lower in the first half of the period in question, and higher in the second half. Linear regression will fit a positive, but misleading, slope through the data, implying that at the end of the period the GISS is on an upward trend when in fact the trend peaked around 2006 and has since declined.
Click for a larger image
Figure 2 is rainbow of colors comparing all four of the metrics we tend to follow here on WUWT. Not surprisingly, the loess lines of HadCRUT, UAH and RSS all track closely together, while GISS is the odd duck of the lot. So what does this kaleidoscope of colors tell us about whether global warming is has gone on an 11 year hiatus? I think it tells us rather more than even Tilo was claiming. All of the loess lines show a net decline in the trend over the 11 year period in question. It is relatively minor in the case of GISS, but rather pronounced in the case of the other three. Of the other three, the median anomaly at the beginning of the period, as represented by the loess lines, was 0.125; at the end of the period, the median anomaly had dropped to -0.071, for a total decline of 0.196, or almost 0.2C.
Global warming on hiatus? It looks to me like more evidence of global cooling. Will it continue? Neither linear regression nor loess lines can answer that question. But the loess lines certainly warn us to be cautious in naively extrapolating historical trends derived by simple linear regression.
Not even GISS can support the conclusion from the last 11 years of data that global warming continues to march upward in unrelenting fashion.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Do we have this same kind of fit for the measurements since 1900? I’d love to see the full 100+ year cycle fitted to the Loess curves.
Basil,
There seems to be a bit of conflict between your point that
“Especially in an environment where the charge of cherry-picking the data — choosing starting and ending points to produce a particular result – is routinely made, loess lines are a relatively robust alternative to simple trend lines from linear regression.”
and
“All of the loess lines show a net decline in the trend over the 11 year period in question.”
For example, if I were to choose 2000 as my starting point, GISS would not show a “net decline”. If I were to choose 1990 as my starting point, all four series would show a net increase. If I choose 1997 as my starting point, all series show a net decline.
Net reductions aren’t really a useful concept in this context as you are just comparing the starting and ending point; rather, loess is good at showing local trends in the data. If you could show the loess lines for the 1979 to present data for all four temperature records (or 1880 to present data for the land-based records), it might be interesting to see how similar or dissimilar the recent trends have been to those in the full record. I’d do it myself, but excel does not support loess, unfortunately.
Jeff and Zeke,
Within the limits of the software I use (gretl), I’ll see what I can come up with. The longer the period, the more smoothing there will be, so some prospect remains for “cherry picking” and I would expect that the downward trend at the end can be “erased” by going far enough back in time. Keep in mind that it is referred to as “local regression” and if you include enough data from the past, you’ll begin to dilute the weight of the data at given points. It will still be better than a linear trend through the data, though.
More later.
Basil
Thank you very much for serious information about global warming. In any case I´d like my grandson to live in a warm, but not too hot world.
Sincerely
Galina Vitkova
Zeke,
Don’t worry about the “cherry picking” argument. Solar cycle 23 (SC23, now 12 years long) has been weaker than the previous 3 solar cycles that encompassed, and probably caused, the global warming from 1972 to 1998. Those 3 solar cycles were some of the strongest since Galileo first noted sunspots. The current minimum between solar cycles 23 and 24 has already been weaker than the SC21/22 and SC22/23 minima for over a year. SC24 just can’t seem to get going with any appreciable strength, and some reputable hypotheses predict that SC/24 will only be 1/3 as strong as SC23.
Two or 3 more years will determine if all bets are off, regarless of cherry picking.
Zeke,
Here’s what I could easily come up with (I’ll need to collate more data to do more):
http://i32.tinypic.com/dggj2u.jpg
It is just UAH, not all of them. And I confess to being a little surprised — not at the lack of decline at the end, but that the loess line comes out higher at the end than the linear fit. Maybe I shouldn’t be surprised, since at this scale, the 1998 El Nino is going to weigh more heavily on the loess result than on the linear trend. Still, the overall warming is less with the loess line than with the linear trend, though at the end the slopes are visually indistinguishable.
Basil
Nice post. I’ve been very surprised to see the use of linear OLS trend lines in analyzing these data-sets. It’s pretty obvious from eyeballing the data that there is serial correlation, which violates the statistical assumptions of OLS. So OLS is invalid in these contexts.
Some claim that OLS is robust to these kinds of statistical violations. But in this case it really matters. Serial correlations means that the trend line is highly sensitive to … you guessed it … starting and end points of the data!
Basil,
I think your UAH loess graph for 1979-2008 illustrates exactly the point of cherry-picking. The argument that using 1997 as the starting point does not constitute cherry-picking since it does not start at the top of the 1998 ENSO is flawed, since it still contains the dominating 1998 ENSO near the first half of the graph, where it will still tend to cause an overall linear regression to be negative. Using the larger range shifts it to the second half of the graph (albeit not by much) which will tend to weight the linear regression upward. Looks to me like the loess is mainly affected by the relatively warm 2002-2006 period.
It seems like the website below needs a conversely stated position website for “How to Talk to a AGW”.
How to Talk to a Climate Skeptic
http://gristmill.grist.org/skeptics#Stages%20of%20Denial
Mark,
I see your point. I do think that as a general principle, loess lines are more robust than linear regression. So they may reduce the problem of cherry picking, but not eliminate it entirely.
I didn’t choose the period in question. I simply those to apply loess lines to provide another perspective to a discussion already underway.
I’ve posted above what happens if we go back to the beginning of the satellite era for UAH, which I imagine makes your point for you.
Basil
Gary P: That SC24 would be as weak as 1/3 of SC23 is likely to be an underestimate. The polar field precursor method predicts SC24 to be 3/5 of SC23. That is still the lowest in the last 100 years. The solar minimum in 1954 [between SC18 and SC19] was even lower than the current minimum, yet SC19 was the biggest ever, so a weak minimum is a poor predictor of the next maximum.
MDDwave (16:07:28 ) : It seems like the website below needs a conversely stated position website for “How to Talk to a AGW”.
I noticed that none of the answers in the above web site were dated after November 2006. Many of them are just factually incorrect with updated data (e.g., satellite records showing no cooling). But you have a great idea; it’d be interesting to see point-by-point debates.
Leif,
Whether 1/2 or 3/5, I think we’ll see moderate temperatures at least until the ramp up to the peak of SC25, and then longer if SC25 is weak. The upward trend in “global warming” may return once SC24 gets underway, but we will not see the same rate of increase we saw in global temperatures through the 1980’s and 1990’s.
I think, then, that regardless of the specifics, Gary P.’s main point is valid: all of our bickering about cherry picking will become moot as the future proves one or the other side of this debate right and the other side wrong.
Thanks, though, for your observation and insight (even if we don’t agree on everything).
Basil
Loess is locally-fitted least squares with weights that decrease as the distance from the data point increases. Two basic parameters determine the “smoothed” line: alpha, the smoothing parameter that governs the weight/distance, and lambda, the parameter that determines whether the fit is linear or quadratic.
Loess gives the analyst choices, as do many smoothing functions. The choices may be evaluated by examining the residuals (actual minus fitted), and sometimes a second loess line can be run through the residual plot to check their pattern (if any).
There is much flexibility in loess fitting, in other words, and some art to it. It cannot be said to be definitive, since there is so much flex. However, loess is just as good as (or better than) a straight line fit for many kinds of data, including the temp time series above.
IMHO, though, one does not need a fit of any kind, but merely examination of the raw data, to see that there is no linearity to it and certainly no clear trend upward as AGW theory predicts. The IPCC predictions (or scenarios, if you will) are for linear temp increases, and that is obviously not happening.
Loess fitting was popularized by William S. Cleveland. His book, Visualizing Data, is a modern classic and I highly recommend it to all data miners. Cleveland was one of those Bell Labs geniuses who invented S, now R, a phenomenal contribution to all sciences, akin, IMHO, to the invention of the wheel.
Since cherrypicking does seem to be a problem with the temperature data, let’s just pick the first data point for the satellite temps and the last.
UAH
December 1978 : -0.199C
May 2008 : -0.18C
RSS
January 1979 : -0.268C
May 2008 : -0.083C
That is an average of 0.1C of warming over the last almost 30 years.
The Models predict 0.6C of warming over 30 years so they are just WRONG.
If 30 years is not enough time to determine whether the models are wrong, then there will never be enough time.
And regression lines sometimes introduce artifact trend lines. Question: What is the regression line for a perfect half circle? Answer: A line at 45 degrees which is an incomplete and inaccurate description of the data of course.
Basil, in my own analysis, I fit rolling linear trends, and then fit the changing slopes to show how trends change over time. These Loess lines diagrams seem to be a fancy way of doing a similar thing, though I don;t think your conclusion is all that robust. I think all you have done is demonstrate 2 things: (1) very recent data given predominant weight will show a negative trend line (which isn’t much different from simply fitting a linear trend line to, say, the last 60 months), and (2) the rate of warming has been declining. I agree with this, as well, but this can easily be demonstrated by fitting rolling slopes.
Nonetheless, it’s always interesting to view things from a different perspective.
Joe
The Idiot
GP: “some reputable hypotheses predict that SC/24 will only be 1/3 as strong as SC23”.
Don’t worry. We understand by the syntax your money may not be on 40. Considering the lateness, and thereafter, the slowness of 24, anything under 67 (= 75 – 8) should be safe.
MDDwave,
I went to the site you posted concerning “How to talk to a Climate Skeptic”, I have also read the simplistic RealClimate.org tripe on solar influences on climate change, and many peer reviewed studies as well.
The posted article stated “…there has been no increase in solar irradiance since at least 1978, when satellite observations began.” How simplistic. It is true that solar irradiance, on average, has changed very little over that time span. During the average 11.3 year solar cycle solar irradiance in the visible spectrum varies about 0.2% and the overall change is even more insignificant (an AGW talking point). The fun part is that the cosmic ray intensity on the earths atmosphere changes by as much as 25% magnitude comparing solar minimum to maximum every cycle. See http://neutronm.bartol.udel.edu// and http://cosmicrays.oulu.fi/ for examples. Also note that cosmic ray intensity is particularly high during the current solar minimum.
According to Wikipedia:
“After the discovery of radioactivity by Henri Becquerel in 1896, it was generally believed that atmospheric electricity (ionization of the air) was caused only by radiation from radioactive elements in the ground or the radioactive gases (isotopes of radon) they produce. Measurements of ionization rates at increasing heights above the ground during the decade from 1900 to 1910 showed a decrease that could be explained as due to absorption of the ionizing radiation by the intervening air. Then, in 1912, Victor Hess carried three Wulf electrometers (a device to measure the rate of ion production inside a hermetically sealed container) to an altitude of 5300 meters in a free balloon flight. He found the ionization rate increased approximately fourfold over the rate at ground level. He concluded “The results of my observation are best explained by the assumption that a radiation of very great penetrating power enters our atmosphere from above.” In 1913-14, Werner Kolhörster confirmed Victor Hess’ results by measuring the increased ionization rate at an altitude of 9 km. Hess received the Nobel Prize in Physics in 1936 for his discovery of what came to be called “cosmic rays”.
Now that was an earned Nobel Prize. When I was in high school, we even had a cloud chamber in our physics class and observed cloud streaks caused by an alpha particle source.
Henrik Svensmark et al. have hypothesized that the solar variations that modulate the cosmic ray signal seen at the can affect cloud formation and hence climate. The mechanism will soon get quantitative testing as soon as the CERN super collider starts up in a year and commences the CLOUD experiments.
Do your homework MDDwave. The latest counter argument to Svensmark is Sloan et al. which looked at short duration bursts of solar activity (not the cosmic radiation itself) and found a slight affect. Imagine, it takes a long time for solar radiation to reach the edge of the heliosphere and then again the time for the moderated the cosmic rays to reach the earth. They were looking for a decrease in existing clouds instead of an increase in newly formed clouds. Give me a break. Sloan et al. said that they detected less than expected changes. Well of course, given the biases noted above. But most importantly they found a link.
As for the IPCC saying they determined that their GCMs predict CO2 forcing would be more than 12 times the solar forcing, they were referring to solar irradiance which as stated above varies only a fraction of a percent.
So there!
Your analysis is very thought provoking. Yet…
I’m fear that presenting data with supportable analysis will sway the AGW/ACC crowd about as much as presenting demonstrable genetic trends will sway ‘Intelligent Design’ proponents. It is no longer a question of science, it it ever really had been about science at all.
Why do I say this?
Noting that the temperatures are rising due to human fossil-fuel use was first noted in a study specifically commissioned by Ms. Thatcher in England to show this result, in order for her to use it as a weapon against striking coal miners.
It was ‘political interference with the scientific method’ at the outset, and there has not been any evidence that this has ever changed. Now, uniformly denying access to the data the IPCC used for its latest ‘report’ – even though these materials are to be fully accessible for at least 5 years according to IPCCs own rules, has further undermined the public confidence in the honesty of this debate. No matter how much ‘good science’ follows, the whole field will have been irrevocably tainted.
Now, more and more people are beginning to suspect that the scientific language was hijacked and the scientific method subverted in order to advance a specific dogma, which is not at all related to science – and never had been. I suspect that ‘science’ as a whole will never fully recover from this debacle.
Another blow to Hansen. Empirical reality is a hard concept for someone who has staked their whole life on a hypothesis.
That Hansen guy is nuts. msut be where Al Gore got his ideas form.
Oh– I agree linear regression isn’t the only way. But, it’s the one that can be “flat”.
I’m not familiar with Loess lines. But, if we do do linear regression, Giss is flat since 2001– which is the year I think most rational for comparison to the AR4. But that’s only 7 years.
How does this loess fit look if you go back to 1988? (As long as today is the aniversary and all.)
Leif,
Thanks for correcting me on your prediction for SC24 being 3/5 instead of 1/3 the strength of SC23. My bad. I appreciate and respect your work using the polar field precursor method. It is far more acceptable than any other methods.
However, models such as the Low-Frequency Modulation Model of Clilvered et al. that use strictly statistical analysis of past cycles, predict more of a Dalton type minimum for SC24 and SC25. I realize that past performance is no guarantee of future returns, which is what low-frequency analysis is all about, but then, we really don’t know what the sun will do.
I get excited overly excited on this stuff. Every time i see some Fool-aide drinker in a fish costume at a global summit of some sort, I just want to shake ’em and say WAKE UP. MDDwave and those like him will push us to international global socialism to stifle the industrialized nations and also prevent underdeveloped nations from industrializing. AGW is dangerous. Fuel from food is dangerous. Most of their remedies will doom Sub-Saharan Africans to being beasts of burden pedaling man-powered irrigation pumps supplied by carbon offset scams. Sub-Saharan Africans are using dried dung for cooking and heat instead of fertilizing their fields. Those solutions are racist. The underdeveped countries need electricity, and they need it in industrial amounts now. Underdeveloped countries have high birth rates because they are underdeveloped, which means women have 8 or nine children if the women don’t die during labor. It goes on and on.
Now goal of the AGW crowd is 350ppm CO2 in the atmosphere when we already have 385ppm.
I hope the sun busts them all, then we can go back to investing in climate change mitigation instead of thinking Science or Gaia is god and we are more powerful than the sun.
Joe (Diatribical Idiot),
It does sound that there isn’t much difference between your rolling trends and loess lines. As for giving the recent period more weight, that’s obviously true, but if you want to really give the more recent period more weight, try Hodrick-Prescott smoothing. 🙂
Basil
Basil,
I think you will find that it closely follows PDO. SSN not a factor since if you plot decadal smooth you will find that ssn will closely match ssn only because they are both decade in length. The kicker is that if both are in sequence or are they symbiosis. I hope this makes sense.