Much has been written about the Karl et al “pause buster” paper published this past summer, this essay suggests Karl et al actually shot themselves in the foot with the paper
Guest essay by Sheldon Walker
In this article we will:
1) look at an interesting new technique for analyzing global warming
2) use the new technique to analyze the time interval [January 1950 to December 1999]
3) use the results of 2) to show why Karl et al got it wrong, in their paper about “The Pause”
Most people are familiar with the use of linear regression in global warming.
Pick a start time, pick an end time, and calculate the slope of the regression line from the dates and temperature anomalies in the data series. What could possibly go wrong?
One of the common accusations made with global warming, is that the start time and/or end time were cherry-picked to give a particular result.
Accusations are also made that the length of the trend was too short to give a significant result (e.g. trends less than 10 years, or even trends less than 30 years).
What if there was a technique that we could use to get around these accusations?
To overcome the problem of cherry-picking, we use all possible start and end times from the time interval being investigated.
To overcome the problem of short trends, we only look at trends which are at least 10 years in length.
For example, imagine that we were interested in the time interval from [January 1975 to December 1999]. This interval is 24 years and 11 months in length. We divide [January 1975 to December 1999] up into EVERY possible trend of at least 10 years.
When I say EVERY possible trend of at least 10 years, I mean EVERY possible trend of at least 10 years. For the time interval [January 1975 to December 1999] there are 16,920 possible trends, and this method uses them ALL.
Example trends from [January 1975 to December 1999]:
====================================================
10 year trends: e.g. January 1975 to January 1985
10 year trends: e.g. February 1976 to February 1986
10 year and 1 month trends: e.g. February 1980 to March 1990
10 year and 2 month trends: e.g. January 1985 to March 1995
20 year trends: e.g. September 1979 to September 1999
20 year and 3 month trends: e.g. September 1979 to December 1999
24 year trends: e.g. June 1975 to June 1999
24 year and 5 month trends: e.g. May 1975 to October 1999
24 year and 11 month trends: e.g. January 1975 to December 1999 (the entire interval)
plus the other 16,911 trends.
This might appear to be overwhelming, but with Excel and a modern computer, it can be calculated quite easily.
There are several ways to present the results. The simplest way is to plot a “scatter” graph of the warming rate versus the trend length.
What does the graph of every possible combination of warming rate and trend length for [January 1975 to December 1999], look like? Have a look at Graph 1.
This graph holds a lot of valuable information, but it needs a little interpretation.
For example, how does the warming rate change with the trend length.
From the graph:
The warming rate for 10 year trends varies from -0.20 to +2.80 degC/century
The warming rate for 15 year trends varies from +0.65 to +2.20 degC/century
The warming rate for 20 year trends varies from +1.02 to +1.61 degC/century
The warming rate for 24 year and 11 month trends doesn’t vary at all, because there is only one, which is for the entire period, and it equals +1.71 degC/century
These results probably agree quite well with most people’s expectations. One lesson is, be wary of 10 years trends. You can get just about any warming rate that you want from a 10 year trend. Note than in certain circumstances a 10 year trend can be meaningful, but in general, 10 years trends are all over the place.
In general, warming rates become more stable with increasing trend length. But not always. Look at the warming rates for trend length = 22 years. There is a very small range of warming rates varying from +1.43 to +1.52 degC/century. But as the trend length increases to 23 years, the range of warming rates widens considerably. Why?
Also, after having a fairly stable warming rate of about +1.48 degC/century at trend length 22 years, the interval ends up with a warming rate of +1.71 degC/century for the entire interval. What made the warming rate suddenly increase by over 15%, as the trend length increased by just 3 years?
I am going to guess the answer to these 2 question, using the “scatter” graph, and a graph of the temperature anomalies over the interval. If you disagree with my quick guess then let me know what you think the answer is. At the start of the interval there is a La Nina type event from about 1975 to 1977. At the other end of the interval there is the large 1998 El Nino from about 1997 to 1999. As the trend length gets long enough to be influenced by both of these at the same time, the slope of the regression line is increased by the El Nino at one end, and also increased by the La Nina at the other end. So as the trend length exceeds 22 years, there is a double boost to the warming rate, which the “scatter” graph shows quite nicely.
Looking at the “scatter” graph for a single time interval, is only one possible use for this technique. Comparing the “scatter” graphs from different time intervals is another exciting possibility. It is this method that I will use to prove that Karl et al got it wrong in their paper about “The Pause” (“Possible artifacts of data biases in the recent global surface warming hiatus”).
To start, have look at Graph 2. This is similar to Graph 1, but shows every possible combination of warming rate and trend length for a different time interval, this time [January 1950 to December 1974]. This graph looks a bit like the one for [January 1975 to December 1999], but it is also a bit different.
To make it easier to compare these scatter graphs, I will put them onto the same graph. This means that one of the graphs hides some of the other graph, where they overlap. If necessary, this can be improved by plotting only the perimeters of each graph, but I am more interested in where the graphs don’t overlap at the moment, so we will ignore the overlap for now.
See Graph 3 – All combinations of warming rate and trend length that exist in the periods [1975 to 1999] and [1950 to 1974], for trends of at least 10 years.
Now it is easier to appreciate the differences between the 2 graphs. They are sort of similar in shape, but the green curve is translated down from the orange curve. Why is this? Looking at the warming rate for the entire interval for each graph gives the answer.
The orange curve has a 24 year and 11 month trend of +1.71 degC/century. A rate of global warming which is NOT low.
The green curve has a 24 year and 11 month trend of +0.28 degC/century. There is not much global warming in this interval.
Note how there is no overlap between the 2 graphs for trend lengths greater than about 15 years. This reinforces the idea that these 2 time intervals have very different warming rate profiles.
Now, the BIG question. If you add together these 2 periods, [1950 to 1974] and [1975 to 1999], and calculate the warming rate for the combined interval [1950 to 1999], what would the warming rate be? I have done this, and a linear regression over the combined interval has a warming rate of +1.12 degC/century. OK, but what does this value of +1.12 degC/century actually represent.
It is NOT the warming rate for normal anthropogenic global warming.
It is NOT the warming rate for when there is NO anthropogenic global warming.
It is an artificial average rate of warming, for an interval when anthropogenic global warming was present for about 1/2 the time, and absent for about 1/2 the time.
Unfortunately, Karl et al used this value as their “normal” anthropogenic warming rate, and based on this value, they concluded that the warming rate for [2000 to 2014] did NOT support the notion of a global warming “hiatus”.
Recapping quickly on the Karl et al paper:
Karl et al adjusted the NOAA data to account for the 0.12 degC average difference between buoy and ship SSTs. This “correction” had an impact on temperature trends, with the largest impact being on trends from 2000 to 2014 (which is where “The Pause” was meant to be).
So Karl et al calculated the new warming rates for [1950 to 1999] and [2000 to 2014]. They got:
Warming rate [1950 to 1999] = +1.13 degC/century
Warming rate [2000 to 2014] = +1.16 degC/century
Karl et al concluded that since the warming rate from [2000 to 2014] was virtually indistinguishable from the warming rate from [1950 to 1999], it does NOT support the notion of a global warming “hiatus”.
I am NOT questioning the adjustments that Karl et al made to Sea Surface Temperatures (SSTs). I am not qualified to dispute these adjustments, so I will use the adjusted NOAA data as it stands. Special note – I am using the NOAA data. If I find a “Pause” in the NOAA data, then they can not accuse me of using the wrong data.
I am also NOT disputing the calculation results from Karl et al. I get very similar results to theirs.
My issue is with the use of the Warming rate for [1950 to 1999]. Karl et al said this:
“Our new analysis now shows the trend over the period 1950-1999, a time widely agreed as having significant anthropogenic global warming (1), is 0.113°C dec−1, which is virtually indistinguishable with the trend over the period 2000-2014 (0.116°C dec−1).”
Now [1975 to 1999] is an interval having significant anthropogenic global warming.
But [1950 to 1974] is an interval having very little anthropogenic global warming.
By joining these 2 intervals together to form [1950 to 1999], Karl et al have created an interval that basically has half strength anthropogenic global warming (half with warming, and half without warming). But Karl et al used this value as their “normal” anthropogenic warming rate, when they compared it to [2000 to 2014].
If the warming rate for [2000 to 2014] matches the warming rate for [1950 to 1999] (which it does), then that means that [2000 to 2014] also has half strength anthropogenic global warming.
There are 2 simple ways to explain how [2000 to 2014] could have half strength anthropogenic global warming.
1) The period [2000 to 2014] could consist of 2 parts, one part which has anthropogenic global warming, and one part which does NOT have anthropogenic global warming (like [1950 to 1999]). But I do not think that this is the case.
2) The more reasonable explanation is that the period [2000 to 2014] has a lower warming rate than “normal” anthropogenic global warming. The warming rate would be about 50% of the “normal” warming rate. This could be called a “Slowdown”, a “Hiatus”, or a “Pause”. Whichever name you prefer, the data shows that it exists.
So Karl et al, while trying to convince everybody that there is NO Pause, have actually provided strong evidence that “The Pause” does exist (once their error concerning [1950 to 1999] is corrected).
==========
In part 2 of this article, I will analyse [2000 to 2015] using the new technique described in this article.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Sheldon,
Thank you for this novel (to me) and interesting way to analyse data.
You write “In general, warming rates become more stable with increasing trend length.”
This has been generally agreed for a long time, so it is good to move to quantifying the effect.
What does this mean for the BEST temperature/time series, whose basic rationale is the use the scalpel to create shorter and shorter intervals? Do you get into a state of more accuracy by homogenising after the scalpel, being offset by the greater variation of shorter intervals> Compromise point where?
Jeff Id some years ago at The Air Vent showed another effect of shortening data. He cot a trend into two parts and averaged them the recombined. The former rising trend became a staircase with ne step and a much lower averaged trend. Just searched for it, could not find it. It impacts quite a lot on the interpretation of the shape of your figures as graphed.
Geoff
I follow the argument but there is a huge assumption – that anthropogenic warming accounted for a large part of global warming for about half the interval. I am sorry, but claiming that the hiatus resulted in half the amount of man-made warming is not substantiated by the argument. I get it that the point is there was a slow down in warming – but there is no way to attribute this to natural or unnatural causes so you shouldn’t try.
Here is the FACT I cannot reconcile – estimated CO2 emissions have grown over 30% since 2000, while the RATE of temperature increase declined. Does that sound like CO2 drives global warming to you?
RoT, see upthread. This post does not assume Karl’s assumptions are correct. More potently, it shows that EVEN IF they were (not at all conceded), Karl’s conclusion is STILL wrong. Elegant
Checkmate.
Forget about mechanical trendsetters – interesting or not. . Put your brain to work and analyze what you see.This is how it was done before everyone got a computer.
Conquer one issue at a time.
That would not as directly refute Karl et all.
I would like to make two points in response to your excellent analysis.
1. the surface temperature is known to contain memory and persistence. this condition violates OLS assumptions. OLS trends are spurious under this condition and robust tests for trends are necessary.
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2689425
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2631298
2. Karl, Nieves, Hansen, Lacis, Mann, Trenberth and their camp have been very successful in limiting the debate to temperature whereas the real question in AGW is not whether it is warming but whether warming is related to fossil fuel emissions.
the only empirical evidence of this alleged relationship is a correlation between cumulative emissions and surface temperature (i.e. cumulative warming). this correlation is spurious.
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2725743
Indeed. Temperature is autoregressive ( current value is mainly determined by previous value, +/- a small change ). Much of the change is supposed, by climatologists, to be random or “stochastic” change. The autoregressive accumulation of random change is called a random walk.
Slopes in a random walk will happen all the times and on all time scales. Most correlations will be spurious.
While statistics can be useful refuting bad statistics (Karl), they do not seem useful in separating human from natural warming. For that, we must understand the constraints on what CO2 can possibly do, and subtract this from observed trends.
Though the author tried to show there is a pause but this is not right way of doing it. On one side we are telling the fact the data was adjusted and still we are showing a pause. In fact, prior to 1997/98 [volcanic activities] and after 1997/98 [El Nino activities] present a zero trend from 1979/80 with shift of 0.2 oC. The best way is to derive is a sine curve through iterative regression technique.
Dr. S. Jeevananda Reddy
It is interesting to realise that this graph is like 3D plot of running means on dT/dt.
A plotting all the points on a vertical cut through the graph would be a running mean filter of that period. The OLS slope provides an estimation of the average rate of change over that period. All the different dots are all the slopes as the X-year long window scans the data.
If each year-month was colour coded this would give 3D graph of running mean of dT/dt.
We can see that some running mean transects, like 14.5y, have large variability, while others have less. This is probably more to do with the how the distortions of running means interact with the data than anything more helpful.
Anyone not familiar with that distortion should read the following:
https://climategrog.wordpress.com/2013/05/19/triple-running-mean-filters/
+1.
The signal processing folks need to step up and start doing this work, not statisticians. IMHO almost all of standard statistics tools are wrong for studying an AR, non stationary, quasiperiodic signal, aka global temperature.
I must have missed it, but I couldn’t find anywhere that the source of the data was given. Was it original data, or “homogenized”? Ie, real or fake?
To refute the methodology of a paper, you use the same data, right or wrong.
The argument isn’t “Is this the right/true/unreasonably tampered with” data, but whether the methodology applied to it is correct.
the refugees to germany are wether desinfected neither high temperature treated, whatsoever.
that’s a REAL environmental problem for europa.
very interesting, but it would be even better if the warming of the 30’s would be added as then you compare the last warming with the first warming episode, and i would not be surprised to see them “overlap nearly perfectly” which totally would debunk the global warming theory….
Sheldon…Thanks for this. The graphic you came up with is inspired. As for whether it refutes Karl or not, I cannot say but it certainly provided me with a picture that will stick in my mind.
Now…if only “climate science” could come up with a reliable temperature record that 1) goes back far enough to be meaningful (where is that time machine when you need it?) and 2) actually covered the Earth, its oceans, and its atmosphere with enough direct measurements distributed uniformly, we might have something to support all of the half baked hypotheses we are constantly being exposed to.
Interesting technique and discussion. It seems that sampling all trends will necessarily over-represent any trend in the middle of the dataset. For example, when the minimum trend length exceeds half the dataset length, the middle portion of the dataset will be part of every trend plotted. How might this influence the interpretation?
Couldn’t you have skipped all the trend charts, and just pointed out that they used a double length time period to cheat?
Very clearly written and illustrated explanation of this novel way to compare warming rates in various periods for an educated but non expert like me. Bravo!
Question – how does the temperature data massaging weather bureaus have done affect this?
So, 25 years of ever more rapidly increasing atmospheric CO2 levels had ZERO effect on the “global warming” rate.
Cool. Time to stop worrying about CO2.
Sheldon,
This is the reference I was chasing in an earlier note here.
https://noconsensus.wordpress.com/2014/03/16/proxy-hammer/
If you have not read it, it could help because it shows a way in which your graphs are shaped the way they are.
In year 2014, Jeff Id from The Air Vent, plus Steve McIntyre from Climate Audit, with substantial input from regular CA blogger named “Roman”, a high-level statistician, worked on this matter together. It is all about segments, changes in trends after creating more segments in a time series.
I hope this helps.
Geoff
“But as the trend length increases to 23 years, the range of warming rates widens considerably. Why?”
I wondered about that too. But you can check that out using this gadget. I’ve set it up to show that max 23 year trend:
http://www.moyhu.org.s3.amazonaws.com/2016/2/23year.png
It runs from the red dot to the blue. And there is a peak at 1998, and a dip at 1976. 23 years spans this nicely and gets the benefit of both, for a big trend. But if you cut to 22 years, it can only contain one of these features, wherever you slide it to in the range. So the max trend is smaller.
It’s pretty easy to show what Karl et al. did to ‘bust’ the dreaded “Pause”. They simply lifted the global SST anomaly up en bloc by ~0.05K across the May-June 2006 interval and voilà! That’s all it took:
No “buoy/ship correction” argument justifies such a move …
They claim that they avoid cherry-picking by choosing to calculate every 10+ year trend in the dataset, and then they only focus on two specific trends they calculated, the ~25 year trends from 1950-1974 and 1975-1999, and use these two trends exclusively to argue that Karl et al. did not correctly identify a “normal” AGW warming rate.
You can plot 17,000 data points, but if you then only select two and build your entire case on it, the other 16,998 data points aren’t being used and you’re simply back to cherry-picking.
What the data actually shows is that 15-year trends within a 25-year dataset can vary such that a their prediction of the 25-year trend has a huge margin of error, something like +/- 25-50% of the 25-year trend, based on the plots provided. Which produces a conclusion: A sub-multi-decadal trend does a poor job of resolving the multidecadal trend. This is probably due to the competitive influence of natural climate variability on the trend over sub-multi-decadal periods, and is something we already know.
It also means that all of this bickering about whether there was/is or wasn’t/isn’t a hiatus over the last 15 years is a largely pointless exercise. It’s a sub-multi-decadal time period, so it doesn’t do a good job of resolving the multi-decadal (AGW) trend. So it doesn’t say very much about the multi-decadal (AGW) trend.
There’s no reason to be having this fight. It’s just pulling statistics out of noise and trying to argue that they represent signal.
https://www.reddit.com/r/skeptic/comments/47jxw6/a_new_way_of_looking_at_the_pause/d0dk344?context=3
I am trying to figure out what this graph is really showing. Is this something commonly used in statistics?