Image Credit: WoodForTrees.org
Guest Post By Werner Brozek, Edited By Just The Facts, Update/Additional Explanatory Commentary from Nick Stokes
UPDATE: RSS for October has just come out and the value was 0.207. As a result, RSS has now reached the 204 month or 17 year mark. The slope over the last 17 years is -0.000122111 per year.
—
The graphic above shows 5 lines. The long horizontal line shows that RSS is flat since November 1996 to September 2013, which is a period of 16 years and 11 months or 203 months. All three programs are unanimous on this point. The two lines that are sloped up and down and which are closer together include the error bars based on Nick Stokes’ Temperature Trend Viewer page. The two lines that are sloped up and down and which are further apart include the error bars based on SkS’s Temperature Trend Calculator. Nick Stokes’ program provides much tighter error bars and therefore his times for a 95% significance are less than that of SkS. In my previous post on August 25, I said: On six different data sets, there has been no statistically significant warming for between 18 and 23 years. That statement was based on the trend from the SkS page. However based on the trend from Nick Stokes’ page, there has been no statistically significant warming for between 16 and 20 years on several different data sets. In this post, I have used Nick Stokes’ numbers in section 2 as well as row 8 of the table below. Please let us know what you think of this change. I have asked that Nick Stokes join this thread to answer any questions pertaining to the different methods of calculating 95% significance and defend his chosen method. Nick’s trend methodology/page offers the numbers for Hadsst3 so I have also switched from Hadsst2 to Hadsst3. WFT offers numbers for Hadcrut3 but I can no longer offer error bars for that set since Nick’s program only does Hadcrut4.
In the future, I am not interested in using the trend methodology/page that offers the longest times. I am not interested in using trend methodology/page that offers the shortest times. And I am not interested in using trend methodology/page that offers the highest consensus. What I am interested in is using the trend methodology/page that offers that is the most accurate representation of Earth’s temperature trend. I thought it was SkS, but I may have been wrong. Please let us know in comments if you think that SkS or Nick Stokes’s methodology/page is more accurate, and if you can offer a more accurate one, please let us know that too.
According to NOAA’s State of the Climate In 2008 report:
The simulations rule out (at the 95% level) zero trends for intervals of 15 yr or more, suggesting that an observed absence of warming of this duration is needed to create a discrepancy with the expected present-day warming rate.
In this 2011 paper “Separating signal and noise in atmospheric temperature changes: The importance of timescale” Santer et al. found that:
Because of the pronounced effect of interannual noise on decadal trends, a multi-model ensemble of anthropogenically-forced simulations displays many 10-year periods with little warming. A single decade of observational TLT data is therefore inadequate for identifying a slowly evolving anthropogenic warming signal. Our results show that temperature records of at least 17 years in length are required for identifying human effects on global-mean tropospheric temperature.
In 2010 Phil Jones was asked by the BBC, “Do you agree that from 1995 to the present there has been no statistically-significant global warming?”, Phil Jones replied:
Yes, but only just. I also calculated the trend for the period 1995 to 2009. This trend (0.12C per decade) is positive, but not significant at the 95% significance level. The positive trend is quite close to the significance level. Achieving statistical significance in scientific terms is much more likely for longer periods, and much less likely for shorter periods.
I’ll leave it to you to draw your own conclusions based upon the data below.
Note: If you read my recent article RSS Flat For 200 Months (Now Includes July Data) and just wish to know what is new with the August and September data, you will find the most important new information from lines 7 to the end of the table. And as mentioned above, all lines for Hadsst3 are new.
In the sections below, we will present you with the latest facts. The information will be presented in three sections and an appendix. The first section will show for how long there has been no warming on several data sets. The second section will show for how long there has been no statistically significant warming on several data sets. The third section will show how 2013 to date compares with 2012 and the warmest years and months on record so far. The appendix will illustrate sections 1 and 2 in a different way. Graphs and a table will be used to illustrate the data.
Section 1
This analysis uses the latest month for which data is available on WoodForTrees.com (WFT). All of the data on WFT is also available at the specific sources as outlined below. We start with the present date and go to the furthest month in the past where the slope is a least slightly negative. So if the slope from September is 4 x 10^-4 but it is – 4 x 10^-4 from October, we give the time from October so no one can accuse us of being less than honest if we say the slope is flat from a certain month.
On all data sets below, the different times for a slope that is at least very slightly negative ranges from 8 years and 9 months to 16 years and 11 months.
1. For GISS, the slope is flat since September 1, 2001 or 12 years, 1 month. (goes to September 30, 2013)
2. For Hadcrut3, the slope is flat since May 1997 or 16 years, 5 months. (goes to September)
3. For a combination of GISS, Hadcrut3, UAH and RSS, the slope is flat since December 2000 or 12 years, 10 months. (goes to September)
4. For Hadcrut4, the slope is flat since December 2000 or 12 years, 10 months. (goes to September)
5. For Hadsst3, the slope is flat since November 2000 or 12 years, 11 months. (goes to September)
6. For UAH, the slope is flat since January 2005 or 8 years, 9 months. (goes to September using version 5.5)
7. For RSS, the slope is flat since November 1996 or 17 years (goes to October)
RSS is 203/204 or 99.5% of the way to Ben Santer’s 17 years.
The next link shows just the lines to illustrate the above for what can be shown. Think of it as a sideways bar graph where the lengths of the lines indicate the relative times where the slope is 0. In addition, the sloped wiggly line shows how CO2 has increased over this period.
When two things are plotted as I have done, the left only shows a temperature anomaly.
The actual numbers are meaningless since all slopes are essentially zero and the position of each line is merely a reflection of the base period from which anomalies are taken for each set. No numbers are given for CO2. Some have asked that the log of the concentration of CO2 be plotted. However WFT does not give this option. The upward sloping CO2 line only shows that while CO2 has been going up over the last 17 years, the temperatures have been flat for varying periods on various data sets.
The next graph shows the above, but this time, the actual plotted points are shown along with the slope lines and the CO2 is omitted:
Section 2
For this analysis, data was retrieved from Nick Stokes moyhu.blogspot.com. This analysis indicates for how long there has not been statistically significant warming according to Nick’s criteria. Data go to their latest update for each set. In every case, note that the lower error bar is negative so a slope of 0 cannot be ruled out from the month indicated.
On several different data sets, there has been no statistically significant warming for between 16 and 20 years.
The details for several sets are below.
For UAH: Since November 1995: CI from -0.001 to 2.501
For RSS: Since December 1992: CI from -0.005 to 1.968
For Hadcrut4: Since August 1996: CI from -0.006 to 1.358
For Hadsst3: Since May 1993: CI from -0.002 to 1.768
For GISS: Since August 1997: CI from -0.030 to 1.326
Section 3
This section shows data about 2013 and other information in the form of a table. The table shows the six data sources along the top and other places so they should be visible at all times. The sources are UAH, RSS, Hadcrut4, Hadcrut3, Hadsst3, and GISS. Down the column, are the following:
1. 12ra: This is the final ranking for 2012 on each data set.
2. 12a: Here I give the average anomaly for 2012.
3. year: This indicates the warmest year on record so far for that particular data set. Note that two of the data sets have 2010 as the warmest year and four have 1998 as the warmest year.
4. ano: This is the average of the monthly anomalies of the warmest year just above.
5. mon: This is the month where that particular data set showed the highest anomaly. The months are identified by the first three letters of the month and the last two numbers of the year.
6. ano: This is the anomaly of the month just above.
7. y/m: This is the longest period of time where the slope is not positive given in years/months. So 16/2 means that for 16 years and 2 months the slope is essentially 0.
8. sig: This the first month for which warming is not statistically significant according to Nick’s criteria. The first three letters of the month is followed by the last two numbers of the year.
9. Jan: This is the January, 2013, anomaly for that particular data set.
10. Feb: This is the February, 2013, anomaly for that particular data set, etc.
21. ave: This is the average anomaly of all months to date taken by adding all numbers and dividing by the number of months. However if the data set itself gives that average, I may use their number. Sometimes the number in the third decimal place differs by one, presumably due to all months not having the same number of days.
22. rnk: This is the rank that each particular data set would have if the anomaly above were to remain that way for the rest of the year. It may not, but think of it as an update 45 minutes into a game. Due to different base periods, the rank is more meaningful than the average anomaly.
| Source | UAH | RSS | Had4 | Had3 | Sst3 | GISS |
|---|---|---|---|---|---|---|
| 1. 12ra | 9th | 11th | 9th | 10th | 9th | 9th |
| 2. 12a | 0.161 | 0.192 | 0.448 | 0.406 | 0.346 | 0.58 |
| 3. year | 1998 | 1998 | 2010 | 1998 | 1998 | 2010 |
| 4. ano | 0.419 | 0.55 | 0.547 | 0.548 | 0.416 | 0.67 |
| 5. mon | Apr98 | Apr98 | Jan07 | Feb98 | Jul98 | Jan07 |
| 6. ano | 0.66 | 0.857 | 0.829 | 0.756 | 0.526 | 0.94 |
| 7. y/m | 8/9 | 16/11 | 12/10 | 16/5 | 12/11 | 12/1 |
| 8. sig | Nov95 | Dec92 | Aug96 | May93 | Aug97 | |
| Source | UAH | RSS | Had4 | Had3 | Sst3 | GISS |
| 9. Jan | 0.504 | 0.440 | 0.450 | 0.390 | 0.292 | 0.63 |
| 10.Feb | 0.175 | 0.194 | 0.479 | 0.424 | 0.309 | 0.51 |
| 11.Mar | 0.183 | 0.204 | 0.405 | 0.384 | 0.287 | 0.60 |
| 12.Apr | 0.103 | 0.218 | 0.427 | 0.400 | 0.364 | 0.48 |
| 13.May | 0.077 | 0.139 | 0.498 | 0.472 | 0.382 | 0.57 |
| 14.Jun | 0.269 | 0.291 | 0.457 | 0.426 | 0.314 | 0.61 |
| 15.Jul | 0.118 | 0.222 | 0.514 | 0.488 | 0.479 | 0.54 |
| 16.Aug | 0.122 | 0.167 | 0.527 | 0.491 | 0.483 | 0.61 |
| 17.Sep | 0.297 | 0.257 | 0.534 | 0.516 | 0.455 | 0.74 |
| Source | UAH | RSS | Had4 | Had3 | Sst3 | GISS |
| 21.ave | 0.205 | 0.237 | 0.474 | 0.444 | 0.374 | 0.588 |
| 22.rnk | 6th | 8th | 9th | 7th | 6th | 9th |
If you wish to verify all of the latest anomalies, go to the following links, For UAH, version 5.5 was used since that is what WFT used, RSS, Hadcrut4, Hadcrut3, Hadsst3,and GISS
To see all points since January 2013 in the form of a graph, see the WFT graph below:
Appendix
In this section, we summarize data for each set separately.
RSS
The slope is flat since November 1996 or 16 years and 11 months. (goes to September) RSS is 203/204 or 99.5% of the way to Ben Santer’s 17 years.
For RSS: There is no statistically significant warming since December 1992: CI from -0.005 to 1.968
The RSS average anomaly so far for 2013 is 0.237. This would rank 8th if it stayed this way. 1998 was the warmest at 0.55. The highest ever monthly anomaly was in April of 1998 when it reached 0.857. The anomaly in 2012 was 0.192 and it came in 11th.
UAH
The slope is flat since January 2005 or 8 years, 9 months. (goes to September using version 5.5)
For UAH: There is no statistically significant warming since November 1995: CI from -0.001 to 2.501
The UAH average anomaly so far for 2013 is 0.205. This would rank 6th if it stayed this way. 1998 was the warmest at 0.419. The highest ever monthly anomaly was in April of 1998 when it reached 0.66. The anomaly in 2012 was 0.161 and it came in 9th.
Hadcrut4
The slope is flat since December 2000 or 12 years, 10 months. (goes to September)
For HadCRUT4: There is no statistically significant warming since August 1996: CI from -0.006 to 1.358
The Hadcrut4 average anomaly so far for 2013 is 0.474. This would rank 9th if it stayed this way. 2010 was the warmest at 0.547. The highest ever monthly anomaly was in January of 2007 when it reached 0.829. The anomaly in 2012 was 0.448 and it came in 9th.
Hadcrut3
The slope is flat since May 1997 or 16 years, 5 months (goes to September, 2013)
The Hadcrut3 average anomaly so far for 2013 is 0.444. This would rank 7th if it stayed this way. 1998 was the warmest at 0.548. The highest ever monthly anomaly was in February of 1998 when it reached 0.756. One has to go back to the 1940s to find the previous time that a Hadcrut3 record was not beaten in 10 years or less. The anomaly in 2012 was 0.406 and it came in 10th.
Hadsst3
For Hadsst3, the slope is flat since November 2000 or 12 years, 11 months. (goes to September, 2013).
For Hadsst3: There is no statistically significant warming since May 1993: CI from -0.002 to 1.768
The Hadsst3 average anomaly so far for 2013 is 0.374. This would rank 6th if it stayed this way. 1998 was the warmest at 0.416. The highest ever monthly anomaly was in July of 1998 when it reached 0.526. The anomaly in 2012 was 0.346 and it came in 9th.
GISS
The slope is flat since September 1, 2001 or 12 years, 1 month. (goes to September 30, 2013)
For GISS: There is no statistically significant warming since August 1997: CI from -0.030 to 1.326
The GISS average anomaly so far for 2013 is 0.588. This would rank 9th if it stayed this way. 2010 was the warmest at 0.67. The highest ever monthly anomaly was in January of 2007 when it reached 0.94. The anomaly in 2012 was 0.58 and it came in 9th.
Conclusion
It appears as if we can accurately say from what point in time the slope is zero or any other value. However the period where warming is statistically significant seems to be more of a challenge. Different programs give different results. However what I found really surprising was that according to Nick’s program, GISS shows significant warming at over 95% for the months of November 1996 to July 1997 inclusive. However during those nine months, the slope for RSS is not even positive! Can we trust both data sets?
———-
Update: Additional Explanatory Commentary from Nick Stokes
Trends and errors:
A trend coefficient is just a weighted average of a time series, which describes the rate of increase. You can calculate it without any particular statistical model in mind.
If you want to quantify the uncertainty you have about it, you need to be clear what kind of variations you have in mind. You might want to describe the uncertainty of actual measurement. You might want to quantify the spatial variability. Or you might want to say how typical that trend is given time variability. In other words, what if the weather had been different?
It’s that last variability that we’re talking about here, and we need a model for the variation. In all kinds of time series analysis, ARIMA models are a staple. No-one seriously believes that their data really is a linear trend with AR(1) fluctuations, or whatever, but you try to get the nearest fitting model to estimate the trend uncertainty.
In my trend viewer, I used AR(1). It’s conventional, because it allows for autocorrelating based on a single delay coefficient, and there is a widely used approximation (Quenouille). I’ve described here how you can plot the autocorrelation function to show what is being fitted. The uncertainty of the trend is proportional to the area under the fitted ACF. Foster and Rahmstorf argued, reasonably, that the AR(1) underfits, and a ARMA(1,1) approx does better. Here is an example from my post. SkS uses that approach, following F&R.
You can see from the ACF that it’s really more complicated, The real ACF does not taper exponentially – it oscillates, with a period of about 4 years – likely ENSO related. Some of that effect reaches back near zero, where the ARIMA fitting is done. If it is taken out, the peak would be more slender that AR(1). But there is uncertainty with ENSO too.
So the message is, trend uncertainty is complicated.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Nick Stokes:
I write to ask for a clarification.
Nearly half a century has passed since I used to conduct regression analyses by hand (there were no electronic calculators in those days) but I still possess the ink pens and templates I used to plot graphs of the resulting regression results. Clearly, methods may have changed recently, but I would welcome knowledge of how regression analyses of different time series data sets with different lengths can provide confidence limits of their trends which all radiate from the same single point on a graph.
Please explain how you calculated the confidence limits for each calculated linear trend and what do the plotted confidence limits indicate.
I ask because the plotted values of confidence limits seem to make no sense. There are three confidence bounds which can be calculated from the result of a linear regression of a data series; viz.
1. The confidence limits of the data set.
And
2. The confidence limits of the trend line.
And
3. The confidence limits of the regression.
Simplistically, the confidence limits of the data set is an ‘average’ of the confidence limits for each datum in the set, and results in error bars for the data set. These error bars are two lines which parallel the trend line, and they show the band within which the data can probably be found; e.g. if they are 95% confidence limits for the data set then one in twenty of the data points is probably outside the lines.
Clearly, the confidence limits shown above are not the confidence limits of the data set. They are not a pair of lines which parallel the trend line of each data set.
The confidence limits of the trend line is the range of values within which the slope of the trend is calculated to exist, and they are error bars which have the form of straight lines which intersect with the centre of the trend line. The trend line can be rotated around the central point of the trend line to any value within those limits, and they show the range within which the slope of the trend line can probably be found; e.g. if they are 95% confidence limits then there is a 20:1 probability that the slope of the trend line is within those limits.
Clearly, the confidence limits shown above are not the confidence limits of the trend line. They are not a pair of lines which intersect the centre of the trend line of each data set. Indeed, they all intersect at the same place on the graph for all the data sets which have different lengths.
The confidence limits of the regression is the range of values within which the data and the slope of its linear trend are calculated to exist. They have the form of two curves (one on each side of the trend line) and they are each closest to the trend line at the centre of the trend line. They are error bars which combine the confidence limits of the data set and the trend line; e.g. if they are 95% confidence limits then there is a 20:1 probability that the slope of the trend line is within those limits, and there is a one in twenty of the data points will fall within those limits for any trend line which is within those limits.
Clearly, the confidence limits shown above are not the confidence limits of the regression. They are not a pair of curves for each data set.
So, I write to ask what your calculated confidence limits indicate and how they were calculated.
Richard
Ooops!
I wrote
e.g. if they are 95% confidence limits then there is a 20:1 probability that the slope of the trend line is within those limits, and there is a one in twenty of the data points will fall within those limits for any trend line which is within those limits.
I intended to write
e.g. if they are 95% confidence limits then there is a 20:1 probability that the slope of the trend line is within those limits, and there is a one in twenty CHANCE ALL the data points will fall within those limits for any trend line which is within those limits.
Sorry, Richard
Nick Stokes says:
November 3, 2013 at 4:30 pm
Joe says: November 3, 2013 at 4:15 pm
“I know I can’t really drive through rush houor at the speed limit, but, for convenience, I’m going to model my commute as if I can”
Oddly, I was going to use that analogy too. Not speed limit, but suppose you work out an average speed for your daily commute. That doesn’t mean that you expect to drive at that speed uniformly; in fact it doesn’t imply any speed model at all. But suppose you then plan to arrive at work reliably on time. You need a model of variability. It doesn’t need to perfectly predict how your journey will go, but it needs to give average variation.
Most people in that situation do do something like that. And it mostly works.
———————————————————————————————————————
But it doesn’t work if you choose a model (such as “drive at the speed limit”) which you know doesn’t reflect reality and we know that simple linear trends don’t reflect any part of the real climate system!
So, while such “simplifications” may be convenient, and may appear to work occasionally, any such appearance is happenstance and can’t be relied on to give any indication at all of the future.
richardscourtney says: November 4, 2013 at 2:33 am
“Nick Stokes:
I write to ask for a clarification.”
Well, it isn’t my graph. But as I understand, the slopes of the lines are equal to the 95% CI values, computed by my method (AR(1)) and the SkS values. The central line, of course, has the slope of the actual trend. They aren’t themselves fitted lines; the purpose is, I believe, simply to show how the CI extremes look against the actual results.
As I said above, finding that the trend over a period is not significantly different from zero doesn’t really tell you very much. It could still be quite large, and might indeed be consistent with AGW predictions, without being significantly different from zero. It depends on the noise. This plot shows the limits of slopes that could be in the range.
Joe says: November 4, 2013 at 2:51 am
“we know that simple linear trends don’t reflect any part of the real climate system!”
Well, we know that travelling at constant speed doesn’t reflect any part of city driving. But the idea of average speed is still useful, and it’s how we (and electronic navigators) plan journeys.
A trend is an average rate of change.
davidmhoffer says:
November 3, 2013 at 9:03 pm
Thanks, David. Your analogy is perfect!
Nick Stokes:
Thankyou for your post at November 4, 2013 at 3:13 am in response to my request for clarification at November 4, 2013 at 2:33 am (and its corrigendum at November 4, 2013 at 2:40 am).
My post explained what I failed to understand about the plotted lines supplied by you in the above graph and why I failed to understand. And I asked
Unfortunately, your response leaves me uninformed as to the answers to my requests for explanation. It says
Sorry, but I am told nothing by your statement that “the purpose is, I believe, simply to show how the CI extremes look against the actual results”. I think it says that the plotted confidence limits are the 95% confidence limits of the trend lines but each displaced to the same and wrong position. Indeed, my interpretation seems to be supported by your saying, “This plot shows the limits of slopes that could be in the range”. But I could be wrong in that interpretation.
And I did see your linked comment above; i.e.
http://wattsupwiththat.com/2013/11/03/statistical-significances-how-long-is-the-pause-now-includes-september-data/#comment-1465406
However, frankly, I fail to see how that addresses my request. So, to be clear, I repeat it.
Please explain how you calculated the confidence limits for each calculated linear trend and what do the plotted confidence limits indicate.
Richard
richardscourtney says: November 4, 2013 at 3:56 am
“Please explain how you calculated the confidence limits for each calculated linear trend and what do the plotted confidence limits indicate.”
Again, it’s Werner’s plot and he calculated the numbers as he indicated. I can tell you how my gadget that he used gets the CI’s. It is explained in this original post, with further commentary here and in linked plots. The standard regression assumption is that the residuals are random variables, which may be correlated. In AR(1), a linear expression can be written which is a linear combination that is expected to be an iid random variable. Its variance is estimated from the sum of squares of residuals, and that scales the CI’s.
In this post I have compared some of my calculations, using the Quenouille approximation, with the results of the R ar() routine. It’s in the table near the end.
Nick Stokes:
Many thanks for your post at November 4, 2013 at 4:16 am in response to my request for clarification. Especial thanks for this link
http://www.moyhu.blogspot.com.au/2011/11/picture-of-statistically-significant.html
which provides your calculation method.
Firstly, as I understand the calculation method in that link it ignores autocorrelation and assumes the residuals are independent (although you admit this is not true) then calculates the standard error of the trend at each point along the time series.
If I have understood you correctly then that method can be disputed on several grounds, and I am grateful for your links which would assist such dispute. However, I am avoiding such a dispute because my request for clarification is an attempt to understand what is presented in the above graph: my request is NOT an attempt to side-track the thread into discussion of the validity of your method.
I fail to understand how your method provides 95% confidence limits for each of the data sets which consist of lines which radiate from the same point as is presented in the above graph. Indeed, at the intersect of those lines the confidence limits differ from each trend by zero (which is impossible).
I accept that – as you say – “it’s Werner’s plot and he calculated the numbers as he indicated”, but it is your method and you are not objecting to how Werner has presented the results of his calculations. Clearly, you are the person most familiar with your method and its use which is why I am asking you for clarification.
You have kindly explained your method for calculation of 95% confidence limits of the trend. But, if my understanding of your method is correct, then my failure to understand the graph is increased.
I asked
And you have answered my request for clarification of “how you calculated the confidence limits for each calculated linear trend” by referencing your method and saying that Werner used it. Importantly, you have not disputed how Werner used your method and presented its results so I understand you to agree that the above graph represents results which you agree. Hence, I am asking you what the graph shows because it is the results of your method which are being presented.
Hence, I again ask with specific reference to the above graph,
what do the plotted confidence limits indicate?
Please note that I am grateful for your responses so far and I am choosing to not debate the validity of your method. I am trying to understand what is indicated by the results of your method which are displayed in the above graph.
Richard
richardscourtney says: November 4, 2013 at 4:57 am
“I am trying to understand what is indicated by the results of your method which are displayed in the above graph.”
If, as Werner did, you use my gadget as linked above, either by clicking on the triangle or adjusting the red and blue disks until the time range is right, then it will report that the trend from Nov ’96 to Sep ’13 is -0.005°C/century and the CI’s of that are -1.274 to 1.264°C/century. That is the extent of my contribution to the plot, and the numbers are calculated as in the linked post.
Werner has plotted lines with slopes corresponding to those extreme values, as well as the central value. The numbers shown on the plot are actually the product of the slope with the time interval (that’s what WFT uses in the detrend facility).
What it means is that, although the observed trend was zero, the vagaries of weather could have produced, in the same circumstances, a trend of anything from -1.274 to 1.264. That is, if the random weather could be re-run, the trend would be within that range with 95% probability.
As I said above, this is not the normal logic of statistical testing. Orthodox would be to test whether, given a hypothesis of say warming at 1.2 °/cen, an observation of -0.005 is within the bounds of reasonable (95%) probability. If not, you can reject the hypothesis.
However, that assumes that you had no prior knowledge. If you are testing it because you already suspect it was a “pause”, then the test doesn’t work. The reason is that 1 in 20 events do occur, and if you look where you know there is a better chance of finding them, then they won’t be 1/20 any more.
No-one seriously believes that their data really is a linear trend with AR(1) fluctuations, … In my trend viewer, I used AR(1).
=============
that pretty much sums up climate science. use a model that everyone agrees doesn’t match the characteristics of the underlying data, and then place your faith in the results of the model.
linear approximations deliver nonsense when applied to nature, because nature is inherently cyclical with scale independent variability. This causes linear methods to find trends where no trends exist, and to significantly underestimate natural variability.
fit a linear trend to spring-time temperatures and what does it tell you about the coming winter? that the winter will be warmer than the spring. climate science 101.
fit a linear tend to summer-time temperatures, and you will find a “pause” in the spring-time warming. but climate science is confident that by the fall warming will reappear.
Greg Goodman says:
November 4, 2013 at 2:29 am
Good informative post but the way you are representing CO2 is totally misleading. You are deliberately scaling and offsetting so that the slope fills the graph range. You could do that with absolutely anything that has a positive trend and it would not show a damn thing about its relation to surface temps.
There are different ways of looking at this. According to Nye, the CO2 is rising “extraordinarily fast” as shown by the link below. And if you go by percentages for CO2 versus temperature since 1750, there is no comparison. CO2 went up from 280 ppm to 400 ppm or an increase of 43%. Meanwhile, temperatures went up from about 287.0 K to 287.8 K or an increase of 0.0028%. You raise good points, and while no representation is perfect, in view of the percentages above, I do not agree that my way is “totally misleading”. As a direct result of Dr. Brown’s comments on my last post, I added the following comment to this post:
“The upward sloping CO2 line only shows that while CO2 has been going up over the last 17 years, the temperatures have been flat for varying periods on various data sets.”
NYE: So here’s the point, is it’s rising extraordinarily fast. That’s the difference between the bad old days and now is it’s —
MORANO: Carbon dioxide —
NYE: It’s much faster than ever in history. And so —
Read more: http://newsbusters.org/blogs/noel-sheppard/2012/12/04/climate-realist-marc-morano-debates-bill-nye-science-guy-global-warmi#ixzz2jgCV7ZVs
Nick Stokes:
Many thanks indeed for your post at November 4, 2013 at 5:29 am in response to my request for clarification. That does address what I was trying to determine. Thankyou.
I write to feed back what I understand you to be saying because, as I have repeatedly said, originally in my first post in this thread, at November 4, 2013 at 2:33 am,
As I understand it the problem is an error of positioning provided by your gadget which Werner carried over in his plot of the 95% confidence limits. I explain this understanding as follows.
In my first post in this thread, at November 4, 2013 at 2:33 am, I also wrote
But your gadget – in effect – provides those limits from one end of the assessed trend line. This is an error because, although the numerical value of the confidence limits is the same, it completely alters the possible range of trend lines which can be plotted on a graph.
Importantly, one end of the considered period is assumed to be a fixed and the possible trend lines rotate around that point. But the centre of the considered period should be fixed on a graph. Fixing the trend at one end of the period reduces the estimated error to zero at that end and doubles the estimated error at the other end as plotted on the graph.
In the case of the above graph one end point is assumed to be the same for all the analysed time series. And that increases the distortion as presented in the graph. It also has serious effect on a calculation of the confidence limits of the regression.
As I also said in my first post (with corrigendum)
Please explain and refute any misunderstanding which I have stated in this post.
Richard
To see why linear methods underestimate natural variability, consider for a moment the annual cycle of temperatures at some location. plot this on a piece of paper, you get the familiar sin/cos curve from high school math class.
now consider that “the pause” is summertime temperatures. fit the linear approximation and you will get the flat line in the opening graph. Assume that the variability is due to noise, and you will get the “V” shaped lines opening up to the right. This is the 95% projection of where your future temperatures must be.
But what you have is nonsense. Look ahead 6 months and the wintertime temperatures will be lower than the lower bound for your 95% projection, and the upper bound for the 95% projection is so high as to be ridiculous.
The problem is the underlying assumption. the assumption that your data (climate) is linear with noise (weather) is wrong. weather is the result of daily and annual cycles in nature. climate is a similar process, with time scales measured in hundreds, thousands and millions of years.
Our mistake is in seeing climate as the average of weather. It isn’t. We measure climate as the average of weather, so our minds create the illusion that climate is the average of weather, but nature isn’t controlled by our measuring process.
Weather is the chaotic response of nature to short term cycles. Climate is the chaotic response of nature to longer term cycles. Both are inherently unpredictable given our current level of understanding, except to the degree that cyclical systems tend to repeat.
ferd berple:
You make a very good point in your post at November 4, 2013 at 5:39 am. This link jumps to it to help people who may have missed it
http://wattsupwiththat.com/2013/11/03/statistical-significances-how-long-is-the-pause-now-includes-september-data/#comment-1465650
The problem is compounded by so-called ;climate scientists’ having modern computer power and computational packages available to them. They use – as you say – incorrect models which they ‘pull off the shelf’ and amend to suite, but have no real idea of what the calculation does.
Richard
imagine for a moment that you were an extremely long lived creature, that years passed as seconds. each day of your life would be 32 million years long, and your 80 year lifespan would cover about 2.5 billion years.
daily weather would be imperceptible. annual weather would be a 1 second pulse. 100 thousand year ice ages would last slightly more than 1 day. These 100 thousand year cycles would show a warming and cooling cycle very much like we experience day to day. Over time, some “days” would be warmer and some would be cooler – very much like our daily weather. As time scales increase we see hot house earth – summer and we see ice house earth – winter. We see some “months” that are wet and some that are dry – long term climate change.
But in all of this, there is nothing to suggest that climate is the average of weather. Rather, that weather and climate are the same process, differing only in time scales. We create the illusion that climate is the average of weather over time as a result of our measuring process. But this is not the reality.
wbrozek says:
November 3, 2013 at 10:31 pm
“However I use GISS and September for GISS was at a record high for September that it shared with 2005, namely 0.74.”
Werner, how long do you think the data gatekeepers are going to tolerate this flat-lining? Compare Hadcrut 3 with hadcrut 4 for example. Trust me, there is going to be a hadcrut 5 under all this pressure and GISS is fiddling with the whole data string on a continuing basis. I suppose the satellite data is open to fiddling, too, although it is in honest hands for the present. Notice that, with arctic ice extent increasing at the fastest rate in decades, that there is a big pause in the data stream from US sites – they’ll be shifting the measure to 25% ice from 15% as the measure and giving us a big explanation as to why. Remember what happened when sea level measurements were beginning to flatten? They started fiddling that too with a change for crustal rebound and the actual sea level metric is no more. Its kinda, sorta like ocean basin volume measurements. The only reason we have a “pause” (a loaded word indeed) is it sneaked up on them and now the whole world has its eyes on it.
The mathematical definition of climate as the 30 year average of weather has created an illusion of climate predictability that does not match reality. Because we define climate as the average of weather, we apply all sorts of statistical properties to climate based on our understanding of averages.
For example, we assume that climate can never “naturally” get hotter than the hottest day in summer, or colder than the coldest day in winter. This is how average work. So, when we see climate starting to get hotter or colder than past weather, we start to think something “unnatural” is happening. That humans must be changing the climate.
What we fail to consider is that the formal definition of climate as the 30 year average of weather is a human definition. It is not nature’s definition of climate and nature is not in the slightest bound by our definition.
Our formal definition of climate as the 30 year average of weather is fundamentally wrong. It has led climate science down a path of statistical prediction based on mathematical properties that do not match reality. It has caused us to significantly under-estimate natural variability and to seriously over-estimate our confidence levels in our predictions.
Climate is not the average of weather over time, and thus cannot be reliably modeled using weather averages.
Gary Pearse says:
November 4, 2013 at 6:52 am
Trust me, there is going to be a hadcrut 5
Thank you for your excellent points. But who needs Hadcrut5 when they can just fix Hadcrut4 which they have already done? See:
http://wattsupwiththat.com/2013/05/12/met-office-hadley-centre-and-climatic-research-unit-hadcrut4-and-crutem4-temperature-data-sets-adjustedcorrectedupdated-can-you-guess-the-impact/
ferd berple says: November 4, 2013 at 5:39 am
“No-one seriously believes that their data really is a linear trend with AR(1) fluctuations, … In my trend viewer, I used AR(1).
=============
that pretty much sums up climate science. use a model that everyone agrees doesn’t match the characteristics of the underlying data, and then place your faith in the results of the model.”
It isn’t climate science – it is standard statistics, used by all sorts of people. In fact, these are Box-Jenkins models. You know, the Box who famously said – “all models are wrong, but some are useful”. These are the models he was talking about.
The starting place for climate models is not with the annual temperature data. That is no different than trying to predict the stock market based on the Dow. You are trying to fit parameters (CO2, aerosols, sunspots, TSI, etc) to essentially random data. You end up with schizophrenic models. They deliver super rational answers that latter prove to be crazy.
If you want to know about the market in the short term, you need to know what it looks like in the long term. Otherwise you will miss the pattern. The low long-term rise punctuated with short, steep drops. The novice investor, seeing the long slow rise, predicts that it is now safe to enter the market. Only to be burned by the short, steep drop. Having now lost their money, they leave the market to avoid further loss, while the market responds with a long, slow rise.
The starting place for climate models is the long term climate data, stretching back over thousands and millions of years. When your climate model can accurately capture the ice ages and interglacial, with the rapid pulses of warming and cooling interspersed, they may have some validity in telling us what to expect in the future.
To try and model climate based on thermometer data is no different than stock market chartists trying to predict tomorrows stock prices based on yesterday’s results. If you do everything perfectly, your predictions will be almost, but not quite as good as those of a dart board.
Nick Stokes says:
November 4, 2013 at 7:41 am
Box who famously said – “all models are wrong, but some are useful”.
==============
“some are useful” means “most are useless”. The IPCC models clearly demonstrate this.
richardscourtney says: November 4, 2013 at 6:15 am
“As I understand it the problem is an error of positioning provided by your gadget which Werner carried over in his plot of the 95% confidence limits.”
My gadget does no positioning. It simply calculates a trend from a time series. This is a statistic with standard error and confidence intervals. This is absolutely standard. The Excel function LINEST() returns a regression β with standard error se2. Excel does not allow for autocorrelation; I do. Here is Hu McCulloch explaining why. I use the Quenouille adjustment that he describes.
I think Werner’s choice to illustrate the effect of trend uncertainty by showing how the various trends would diverge from a single point is perfectly reasonable. But if you don’t like it, you should take it up with him.
Nick Stokes:
Thankyou for your post at November 4, 2013 at 8:27 am which agrees my understanding stated in my post at November 4, 2013 at 6:15 am but with this exception: you say to me
It is not “perfectly reasonable” for the reasons I explained in my post.
As I said, I asked for clarification from you (and you have kindly provided it) because Werner used your method so he would reasonably have responded that any clarification be obtained from you. The exception you have stated is Werner’s plot of the results he obtained from your method and you say you agree with that plot. As you say, in this circumstance it is reasonable for you to pass this issue to Werner.
I assume Werner is following this thread and, therefore, it seems reasonable for me to assume he has understood the matter because he has not queried it.
I again thank you for the clarifications you have provided.
Richard
richardscourtney says:
November 4, 2013 at 9:27 am
I assume Werner is following this thread and, therefore, it seems reasonable for me to assume he has understood the matter because he has not queried it.
Yes, I am following the thread although I will not claim to understand all of the technicalities. I agree with what you said earlier that: “They have the form of two curves (one on each side of the trend line) and they are each closest to the trend line at the centre of the trend line.” at richardscourtney says:
November 4, 2013 at 2:33 am
Those two curves would make more sense that what I have shown. However I do not know how to make those curves with the information given.
I am not sure if it is too relevant here, but Ross McKitrick once said something to the effect that you can only tell if a change is significant, but you cannot tell if a straight line is significant. Perhaps I did the wrong thing with the five lines in the top graph.
For RSS, Nick’s version says the slope could be 0 from December 1992 at the 95% level, whereas SkS says it is August of 1989 for the two sigma which is 95.2% if I am not mistaken. But then there is the other issue that I am not sure about. Is the SkS value really 95.2% or is it really 97.6% since there is a 2.4% chance the number is lower and a 2.4% chance the number is higher than the SkS two sigma limits. Do I have that correct?
At the end of the comments on this post, I would just like to know if I should use SkS or Nick’s version for my statistical significances in my future posts. Thank you for your inputs here.