Note: Short term predictions are relatively easy, it remains to be seen if this holds up over the long term. I have my doubts. – Anthony
Guest post by Frank Lemke
The Global Warming Prediction Project is an impartial, transparent, and independent project where no public, private or corporate funding is involved. It is about original concepts and results of inductive self-organizing modeling and prediction of global warming and related problems.
In September 2011, we presented a medium-term (79 months) quantitative prediction of monthly global mean temperatures based on an interdependent system model of the atmosphere developed by KnowledgeMiner, which was also discussed at Climate Etc. in October 2011. This model describes a non-linear dynamic system of the atmosphere consisting of 5 major climate drivers: Ozone concentration, aerosols, radiative cloud fraction, and global mean temperature as endogenous variables and sun activity (sunspot numbers) as exogenous variable of the system. This system model was obtained from monthly observation data of the past 33 years (6 variables in total: the 5 variables the system is actually composed of (see above) plus CO2, which, however, has not been identified as relevant system variable), exclusively, by unique self-organizing knowledge extraction technologies.
Now, more than a year has passed, and we can verify what has been predicted relative to the temperatures, which have really been measured (fig. 1).
Verifying the prediction skill of the system model from April 2011 to December 2012, the accuracy of the most likely forecast (solid red line) remains at a high level of 75%, and the accuracy relative to prediction uncertainty (pink area) is an exceptional 98%. Given the noise in the data (presumably incomplete set of system variables considered, noise added during measurement and preprocessing of raw observation data, or random events, for example), this clearly confirms the validity of the system model and its forecast.
In comparison, the IPCC AR4 A1B projection currently shows a prediction accuracy of 23% (September 2007 – December 2012, 64 months) and just 7% accuracy for the same forecast horizon as applied for the system model (April 2011 – December 2012, 21 months).
The two models, IPCC model and atmospheric system model, use two very different modeling approaches: theory-driven vs data-driven modeling. The IPCC model is based essentially on AGW theory by emission of greenhouse gases, namely CO2, the presented atmospheric system model on the other hand is a CO2-free prediction model. It is described by 5 other variables. The IPCC model shows a prediction accuracy of 7% and the atmospheric system model an accuracy of 75% for the same most recent 21 months of time…
The climate system is a complex system that consists of a number of variables, which are connected interdependently, nonlinearly and dynamically and where it is not clear, which are the causes and which are the effects. The simplistic linear cause-effect relationship “more atmospheric CO2 = higher temperatures” the IPCC model is based on is not an adequate tool to describe the complexity of the atmosphere sufficiently.
Read the complete post here:
http://climateprediction.eu/cc/Main/Entries/2013/1/21_What_Drives_Global_Warming_-_Update.html
Moe, whilst I am fascinated by your ability to take such a beating and keep coming back, it’s time to follow The First Law of Holes.
RichardCourtney and D.B. Stealey – this is starting look like a couple of Rottweilers tearing apart a kitten; please let Moe go…
Richard, thank heavens you are back, so I can educate you again. I must admit I admire your persistence at showing your ignorance.
They may not have covered this in the math subject in your diploma course in divinity or whatever it was you did, so here goes.
You have a time series of the Earth’s temperature. That is temperatures taken at regular intervals. In this case yearly (year is a unit of TIME).
When you plot temperature against TIME, the plot goes up and down in a zig zag fashion. You can put a trend line through it. It will have a gradient. If you wish to see which way the gradient is slopinging, you have to take enough years of data to make sure you are confident of the underlying trend. I will stress this for the dim, ENOUGH YEARS OF DATA. That Richard is time…..
You admit 16 years is not enough TIME, so I ask you to work put how much TIME you need to be confident that your conclusion about the slope of the trend line. You will not do this because you know which way the trend slopes. Upwards, ie the Earth is warming.
Then you persist in your repeated error that time has nothing to do with it. It doesn’t matter as a few of my friends are injoying reading your pathetic attempts to redeem your credibility, but with each post, you damage it further.
Thanks horse, but I am not being mauled or taking a beating. As you can see Richard is hung up on an error, which he thinks he can bluff his way into making better by repeating it. (Along with his customary abuse).
Nice of you to take an interest in me, but I am quiet ok. I keep goading Richard back as I am quiet enjoying him showing his ignorance (surely, you can give me credit for that as earlier he vowed to stop responding).
Horse:
I appreciate your post at January 29, 2013 at 2:39 am which says in total
I accept your point about what this “looks like” but, with respect, there are three reasons why I do not think Moe can be “let go” but needs to be forced to desist.
Firstly, Moe has been successful in destroying this thread (does anybody now remember that the subject of this thread is a specific climate model?). Failure, to defeat Moe’s behaviour will encourage other trolls to disrupt other WUWT threads in similar manner.
Secondly, his disruptions have consisted solely of falsehoods and misrepresentations. The Moderation policy of WUWT allows free expression and if that is to be of value then it has to enable complete refutation of presented falsehoods and misrepresentations.
Thirdly, from behind the cowardly shield of anonymity, Moe made a post aimed at me which consisted solely of lies, smears and personal defamations. He has not retracted or apologised for any of that, so I feel no reason to be gentle when correcting falsehoods and misrepresentations from the egregious coward.
Richard
Richard, cry me a river of tears. It appears you can dish it put, but can’t take it.
Why don’t you spend your time constructively and work out how many years data you would need to get a statistically significant answer to what is happening to the Earth’s temperature. You know what I mean… Be a REALskeptic for once.
Moe:
Your post at January 29, 2013 at 3:24 am says in total
What is it I can’t take?
I have kicked your butt from here to Alaska and all you have done is tell lies.
There has been no statistically significant trend (at 95% confidence) in global temperature for the last 16 years. There was for the previous 16 years.
I explained how and why regression analysis is conducted and how the validity of a regression result is demonstrated by the r^2 statistic.
You have repeatedly asserted the nonsense that the validity of a regression analysis depends on the length of the time series and not the variance of the data.
OK. Demonstrate your silly assertion. Explain why it is so if you really think it is.
Stop shouting ‘It is, it is!’ and explain why you think it is.
Your baby talk is earning your pay as an employed troll, but it is achieving nothing else except to show how warmunists will go to any lengths to spread disinformation.
Explain your ignorant and silly assertion. Put up or shut up.
Richard
PS I anticipate more lies and irrelevance instead of the required explanation from you.
Moe says:
January 28, 2013 at 1:12 pm
It is longer than 17 years and the result will show you the Earth is warming.
If the earth were in fact warming as the models predicted, then 16 years would be sufficient. As proof, see the four years below. Note the period from 1995 to 2010.
Start of 1995 to end 2009: 0.133 +/- 0.144. Warming for 15 years is NOT significant which DOES agree with Phil Jones.
Start of 1995 to end 2010: 0.137 +/- 0.129. Warming for 16 years IS significant which DOES agree with Phil Jones.
Start of 1995 to end 2011: 0.109 +/- 0.119. Warming for 17 years is NOT significant.
Start of 1995 to October 2012: 0.098 +/- 0.111. Warming for 18 years is NOT significant.
Werner Brozek:
Thankyou for your post at January 29, 2013 at 10:13 am which clearly demonstrates the scientific fact that significance is a function of the variance and not the length of the data set.
However, that scientific point is not now the issue. If science were the issue then we would still be evaluating the climate forecast model which is the proper subject of this thread.
The subject now is that a troll, Moe, has completely derailed this thread by use of a series of falsehoods. It is imperative that Moe needs to be forced to admit that at least one of his false assertions is wrong: otherwise other trolls will assault WUWT threads to cause similar derailment by similar method.
Anybody who knows anything about elementary statistical analysis knows Moe is wrong. The need is to force Moe to admit that he knows he is wrong because that will defeat his method of disruption. I have explained Moe is wrong, and you have demonstrated Moe is wrong, but it needs to be admitted by Moe that Moe is wrong.
Richard
Werner, you are wrong, even Richard concedes that sixteen years is insufficient time to get statistical significance as in ‘there is no statistical significance in 16 years’. Of course Richard is using the accepted 95% confidence level.
Moe,
Werner Brozek is correct, you are wrong, and the more you comment the clearer it becomes that you are a noob to this subject. You cannot even get the scientific method right. And trying to nitpick your way out of the fact that global warming has stalled only shows that you are blind to extensive scientific evidence.
Cognitive dissonance. It is usually incurable, and it infects legions of wild-eyed climate alarmists like you.
Moe:
At January 29, 2013 at 6:36 am I wrote to you saying
And I added
Your immediately subsequent post (at January 29, 2013 at 12:12 pm) fails to explain why you assert your nonsensical falsehood and spouts more falsehoods. Saying in total
Quad Erat Demonstrandum.
I have repeatedly told you that the most recent 16 years DOES give a statistically significant trend and the trend is indistinguishable from zero (at 95% confidence) . Your assertion that I have said otherwise is merely another of your lies.
I have explained how and why the length of time is not relevant but the variance of the data is when assessing the statistical significance of a time series.
You make the daft claim that the length of time determines statistical significance. I have challenged you to justify that falsehood. You have ignored the challenge and – as I predicted you would – instead you have provided more irrelevant falsehood.
PUT UP OR SHUT UP.
Richard
Moe says:
January 29, 2013 at 12:12 pm
Werner, you are wrong
You may want to take this up with Phil Jones from an article on June 2011:
“Climate warming since 1995 is now statistically significant, according to Phil Jones, the UK scientist targeted in the “ClimateGate” affair.”
http://www.bbc.co.uk/news/science-environment-13719510
I see there are 4 independent pieces of evidence that show CO2 does not control climate
1)This model that can reproduce temperature trends better without referencing CO2, than the CO2 based GCM’s.
2)Temperature trends have flattened off for a length of time that climate modelers define as improbable, as CO2 continues to increase.
3)My investigation of night time cooling shows no loss of cooling for the bulk of the surface record for the periods that have a reasonable number of station records (1950-2010). Included in these records are a few stations with very low humidity, and low winds that had a 60 degree F swing up and down within 24 hr’s.
4)Utilizing a handheld IR thermometer, on a 35F day, pointing it at a clear sky, reads below minimum scale of -40F, or it reads max scale of over 608F in the general direction of the Sun.
@moe,
I think you’re coming at statistical significance from the wrong direction, maybe I can help illustrate with an example, leaving out the computations:
Say I’m out measuring Subway ‘foot long’ sandwiches. I measure two sets of 20 sandwiches each set.
In the first set, all of the measurements are more or less evenly distributed between 11.9 and 12.1 inches.
In the second set, lets say my measurements come out as more or less evenly distributed between 11 and 13 inches.
The variance in this second set is larger (and hence the standard deviation is larger), and this alone affects what we can say with any desired degree of confidence about the set. Clearly, we have greater confidence that futher samples taken from the first set are going to be within + or – .5 inches of 12 inches than the second set, right? Time has nothing to do with it, the size of the sample set doesn’t have a whole lot to do with it; it’s the variance you compute from the data that determines what you can say with what degree of confidence. The only reason the size of the set matters is that the odds of getting a bad estimate of mean and variance goes down with a greater number of samples / observations.
Hope this helps.
Also note that a single yearly average is made from ~3.5 million daily records (based on the increasing record counts for almost every year we’ve been taking measurements and the record count of 2010)
richardscourtney says:
January 29, 2013 at 12:44 pm
Moe:
At January 29, 2013 at 6:36 am I wrote to you saying
You have repeatedly asserted the nonsense that the validity of a regression analysis depends on the length of the time series and not the variance of the data.
OK. Demonstrate your silly assertion. Explain why it is so if you really think it is.
Stop shouting ‘It is, it is!’ and explain why you think it is.
[self snip]
Put up or shut up.
Richard, I’m afraid you’re missing something.
In order to determine the statistical significance of a trend in time-series data you first calculate the correlation coefficient, r, then you determine the critical value for a certain degree of significance, say 0.05, that threshold value depends on the number of points.
See attached table:
http://www.gifted.uconn.edu/siegle/research/correlation/corrchrt.htm
The number of degrees of freedom, df=Number of points-2
Note that you need a r value of .990 for 0.01 level of significance with 4 points but only r of 0.254 for the same significance with 100 points.
I have explained how and why the length of time is not relevant but the variance of the data is when assessing the statistical significance of a time series.
You make the daft claim that the length of time determines statistical significance. I have challenged you to justify that falsehood. You have ignored the challenge and – as I predicted you would – instead you have provided more irrelevant falsehood.
Consider it justified, I suggest you learn some statistics before pontificating on the subject next time.
Phil. says:
February 1, 2013 at 8:46 am
Let me note that each of these 16 + points include 365 daily points, made from just under 10,000 individual measurements, so these 16 years of data are actually derived from about 47 million measurements. That seems significant to me.
Phil.:
I read your ignorant arm-waving at February 1, 2013 at 8:46 am.
I will bother to detail your error when you have addressed the point by MiCro at February 1, 2013 at 9:38 am.
I suggest that you use your full name when you next spout rubbish. There is no fun in chopping off the head of an anonymous troll even one who – like you – has been called in as a last-ditch defence of a defenestrated anonymous troll.
For now, I point out that using http://www.skepticalscience.com/trend.php to determine how long it has been that the global temperature trend is not different from zero at 95% confidence one obtains the following values from the different data sets.
RSS
Warming is NOT significant for over the most recent 23 years.
Trend: +0.126 +/-0.136 C/decade at the two sigma level from 1990
UAH
Warming is NOT significant for over the most recent 19 years.
Trend: 0.143 +/- 0.173 C/decade at the two sigma level from 1994
Hacrut3
Warming is NOT significant for over the most recent 19 years.
Trend: 0.098 +/- 0.113 C/decade at the two sigma level from 1994
Hacrut4
Warming is NOT significant for over the most recent 18 years.
Trend: 0.095 +/- 0.111 C/decade at the two sigma level from 1995
GISS
Warming is NOT significant for over the most recent 17 years.
Trend: 0.116 +/- 0.122 C/decade at the two sigma level from 1996
The times to the nearest month when warming is not significant for each set are:
RSS since September 1989;
UAH since April 1993;
Hadcrut3 since September 1993;
Hadcrut4 since August 1994 and
GISS since October 1995
Richard
MiCro says:
February 1, 2013 at 9:38 am
Phil. says:
February 1, 2013 at 8:46 am
Note that you need a r value of .990 for 0.01 level of significance with 4 points but only r of 0.254 for the same significance with 100 points.
Consider it justified, I suggest you learn some statistics before pontificating on the subject next time.
Let me note that each of these 16 + points include 365 daily points, made from just under 10,000 individual measurements, so these 16 years of data are actually derived from about 47 million measurements. That seems significant to me.
What it ‘seems’ to you is hardly relevant, what’s referred to is the statistical significance of a fit, that is related to the number of points being fitted. If you’re fitting annual data you have 16 points, monthly data 192 points.
Phil.:
Thankyou for your reply to MiCro at February 1, 2013 at 10:34 am which says
Yes! 192 monthly points or only 16 annual points in 16 years.
So, now take the next step and see if you can work out why the time period is not important.
Richard
richardscourtney says:
February 1, 2013 at 10:52 am
Phil.:
Thankyou for your reply to MiCro at February 1, 2013 at 10:34 am which says
What it ‘seems’ to you is hardly relevant, what’s referred to is the statistical significance of a fit, that is related to the number of points being fitted. If you’re fitting annual data you have 16 points, monthly data 192 points.
Yes! 192 monthly points or only 16 annual points in 16 years.
So, now take the next step and see if you can work out why the time period is not important.
Still waiting for the correlation coefficient along with the number of points.
Phil.:
re your post at February 1, 2013 at 11:17 am.
Nice body swerve but it does not avoid the tackle.
You claimed it was the length of the time period which affects the r^2 as a denial of my correct statement that it is the variance of the data set. You now admit that the number of points within a time period affects the variance.
But in attempt to obfuscate the issue you now demand that I produce the variance (presumably for each of the data sets I cited at February 1, 2013 at 10:34 am).
No dice! You have admitted I am right and your assertion was wrong.
Unless, of course, you can show that the variance is the same for the monthly (which I used) and the annual data of at least one of the data sets I cited.
Richard
richardscourtney says:
February 1, 2013 at 12:51 pm
Phil.:
re your post at February 1, 2013 at 11:17 am.
Nice body swerve but it does not avoid the tackle.
You claimed it was the length of the time period which affects the r^2 as a denial of my correct statement that it is the variance of the data set. You now admit that the number of points within a time period affects the variance.
No I did not Richard, as an editor your reading comprehension skills are terrible!
What I clearly said was: “In order to determine the statistical significance of a trend in time-series data you first calculate the correlation coefficient, r, then you determine the critical value for a certain degree of significance, say 0.05, that threshold value depends on the number of points.”
So to spell it out for you again: you first calculate the correlation coefficient, r. (This is not the variance!)
Then in order to determine the ‘statistical significance’ of the fit you compare that with the critical value for the data set, usually one gets that from tables, for any given level of significance (say 0.05) this is tabulated against the degrees of freedom which is given by the (number of data points used -2). For the fit to be significant at that level the correlation coefficient must exceed the critical value.
But in attempt to obfuscate the issue you now demand that I produce the variance (presumably for each of the data sets I cited at February 1, 2013 at 10:34 am).
No I did not, I asked you for the Correlation coefficient and the number of data points used so that we could assess the statistical significance for ourselves.
No dice! You have admitted I am right and your assertion was wrong.
I certainly have done no such thing, I have however demonstrated that you don’t know what you are doing!
Unless, of course, you can show that the variance is the same for the monthly (which I used) and the annual data of at least one of the data sets I cited.
For the monthly series fit to be statistically significant the Correlation coefficient should exceed ~0.125 for the annual series it should exceed ~0.468.
Phil.:
I am responding to your obfuscation at February 1, 2013 at 3:24 pm.
No! I am still refusing to be side-tracked. The reality is as follows.
1.
The anonymous troll posting as Moe wrongly claimed the statistical significance of the linear trend of a time series is determined by the time period of the time series.
2.
I repeatedly explained he was wrong and that the statistical significance of the linear trend of a time series is determined by the variance of the data set. Indeed, I even explained what a linear trend is, what a linear regression does to determine the trend, and how the significance of a determined trend is determined.
3.
The troll posting as Moe continued his erroneous assertion and I challenged him to explain how and why the statistical significance of the linear trend of a time series is determined by the time period of the time series (i.e. not the variance of the data set)..
4.
The troll posting as Moe withdrew when others attempted to explain the matter to him.
5.
Three days after the withdrawal of Moe, and possibly on instruction from ‘troll central’, you entered the thread at February 1, 2013 at 8:46 am. In that post you stated that the number of points affects the confidence then quoted my having said to Moe
And added
6.
Of course, you had not justified it at all: you had mentioned the number of points in the data set (n.b. not the time period) and added an untrue ad hom.
7.
MiCro pointed out that the assessed data was derived from thousands of data points, and I demanded you answer that before I replied to your nonsensical post.
8.
At February 1, 2013 at 10:34 am, your reply to MiCro did as anticipated and admitted that the result of a 16-year trend would differ if one used the 16 annual points or the 192 monthly points; i.e.
You admitted that it was the data set which determined the variance and, thus, the confidence of the trend and NOT the time period of the trend.
9.
At February 1, 2013 at 10:52 am, I pointed out that you had admitted I was right and challenged you to explicitly state you were wrong.
10.
You subsequent posts have each attempted to prevaricate the fact that your post at February 1, 2013 at 10:34 am admits I was right and you were wrong.
I suggest you go back to ‘Troll Central’ and tell them the lie peddled by Moe cannot be defended and that is why you have failed to defend it.
Richard
richardscourtney says:
February 2, 2013 at 3:31 am
Phil.:
I am responding to your obfuscation at February 1, 2013 at 3:24 pm.
No! I am still refusing to be side-tracked. The reality is as follows.
Hardly sidetracked, you’re being shown how to properly assess statistical significance and the data to do so was requested.
1. The anonymous troll posting as Moe wrongly claimed the statistical significance of the linear trend of a time series is determined by the time period of the time series.
Quite correctly as pointed out, when fitting the linear trend to a number of points the correlation coefficient is determined and then tested to see if it represents a significant trend. To do this the critical value for the data is determined using tables and to be significant r must exceed this value. This critical value depends strongly on the number of points used. For a monthly time series the number of points is proportional to the time period over which the fit is made.
2.
I repeatedly explained he was wrong and that the statistical significance of the linear trend of a time series is determined by the variance of the data set. Indeed, I even explained what a linear trend is, what a linear regression does to determine the trend, and how the significance of a determined trend is determined.
But didn’t get it right, I take it you didn’t read the link I gave which does show how to determine significance.
5.
Three days after the withdrawal of Moe, and possibly on instruction from ‘troll central’, you entered the thread at February 1, 2013 at 8:46 am. In that post you stated that the number of points affects the confidence then quoted my having said to Moe
As far as I know the ‘troll central’ only exists in your head.
6.
Of course, you had not justified it at all: you had mentioned the number of points in the data set (n.b. not the time period) and added an untrue ad hom.
The number of points used in the fit is a function of the time period! Apparently you don’t know what an ad hom is, surprising since you constantly use them as you have in this post.
7.
MiCro pointed out that the assessed data was derived from thousands of data points, and I demanded you answer that before I replied to your nonsensical post.
Well I answered Micro before your demand! As I said it’e the number of points used in the fit that counts not how they’re derived.
You admitted that it was the data set which determined the variance and, thus, the confidence of the trend and NOT the time period of the trend.
No your reading comprehension problems again, in a time series the length of the series determines the significance of the fit.
As I told you above for an annual series of 16 points you need an r of 0.497 for significance at p=0.05, for 18 years it’s 0.468 and for 10 years it’s 0.632.
For a monthly series of 192 points you need an r of ~0.125 for significance at p=0.05.
According to you:
“Hacrut4
Warming is NOT significant for over the most recent 18 years.
Trend: 0.095 +/- 0.111 C/decade at the two sigma level from 1995”
So you’re claiming that r for that dataset is less than ~0.125?