Guest Post by Jeff Id:
“Tamino” has made a couple of posts on how the last 10 year drop in temperature is not statistically significant, so it isn’t real. He went too far in his last one and began claiming it was a tactic of some kind of creature called a denialist to confuse and confound the public.
Let’s see what Tamino has been saying on his blog link HERE.
Some of you might wonder why I make so many posts about the impact of noise on trend analysis, and how it can not only lead to mistaken conclusions about temperature trends, it can be abused by those who wish deliberately to mislead readers. The reason is that this is still a common tactic by denialists to confuse and confound the public.
I just hate bad science. First he points out how Bjorn Lomborg made some comments about temperature decreasing, after placing the ever more popular label of denialist on him implying Lomborg’s statements were intended to confound and confuse the public. Heres the main point of what Bjorn Lomborg said.
They (temperatures) have actually decreased by between 0.01 and 0.1C per decade.
Ok, so graphs like the one below are the reason Bjorn Lomborg is a denialist.
I copied this graph from Digital Diatribes of a Random Idiot – A great unbiased site for trends (link on the right). Note the slope of -.0082 (.01C/month units or .00098 degC/year – Thanks to digitial diatribes comment below) in the equation on the graph. Most of us know this is actual data and is correct, in fact every measure is showing similar results. The earth stopped warming- a very inconvenient truth. So Tamino what’s the argument, why are the evil and uncooperative denialists wrong?
Statistics of course.
Here comes the numbers from Tamino.
The most natural meaning of “this decade” is — well, this decade, i.e., the 2000’s. So I computed the trend and its uncertainty (in deg.C/decade) for three data sets: NASA GISS, RSS TLT, and UAH TLT, using data from 2000 to the present. To estimate the uncertainties, I modelled the noise as an ARMA(1,1) process. Here are the results:
Data Rate (deg.C/decade)
Uncertainty (2-sigma)
GISS +0.11 0.28 RSS +0.03 0.40 UAH +0.05 0.42 All three of these show warming during “this decade,” although for none of them is the result statistically significant.
Ok Tamino has calculated GISS, RSS and UAH. One ground measurement and two satellite. For those of you who don’t spend their afternoons and weekends digging into this. ARMA is a fancy sounding method for what ends up being a simple process Tamino has used to estimate the standard deviation of the temperature. Sometimes it seems the global warming guys believe the more complicated the better, but no matter. He has a 2 sigma column which represents about 95%. He then goes on to say that because of the sigma 0.28 or 0.40 is bigger than the trend, the trend is not statistically significant. He repeats the comment below.
Let’s make the same calculation using data from January 1998 to the present:
Data Rate (deg.C/decade)
Uncertainty (2-sigma)
GISS +0.10 0.22 RSS -0.07 0.38 UAH -0.05 0.38
Finally one can obtain negative trend rates, but only for 2 of the 3 data sets. But again, none of the results is statistically significant. Even allowing this dreadfully dishonest cherry-picked start date, the most favorable
Now Tamino claims to be a statistician so I can’t see how he made such a simple boneheaded error but if he wants to pitch softballs, I’ll hit em. Just to make sure he’s in good and deep here’s one more quote.
I’ve previously said “Those who point to 10-year “trends,” or 7-year “trends,” to claim that global warming has come to a halt, or even slowed, are fooling themselves.” I may have been mistaken; is Lomborg fooling himself, or does he know exactly what he’s doing?
So, Mr. Lomborg, we’re all very curious: how did you get those numbers?
Wrong turns everywhere
The first and really obvious error Tamino makes is referring to the short term variation in temperature as noise. Noise in the context of sigma is related to measurement error. How can we determine the measurement error of the three methods GISS, RSS and UAH. Well the graph of the three is below.
The first thing you notice from this graph is that the 3 measurements track each other pretty well. The signal is therefore not completely noise. Well what is the level of noise? We have above 12 measurements per year times 29 years. So we don’t need ARMA or other BS we can simply subtract the data. I put the numbers in a spreadsheet and calculated the difference between RSS and GISS, RSS and UAH and UAH and GISS. With 348 measurments for each type of instrument I was able to get a very good estimate of standard deviation of the actual measurements. Again, no ARMA, just using the difference between the graphs.
GISS – RSS one sigma 0.099 Two sigma 0.198
RSS-UAH one sigma 0.101 Two sigma 0.202
GISS-UAH one sigma 0.058 Two sigma 0.116
These are actual numbers and are substantially lower than the estimated two sigma by Tamino but still bigger than the 0.1 C per decade although the two sigma GISS – UAH is within a 90% confidence interval already!
This isn’t the end though. Tamino ended his discussion there implying shenanigans and other things of those who see a trend.
Both of our standard deviation calcs are for a SINGLE measurement NOT a trend.
This is a big screw up. How can a self proclaimed statistical expert miss this, it’s beyond me. Anyway, none of us is universally right every day but most hold their tongue rather than post a big boner on the internet. Well most scientists realize that when you take more than one measurement of a value you improve the accuracy. So being a non-genius, I used R to calculate what the statistical certainty of the slope is when taken over 10 year trends. Thanks again to Steve McIntyre for pointing me to this software. I don’t love it but it is convenient.
t=read.csv(”c:/agw/giss data/10 year variation.csv”, header=FALSE)
x = (1:length(t[,1]))
y=t[,1]
a=gls(y ~x)
confint(a)
confint(a)[2,1]-confint(a)[2,2]
y=t[,2]
a=gls(y ~x)
confint(a)
confint(a)[2,1]-confint(a)[2,2]
y=t[,3]
a=gls(y ~x)
confint(a)
confint(a)[2,1]-confint(a)[2,2]
What this script does is load the difference files i.e. GISS-UAH, fits a line to them and presents a number for the statistical confidence interval of the slope coefficient at 95 percent confidence which is about two sigma. The confidence of the slope of the trend is as follows
GISS – RSS Two sigma 0.00108 DegC/year
RSS-UAH Two sigma 0.001068 DegC/year
GISS-UAH Two sigma 0.0005154 DegC/year
Despite a standard deviation of .02 We have a twenty times more accurate slope measurement of 0.001degC/year !
Conclusions
1. We can say with a high degree of certainty that we know the trend of temperature for any ten year plot to within .01 degC/decade.
2. We can say that temperatures have dropped this past decade, just as our eyes looking at the graphs had already told us.
3. We can also say that Tamino owes a few more apologies.
He and Real Climate still don’t let me post on their blogs!
I wonder why?
Magnus A made a comment about Tamino’s bad math at the link above. Here’s his reply
“I’ve seen Jeff Id’s post; its reproduced at the garbage dump Anthony Watts’ blog. That’s where it belongs. It’s good for a laugh.”
How do you know when someone is backed into a corner?
Anthony,
Since [Tamino] won’t let you post on his blog you may not have seen this sarcastic parody that insults you, Leif, Senator Inhofe, Sarah Palin and anyone else with a dissenting opinion of AGW in a thinly veiled ad hominem attack.
http://tamino.wordpress.com/2008/10/19/ipc-projection-falsified/
Even one of the commenter’s takes a mocking swipe at you and your blog traffic. Maybe it’s a good thing he doesn’t let you post.
So, the debate on climate change has degenerated to derision and ridicule. The following quote comes to mind:
Ridicule is the first and last argument of fools. -Charles Paul Simmons, (1924), US Author
REPLY: I worry not what Tamino nor his commenters say. When they stop hiding behind web monikers, and use a real name like you have done, then they become real people with valid opinions. As it stands now they just exist in the noise band. – Anthony
How far back do you need to go to get a significant warming trend (outside 2 sigmas) using Taminos method? Has there been a significant trend since, say 1988 or 1979?
The discussion of when “this decade” start reminds me of news storied published in 1999. Squabbles erupted between the traditionalists who point out that the calender system has no year zero, so the centuries are supposed to run from 1-100, 101-200 etc. In this context, all the big parties were supposed to occur on the night of Dec 2000.
However, the new fangled usages assumes decades are 100-199, 200-299 etc. Of course one just pretends there is no difficulty with the first decade AD being 1-99, which is flanked by (-99 to -1 BCE).
Some of this is discussed at wikipedia where they say:
Those following ordinal year names naturally choose
* 2001–2010 as the current decade
* 2001–2100 as the current century
* 2001–3000 as the current millennium
Those following cardinal year names equally naturally choose
* 2000–2009 as the current decade
* 2000–2099 as the current century
* 2000–2999 as the current millennium
So, as you see, there is no “one true answer” telling us when “this decade” began. Some will say 2001; some will say 2000!
Anthony,
I disagree with Derek D (10:21:44).
Statistics has useful tools, and statistical results are meaningful when used correctly and understood in context.
Thank you for what you are doing.
Lyman Horne
Statistics. Actually, you’re both off base. Sigma measures the spread (confidence interval) around the mean for a given sample to a certain degree of accuracy. The 95% accuracy indicates an alpha of .05. This says that the actual mean has a 1 in 20 chance of residing outside the confidence interval. An alpha of .05 is the minumum measure for a statistical sample to be considered statistically signficant. However, what is “statistically significant” will change depending on the consequences of being wrong. Political polls are file with this alpha, new drugs require an alpha of much tighter precision.
To determine the alhpa, you would first need to know what a complete measurement of world temperature would require. I’ve never seen anyone say what that would be and can’t imagine what one would look like. I would be inherently complex, given the size, distribution, and dynamic nature of the atmospheric temperature. The number physical locations would be extreme and would have to be near continuous. From what I’ve read, satellite measurements give a pretty near comprehensive data set. However, in that case, measurements would not be statistical (ie a sample of the whole) but actual measurements of the whole. Statistical calculations would not be relevant. How you combine and evaluate the numbers would obviously involve mathmatics but not confidence intervals for actual temperatures.
Lyman,
So I don’t make any enemies here, understand that I fully agree that statistics are useful tools and can produce meaningful results. Furthermore, I have a TREMENDOUS appreciation for Anthony running this site, and undertaking the efforts he has in the interest of truth.
However in this case of Anthony vs Tamino, we have a case of two different conclusions, implying two totally opposite conclusions derived from THE EXACT SAME DATA SET. This is where statistics become a trap, a point not lost on the Warmists. If you set out to claim something like AGW that certainly will never be proven factually and scientifically, it is in your best interests to stay out of the arena of proof entirely. Thus the warmists prefer to present one manipulated statistical conclusion after another. The result is that right minded people like Anthony feel compelled to diligently rebuke the data, pointing out procedural errors as any good scientist would. Problem is he’s not dealing with scientists, he’s dealing with political propagandists, who feel no compulsion to bear the burden of proof. So after spending much time and effort debunking one claim, he is met with 10 more, and it goes without saying that they will be flawed too. So how many months weeks or years should he waste giving them thorough review? It’s a diversion. The agenda will be pushed on through other avenues, while Anthony struggles pores over reams of statistics that we all know are bunk from the outset. And at that point, his good intentions, and diligent efforts are all for naught.
So while Anthony’s strength is statistics, mine is outwitting shitheads. And in this case, my strengths may pay better dividends against such an enemy as the Warmist crowd. As such my post is nothing more than the best intentioned, best reasoned, most altruistic advice I can offer after considering the bigger picture and the cast of characters involved. Anthony’s talents are too valuable to waste grading the half-assed statistics homework of amateur scientists and professional propagandists.
Derek D (13:07:38) :
Whew! A thread ending blog if I ever saw one!
I read, and appreciated, Derek D’s post. And I think I understand the points he was making. There were two which I would like to add to….
I agree that statistics can certainly be a valuable tool, but a bad master. Derek said that “STATISTICS NEVER PROVES SCIENCE…” – I thought it was the case that statistics do not PROVE anything. The scholium is simply a set of techniques for understanding trends in large masses of data. What never seems to be realised is that statistics will happily indicate any trend you care to look for – people seem to think that just because I have calculated a rising trend in pet ownership amongst left-handed Norwegians that this is somehow of importance….
I am not so sure I am with Derek when he warns against joining battle ‘on the enemies’ ground’. It is true that there you will be mired in manipulation, and your words will be twisted to make it seem as if you are always loosing.
But the battle will not be won only amongst the politicians and warmist propagandists. Many non-mathematical scientists and other opinion formers currently believe because they have no time or inclination to read beyond the propaganda, and cannot believe that some complex maths can be anything but correct. They will need to be led gently away from this belief – just telling them that they are dead wrong will not persuade them. They need to see disputation in maths to understand for themselves that lies can be numeric as well as verbal. This is where Steve McIntyre scores highly. He stresses that he is neither a ‘warmer’ nor a ‘denier’ – he just wants to see good science, so he spends a lot of time uncovering appalling statistical malfeasance. And this difficult position which he has held is now standing him in good stead – ‘warmer’ science institutions are able to listen to him, because he is scrupulously polite and talks their language. It is from work such as his that the huge walls of the AGW hypothesis will first start to crumble.
Well, I read this entire list of posts, looking for some climate insights; but it seems like I am reading from the lecture notes of the statistical mathematics lecturer in the political science department.
The beautiful thing about mathematics, particularly statistical mathematics, is that you can apply such analyses to any set of data; whether such data is the computed output of f(xyz), or is random, or chaotic, or simply quite arbitrary.
A good example would be to start with say the Manhattan telephone directory, where all the numbers derive from some sort of rules, at least they are all ten digits, counting area codes, but otherwise there is no rational connection between the number and the exact location of the telephone that rings, if you dial that number. But it is trivial to compute the average, or mean, the median, or any other statistical parameter associated with that book of raw data. But it doesn’t mean anything to anybody (but a statistician). And when they print a new phone book in say three years; nothing meaningful can be ascertained from the differences between the statistical analysis of the two editions of the telephone book.
Do any of the people who call themselves climatologists, actually do anything that involves Physics, or Physical Chemistry, or Oceanograph, or Geology; or anything that is real science.
Mathematics as you know, is all pure fiction; it isn’t any universal truth. We made it all up in our heads, out of whole cloth. We had darn good reasons for doing that; to create tools to describe the functioning of our science models; which are themselves all fictional as wel.
Absolutely NOTHING, that we use or talk about in mathematics, actually exists anywhere in the real universe.
There are no points, no lines, no planes, no circles or spheres; none of those things exist.
The equation x^2 +y^2 +z^2 = r^2 describes a sphere in Euclidean geometry, but no way does it explain something like 8km high mountains on the surface of such an object !
It is no wonder that the GCM computer geeks seem to rule the roost in climatology; they don’t seem to even need any input form the real universe to work their magic.
All such things are perfectly good data sets to work the wonders of statistical mathematics on.
The results are quite meaningless of course.
A good example of statistics run amok arose in the Viet Nam War era, when somebody decided to have a draft lottery based on birthdates. The calendar days were assigned sequential numbers from 1 for Jan 1, to 366 for dec 31, including Feb 29 of course.
Numbers were drawn out of a hat, figuratively speaking, and perosn born on that date were chosen first to go into the military conscription program.
Within days of that very first draft lottery; the walls leaked statisticians, who started to pronounce the lottery unfair, and biassed, because more people with low numbers in Jan-feb were chosen early, in the view of thes math whizzes.
Now in such a draft lottery, there are factorial 366 possible draft outcomes; each one different. That is one huge number; I’ll let you math jocks calculate that for yourselves, since I don’t have a good calculator handy at the moment.
So these mathematical geniuses pronounced the first draft lottery to be not random; and they made this momentous gaffe, on the result of one single sample out of factorial 366 possible outcomes.
One of those possible outcomes was to have the numbers come out as Jan1, jan2, jan3,…..dec28, dec 29, dec30, dec31, in exact calendar order.
That outcome, which would have shocked the world, is no more unlikely, than the real outcome that occurred with that first draft lottery.
Statistics gives no information about any single event. Now if they had a draft lottery every second, and did so for the entire estimated age of the universe; you might get enough “experimental” data to say the lottery was non random; but genius level statisticians reached that conclusion based on a single data point
Who knew my ranting would provoke such a rational and intellectual discussion. But that is, after all, why I love to come here. Real scientists can handle a little tough language as long as the day is ultimately ruled by the facts.
You are the last of a dying breed. Keep fighting the good fight…
I do not disagree with Derek’s postings.
After spending years in academia with computer modelers (sic), I am disgusted with the output, being as it can only be, based on (often predetermined) assumptions.
I also think Derek has a good point. The AGW group is so well funded that thousands of people can spend their lives producing papers which must individually be disputed. In the meantime some of them might have validity. It is a nightmarishly and impossibly huge task.
I believe the only thing that can stop it is cold weather. No science, statistics, posturing or calling shenanigans will do it, there is too much money involved. Currently we are in a downtrend or flat trend, the sun is quiet and it is becoming evident that the trend is continuing through winter. I am not particularly religious but we are definitely not in the drivers seat on this one.
Pointing out that the downtrend is clearly real, using as simple an example as I could find seemed important. I think Anthony Watts understood the statistics and significance of the demonstration immediately just as he has followed the solar, ice and measurement stations so well.
Statistics are part of the ammunition of the politician. When the government (of any party, in any country) claims prices are stable or educational standards are rising they seek to justify their claim with statistics, and they do so because they know some people will be persuaded (even if they have no understanding of the statistics). We can say those people should not be persuaded, but they are. What are we meant to do, leave them persuaded by something we do not accept and say “bah, it’s only statistics”?
The political advantage to be gained by use of statistics is real. Opposing politicians would be ill-advised to say “bah, it’s only statistics”, that would be an admission of defeat. They have to answer like with like and produce their own statistics to seek to persuade the volatile audience against the government’s position. That the refutation might be based on just as flimsy ground as that being refuted is neither here nor there. It is not, at heart, an argument about who is right but about whether one side is to be allowed to deploy machine guns while the other side limits itself to water pistols.
Undertaking a statistical analysis on a set of data is, of itself, no more useful than doing a crossword. Its utility comes from the conclusions you draw (or invite others to draw) and from the consequences of those conclusions. As has been said here many times no one would give two hoots whether the world is warming unless it had adverse consequences. The warmists use statistics to conclude both that the world is warming and that the warming will have a seriously detrimental effect. From that point they advocate radical changes to the way we live. None of that stands without their supporting statistics and none of it can be challenged effectively without challenging those statistics.
One victory might spur your opponent into undertaking another statistical exercise which (as if my magic) comes to the same conclusion as the one you disproved. Your choice then is to say either “I give up, I’m tired of this game” or “ok, let’s see if his second attempt is any better”. And if you can cast sufficient doubt on his second method you are faced with the same choice when his third appears. There is no way out of this apart from surrender.
There is, however, an important side effect of being able to challenge a succession of analyses sufficiently strongly to force your opponent to have another go. Where his work is relied on by politicians to press for a particular policy, the fact that the reasons for that policy are re-cast time and again because flaws have been identified can (not will, but can) throw doubt on the policy itself. It is a war of attrition – you cannot avoid the attrition if you want to win the war.
I thoroughly enjoyed reading Derek’s posts, and find them enlightening, as they present a view from the 30,000′ level that is easy to overlook.
Are the statistical arguments without value?…I’d say no, because at the 500′ level, which is where many of us engage in this lunacy, it’s nice to have a few things in your hip pocket so you can say “Yeah…well if that’s true, why isn’t THIS true?”, in an attempt to make someone realize that the “science” is maybe not what they think it is.
In a war, you need both privates and generals…and the pay is the same, whether you’re fighting or marching.
JimB
Tamino is part of the “country club” of climate science referenced by Tenured Prof in the thread at Climate Audit titled “Peter Brown and Mann et al 2008” at comment #161. I posted a comment in response which I expect Steve McIntyre will snip. But this needs to be pointed out to the club members who somehow think they should never be subjected to criticism. My comment:
stan Says:
Re: Tenured Prof (#161),
Prof,
The country club of climate scientists would not bother anyone, if they didn’t use their club to influence govt policy. But they use their little club to try to impose enormous costs on everyone else in the world. Some in the club are even demanding that anyone who disagrees with the club be thrown in jail.
This may not be a focus of Steve’s, but it is a really big deal to a lot of his readers.
Bottom line — when the country club members use the club’s activities to demand significant sacrifices be imposed on the rest of the world, the rest of the world should have the right to demand that the club be competent, open, transparent, and honest in its dealings. And that right is not dependent on whether they like McIntyre or want to blackball him.
When the club started pushing the world around, the world got the right to push back. And looking under the rock is the first step.
This just in. I wonder how accurate each of these stations are.
These data are preliminary and have not undergone final quality control by the National Climatic Data Center (NCDC). Therefore, these data are subject to revision. Final and certified climate data can be accessed at the NCDC – http://www.ncdc.noaa.gov.
Record Report
000
SXUS76 KPDT 231815
RERPDT
RECORD EVENT REPORT
NATIONAL WEATHER SERVICE PENDLETON OR
1115 AM PDT THU OCT 23 2008
…NEW DAILY RECORD LOW TEMPERATURES FOR OCTOBER 23RD…
NOTE: STATIONS MARKED WITH * INDICATE THAT THE STATION REPORTS ONCE
PER DAY. FOR CONSISTENCY…THESE VALUES ARE CONSIDERED TO HAVE
OCCURRED ON THE DAY THE OBSERVATION WAS TAKEN BUT MAY HAVE ACTUALLY
OCCURRED (ESPECIALLY FOR MAX TEMPERATURE) ON THE PREVIOUS DAY.
STATION PREVIOUS NEW RECORDS
RECORD/YEAR RECORD BEGAN
PENDLETON(ARPT), OR 29 / 1984 29 (TIED) 1934 :SINCE MID
*UNION ES, OR 20 / 1980 17 1928
WALLA WALLA, WA 32 / 2000 32 (TIED) 1949 :SINCE MID
Sobering, I thank the previous writers.
Three points however.
One. The ENTIRE premise of ALL of the AGW-power-based taxes and government controls is based on a 1/2 of one degree rise in temperatures that (are claimed to) correspond to rising CO2 levels since WWII.
However, the ONLY time in the earth’s 4+ billion year history when both CO2 and temperatures have BOTH been rising at the same time is the 26 year period from 1972 through 1998. In the ten-year period since 1998, temperatures have been steady, or arguably, falling slightly, despite a steady increase in CO2 levels. If only one year in 37 showed a declining temeprature, that’s noise/weather/whether/wonder/why/why not/weird results and means nothing.
but when ten years of the AGW-extremist’s precious 37 year trend show complete opposite results of Hansen’s politics? That becomes meaningful!
Two. Realists MUST use any means available (including statistics) to show that the AGW’a-politically-driven-conclusion of taxes, government-controls and death is wrong.
Three. Why a straight line? Up, down, flat, or vertical. Temperatures across times are NOT linear, and any model that pretends to compare real-world data with a straight line will be wrong.
Use a sine wave. (Or a cosine wave if you are left handed, or prefer the metric system for sum reason.) Model the temperatures since 1908 WITH A SINE WAVE and you will see a long, natural rise and fall that match closely to a 70 year cycle that shows the high point in 1935-1945, the lows in 1970, the high point in 1995-2005, and today’s slow fall from the 2004-2006 peak. That curve, though harder to plot, WILL match real-world measured dates. Er, data. 8<)
But to try to match a flat line? No – You can ALWAYS be shown by a AGW-extremist to be wrong. Because, with a straight line, you ARE going to be wrong.
My eyeball field geologist computer says the last decade trend is flat. The 1998 record El Nino flood year is an anomaly.
1980-1997 = flat
1997-1998=step up about 0.3
1999-2008 = flat
Hard to believe CO2 warming produced this step function, which looks like noise, weather, what have you.
You can also say with very high confidence level that this decade has been almost 0,2 degrees Celcius warmer than the previous decade.
I think that’s what counts. The word “climate” refers to conditions of long period of time. Usually 30 years, but shorter periods can be used if seen necessary. However, I think anything less than 10 years is more weather than climate.
Climate change hasn’t stopped since this decade is record warm and a lot warmer than the previous one.
I think this shows perfectly what’s this all about: http://tamino.wordpress.com/2008/12/31/stupid-is-as-stupid-does/