This is a repost of two articles from John Graham-Cumming’s blog. I watched with interest earlier this month where he and a colleague identified what they thought to be a math error related to error calculation when applied to grid cells. It appears now through a journalistic backchannel that the Met Office is taking the issue seriously.
What I found most interesting is that while the error he found may lead to slightly less uncertainty, the magnitude of the the uncertainty (especially in homogenization) is quite large in the context of the AGW signal being sought. John asks in his post: “If you see an error in our working please let us know!” I’m sure WUWT readers can weigh in. – Anthony
The station errors in CRUTEM3 and HadCRUT3 are incorrect
I’m told by a BBC journalist that the Met Office has said through their press office that the errors that were pointed out by Ilya Goz and I have been confirmed. The station errors are being incorrectly calculated (almost certainly because of a bug in the software) and that the Met Office is rechecking all the error data.
I haven’t heard directly from the Met Office yet; apparently the Met Office is waiting to write to me when they have rechecked their entire dataset.
The outcome is likely to be a small reduction in the error bars surrounding the temperature trend. The trend itself should stay the same, but the uncertainty about the trend will be slightly less.
===============================================
Something odd in the CRUTEM3 station errors
Out of the blue I got a comment on my blog about CRUTEM3 station errors. The commenter wanted to know if I’d tried to verify them: I said I hadn’t since not all the underlying data for CRUTEM3 had been released. The commenter (who I now know to be someone called Ilya Goz) correctly pointed out that although a subset had been released, for some years and some locations on the globe that subset was in fact the entire set of data and so the errors could be checked.
Ilya went on to say that he was having a hard time reproducing the Met Office’s numbers. I encouraged him to write a blog post with an example. He did that (and it looks like he had to create a blog to do it). Sitting in the departures lounge at SFO I read through his blog post and Brohan et al.. Ilya’s reasoning seemed sound, his example was clear and I checked his underlying data against that given by the Met Office.
The trouble was Ilya’s numbers didn’t match the Met Office’s. And his numbers weren’t off by a constant factor or constant difference. They followed a similar pattern to the Met Office’s, but they were not correct. At first I assumed Ilya was wrong and so I checked and double checked has calculations. His calculations looked right; the Met Office numbers looked wrong.
Then I wrote out the mathematics from the Brohan et al. paper and looked for where the error could be. And I found the source. I quickly emailed Ilya and boarded the plane to dream of CRUTEM and HadCRUT as I tried to sleep upright.
Read the details at JGC’s blog: Something odd in the CRUTEM3 station errors
Pamela Gray (10:01:13) :
Oh good heavens Tamino. You are in need of a strong-tempered mother willing to put your stingy behind, after a good paddle, in a corner. Now share your candy. There is one word we mothers use for children who behave like this. Snot.
Great Visual. Reminds me of an incident involving my cousins, I, and
Granma’s cookie jar.For a crippled Cherokee woman she could move fast
and was creative with her cane..
To share is good-don’t be afraid.Being dragged to the front porch by the
ear, now that’s real fear…
Alan the Brit (10:16:50) :
…with a new £30M XBox360 that does 2 billion calculations per second (allegedly)…
Well, there’s the problem right there! You need Sony PlayStation 3’s for decent supercomputing. The US Air Force thinks so, therefore it must be true.
For a readable review of calculating error see the following link. Determinate errors are what I refer to as sloppy math, sloppy data entry, biased assumptions ending up as mathematical calculations, etc. For example, after I entered my data into the computer spread sheet, I rechecked each entry with the original recorded numbers to make sure I had not made a “sloppy work”, IE determinate, error.
Indeterminate errors would happen when the environment in which the data is taken changes slightly (or even greatly). To reduce indeterminate errors, repeated measures are sometime used which should reduce the calculated error value. I did this by taking the same auditory brainstem response reading repeatedly (IE 1500 beeps, 1500 readings). The screen that displayed the electrical brainstem response clearly demonstrated that the more often a reading was taken, the clearer and more stable the signal response became. I was reducing indeterminate errors.
The land-based temperature data set has so far been peppered with both kinds of errors.
http://www.lhup.edu/~dsimanek/errors.htm
The Times reports that the BBC is considering dumping weather forecasting from the Met Office and is out to tendering other weather services. Mind you that is a case of the pot (The BBC) calling the kettle (the Metoffice) black!
Addendum
The ABR equipment included computer code that would average responses together after each one was taken, thus calculating the electrical response with a finer and finer error number as repeated samples were taken. It wasn’t that the brainstem itself was getting better at it the longer it was subjected to the beeps.
JMurphy (09:27:58) :
REPLY: I had a look, but reproducing will be rather difficult when he stakes stands like this:
Tamino’s response to request for code from Zeke:
[Response: I haven’t decided whether or not to publish this (peer reviewed). If I don’t I’ll probably make the code available to those who I consider serious investigators. That does not include denialists.]
Tony; I believe you’ll get the chance;
From;
http://tamino.wordpress.com/2010/02/25/false-claims-proven-false/#comment-39807;
Bob Kutz // February 25, 2010 at 6:45 pm | Reply
I won’t sink to comments such as yours, I will again ask; can I see your work?
[Response: (from Tamino, presumbly) Other than the page caption, the words “GHCN” and “global” don’t appear on that page. And the caption to the graph you linked is “DIFFERENCE BETWEEN RAW AND FINAL USHCN DATA SETS.”
As for seeing my work, you bet your ass you can. I’ve decided to publish.]
I think that’s really his way of saying; I’ll hold it private until it’s published, which is certainly his right to do. We’ll see.
Give him credit though; at least he didn’t delete my comments, as RC would have.
I’d like to know if this error works in favor of the IPCC and AGW or against. Seems like most ‘errors’ have been in their favor.
It seems to me that using ‘95% confidence limits’, +/-2 sigma, exaggerates the quality of the data by showing only 2/3rds of the natural variance. Another way to use sigma is to point out that +/- 3 sigma is considered normal distribution, wherein 99.7% of natural occurrences will fall. A lot of manufacturing processes became very successful by controlling process variance +/- 3 sigma to within output specifications. It is not rational to place a specification (CO2 restrictions) on a process with greater inherent variation than the specifications and further expect (bank on) an outcome within specifications. Not something I would invest in.
So, can you answer the following question?
Have you or any one you know performed a statistical analysis that contradicts Tamino’s claim:
“The claim that the station dropout is responsible for any, let alone most, of the modern warming trend, is utterly, demonstrably, provably false. The claim that adjustments introduced by analysis centers such as NASA GISS have introduced false warming is utterly, demonstrably, provably false.”
What I’m looking for is evidence of any reproducible analysis that supports the claim that the observed warming trend is a result of station dropout and/or adjustments.
Stating that one would need to have access to Tamino’s code is not applicable as stated by Paul Daniel Ash above.
One would assume that since you have been stating the opposite from Tamnio (and others) for some time that you have performed or have access to someone who has performed this analysis.
It’s a simple question with a simple answer: Yes I have or No I haven’t.
REPLY:Two things. 1) I’ve been studying USHCN, not GHCN, different networks 2) Yes it is a simple question, but I’m not the person to answer it. E.M. Smith is the one that raised this claim and did all the GHCN analysis, and I’ve sent him an email with the link and the question. I’m sure he’ll respond either here or at his blog. http://chiefio.wordpress.com
Your defense of the phantom researcher “Tamino” withholding code is rubbish. How well would “I’ve disproved you but I won’t show you how, nor will I give my name” hold up in a court of law? Not at all.
-A
Coming soon in CRUTEMP4… the 30’s now the coldest decade on record
/sarcoff
But seriously, is there any way to plot the delta trends between the different versions of CRU and GISS. Seems like every few years the 30’s get colder and colder.
OT – off topic but I wonder how the folk at http://www.skepticalscience.com/ with their handy iPhone app will rebut this, actually I don’t think I’ve seen a more vile AGW propoganda site or app
Pamela Gray (10:49:38) :
Astronomical imagers perform the same kind of IE reduction, taking multiple images through a noisy atmosphere to improve the signal/noise ratio.
They even ‘dither’ about, moving the centerpoint of the camera with respect to the object to smooth out local atmospheric disturbances and pixel sensitivities.
You can do the same with climate, taking several nearby stations to get a signal means. Erratic readings and missing data can be suppressed in such a manner.
The ratio of rural to urban stations should accurately reflect the current land use of any area, to suppress the noise of land use on the climactic signal.
Current GISS practice of taking a single station to represent a large area is the opposite and noisiest way imaginable to represent climate.
“I couldn’t prove that opposing opinions are moderated out, but have a gut feeling they are.”
Anyone who has spent any time there posting opposing opinions know they are frequently moderated out and selectively-edited (before the poster is banned, of course).
Anthony,
In this case replicating what Tamino did wouldn’t be that difficult. There are two primary components that you would need to figure out:
1) How to undertake the spatial weighting. While this is a bit beyond my programming ability (hence my interest in Tamino’s script), the basic idea is simple: split up the world into grid cells based on lat/lon, and calculate the average anomaly of stations in the “pre-cutoff” and “post-cutoff” series (or calculate the average anomaly of all stations in each grid to compare with GISS et al). Weight each grid cell by its area to determine the contribution to global temp anomalies. This leads us to…
2) How to combine station records within grid cells. Here Tamino developed his own “optimal” method (described briefly here: http://tamino.wordpress.com/2010/02/08/combining-stations/) that would be difficult to replicate at present, since he intends to publish a paper with this as the primary innovation and doesn’t want to jump the gun (something you should be empathize with given your similar position on surfacestations). However, Chad has been nice enough to outline a number of other methods of combining stations over at his blog (http://treesfortheforest.wordpress.com/) that you could easily take advantage of.
Its a bit more work than looking at individual stations or simply averaging anomalies, but it is the “correct” way to go about a rigorous analysis to see if the differing number of stations available in GHCN over time introduces any bias into the global temperature record. Once you do it, you can also easily dive into a regional analysis of station availability, checking to see if the much-touted “Bolivia” and “Canada” station data availability issues over the past decade have any real effect by comparing grid cells with both “pre-cutoff” and “post-cutoff” stations over the period where reports are available for both.
REPLY: Well like I keep telling everyone else, GHCN is E.M. Smith’s specialty, and I’ve notified him. Like you, it’s a bit beyond my programming ability to replicate and I wouldn’t undertake it now anyway, since I’m busy collaborating on the USHCN paper with co-authors. Let’s see what Mr. Smith (who does have that programming skill) has to say about it. – A
On spatial weighting, it might be easy to just break the world up into grid cells of 4 deg of latitude by 5 deg longitude. Per Gavin, the approximate area of each grid cell would be:
grid_area = 4 * pi^2 * 6378.1^2 * cos(lat*pi/180) * 5/360 * 4/360
where lat is the latitude of the grid cell in question in degrees.
er, I meant degrees in the previous post, not radians. Mind editing oh helpful moderator?
That was also my take on a single station measured just once a day for an entire grid instead of several nearby stations measured at the same time once a day and then averaged together with the lower error calculation available. The indeterminate noisy error of one station is copied by every grid filled in with the data from this one sensor. Sloppy research design.
Anthony, Pamela. Take half a teaspoon of bicarbonate of soda (baking soda, an alkaliser) in a little water, four times on day one. The same twice on day two. If you should really need it by then, once on day three. Cold gone. It will work equally well on full blown ‘flu. Courtesy of my grandmother, some seventy odd years ago. It works just as certainly now as it did back then.
JMurphy (09:27:58) :
WRT Tamino.
here is a simple test for you.
Visit this site:
http://statpad.wordpress.com/
This site is run by a published statistics professor.
he reviewed tamino’s work and had a question about Tamino’s method.
he attempted to post a comment.
He was banned.
I attempted to post a comment. ” tamino have you looked at this?”
THAT was banned.
so you try.
In fact EVERYONE should try this. Send a simple note to tamino.
Tamino, have you read this blog by the statistics professor.
http://statpad.wordpress.com/
AND WHEN TAMINO REFUSES TO LET YOUR QUESTION GET THROUGH YOU HAVE A GOOD INDICATION ABOUT THE OPENNESS OF HIS MIND.
I’m sure he wont mind the traffic.
Go ask. everybody ask.
Heck, Tell people on every CAGW site that you asked, have them ask.
Re: sturat (Feb 25 11:21),
It is just as well that Tamino is withholding his code, because that way he does not have to defend the use of his “optimal” method of calculating grid temperatures.
I tried to discuss this with him by leaving a comment on his web site, but the comment got deleted. After I wrote a post discussing his method, he again refused to discuss the methodology.
I guess we’ll have to wait until he publishes it in a (climate science) peer reviewed journal. 😉
“Tamino’s response to request for code from Zeke:
[Response: I haven’t decided whether or not to publish this (peer reviewed). If I don’t I’ll probably make the code available to those who I consider serious investigators. That does not include denialists.]”
Can I ask a question please, Tamino?
can I ask a question?
Have you read RomanM’s site?
Did you read his comment that you banned?
I think Grant Foster has put himself in a funny spot here.
Grant Foster is Tamino.
If Grant decides to publish his paper he has to make a decision.
Does he change his analysis in light of Roman’s work or does he ignore it?
A. If he changes his work, then he has to credit Roman. To make
sure of this people need to alert Tamino to this. If he publishes
we will remember. We will raise the issue of attribution. We will
look at the guidelines from his employer. So, people need to make
this issue more public. Now.
B. If he doesnt change his approach, then I suppose roman will have a
nice comment to make or somebody can use his work to do a nice
paper.
If Grant doesnt publish a paper, we can speculate as to why.
Zeke Hausfather (12:22:17) :
here is a simple request. If you havent read Roman’s work please have a read.
then, ask Tamino what he thinks. include a link to Roman’s site in your request.
That’s a fair request. can you do that? if not, why not?
“…Here Tamino developed his own ‘optimal’ method…”
Oh, developing his own method? At least it’s not called ‘robust.”
I see Tamino has a new comment:
“…As for me, I’ve seen lots of people do an analysis which is wrong. But none have come close to doing it right. In particular, none seems to pay attention to the crucial area-weighting step…”
In other words, his way is the right way, and everyone else is wrong.
It’s interesting that he notes, “It’ll be more work to finish the southern hemisphere.” He admittedly hasn’t even finished, and he’s already got his conclusion.
Steve M. (10:25:37) :
Do I trust “Tamino”, a man supposedly of science but who won’t put his name to his critcisms, who regularly denigrates others, and who now won’t share what he claims falsifies the work of people who do put their name to their work? In a word, no. -A
“I was thinking the same thing. It’s good to see that you openly put your name on this site, along with your credentials. I read through the comments on Tamino’s site, and not that it’s all cheer-leading. I couldn’t prove that opposing opinions are moderated out, but have a gut feeling they are.”
They are moderating us out, I have posted 6 times within 1 hr, lo and behold not one of my comments posted. Read what I was posting below:
I find your claims to be very bold. The fact you are resorting to name calling, however , is very bullish. I know that there is not one scientist, skeptical or not, who would ever say the Holocaust did not happen/was a non-event. When you use such words toward those that are skeptical it is very offensive and very offensive for those that hear it being used that are not skeptical. So, please do everybody a favor and retract the word “Denialists” from your paper. Otherwise, in my book weather I agree with you or not, you sound like a heartless, and racist weather man. Besides, the skeptics are not out to destroy the science, they are trying to help you get it right. The Science is not even 5% perfectly understood. They are your counter part parts in a system of checks and balances. So, again, refrain from bringing your self down to the MSM, Greep Peace, Eco-Terrorists’s level. You are better than that. We all are.
I have also posted this:
Zach // February 25, 2010 at 2:05 pm | Reply………………………………………”and that their preferred politicians and commentators say it’s a reasonable position.”
I have to disagree with you on this. According to the constitution politicians do not tell the people what the issues are and if they are right or wrong. The consistution clearly gives power to the people to decide what the issues of the day are and what the government should do about it. When I say the people, I mean normal every day average citizens, not a small group of scientists. The Scientists are wrong on both sides of the fence as well. They should not be trying to convince the Government about anything, unless the citizens on a majority vote want the scientists to be heard by the congress, then the Judicial Branch will determine the legality of any plan of action to be taken by the government