I was driving in the middle of Nevada when all this happened, and was offline the entire time. So I can’t claim any credit here. But, it sure is nice to see that the collection of people who visit and post here have had an impact. I offer my particular thanks to WUWT contributor John Goetz. It seems Gavin agrees. I’m up early this morning on my trip back home, and I was quite surprised to find this on RC. See this comment from Gavin Schmidt on Real Climate:
You and McIntyre are mistaken. The first intimation of a problem was posted on Watt’s blog in the comments by ‘Chris’ at around 4pm EST. By 7pm in those comments John Goetz had confirmed that the NOAA file was the problem. Notifications to the GISTEMP team of a problem started arriving very shortly after that, and I personally emailed them that evening. However, no action was taken until the next morning because people had left work already. They had decided to take the analysis down before 8.14am (email time stamp to me) since the overnight update to the NOAA file (uploaded 4.30am) had not fixed the problem. McIntyre’s intervention sometime that morning is neither here nor there. Possibly he should consider that he is not the only person in the world with email, nor is he the only person that can read. The credit for first spotting this goes to the commentators on WUWT, and the first notification to GISTEMP was that evening. – gavin
John Goetz writes later in RC comments, it appears that Steve McIntyre at least raised the consciousnous level with his first blog posting:
For what it is worth, Chris posted his discovery on WUWT about 45 minutes before I made my update indicating an error existed. However, I made my posting because of two emails Steve Mc sent me about two hours prior. The first was the email John S. sent him, quickly followed by a confirmation from Steve. I simply had not checked email due to being busy with work. Steve had already written most of his post by the time I saw the emails.
Not sure it really matters who was there first. I am ashamed to say I saw the big red blotch in central Asia and was so insensitized that I did not investigate it further. Perhaps I’ve lost my critical eye.
Despite some commenters there at RC referring to WUWT in less than glowing terms, I’m pleased that this blog has been a vehicle for, ahem, “climate change”. 😉 Being first really isn’t all that important, getting the data right is.
Quality control is the issue. It is the reason that I started the surfacestations.org project. If after QC for climate data has been suitably addressed with a standards compliant methodology, such as ISO-8000, I’ll be satisfied with the final data. Right now I’m not convinced that the surface data is truly representative.
Gavin, if you’d like to do a guest post here on this error, the floor is yours. Just leave a comment or drop an email. – Anthony
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Thanks for the input and corrections George,
So raw data is passed from outlying weather stations covering a given grid to regional centres to be passed on to a central point, this is basic raw data that is then put through a computer process at GISS to highlight possible climate anomalies/variation/trends which is then used by organisations like the IPCC to formulate models which then influence policy makers, is that a fair basic summary?
The fancy computer cartoons so beloved of NGOs/MSM/governments are actually all derived from thousands of individual weather stations recording actual weather characteristics measured by non standard and unverified equipment with unknown people with unknown qualifications that are not checked or screened by the GISS which takes these readings as gospel and distributes the NASA GISS processed conclusions to AGW/MMCC supporting institutions in the form of climate anomaly maps which they then use to inform the masses of the urgent need for draconian laws limiting human behaviour?
Forgive my slowness in grasping the bigger picture BUT does the above complicated process seem reliable? Is it likely that errors can creep in unoticed giving unreliable readings? From heat islands to poor siting to old badly maintained/non standard equipment to unknown and unchecked data contributers doesnt it seem likely that the GISS cartoons are as reliable a guide to climate as the whacky races are to formula one?
So in fact daily weather reports compiled over time is a guide to climate variation? If so, the tens of thousands of Royal/Merchant navy logs going back centuries would be a goldmine to any climatologist wouldnt it? after all, the great oceans seem to be poorly represented in the GISS cartoons.
Wow I’m famous!
Happy to be recognised as first to post about the errors.
However, can I say for the record that I realise by the time I’d noticed them, the process of investigating and reporting them appears to have been already underway at both CA and WUWT, and in any event without the dedication of Steve McIntyre and Anthony Watts in maintaining thoughtful, probing and credible websites, someone like myself would not have been in a position to make such a post in the first place.
I can imagine the derision of some RC posters at my use of the words “thoughtful, probing and credible” above. Well, it’s not a zero sum game. RC is a thoughtful, probing and credible website as well, and is the best at what it does, just as CA and WUWT are good at their own distinctive specialities. My perception is that RC is diminished by its attitude towards Steve McIntyre, amongst other things, but I guess that’s between certain people and for them to thrash out between themselves.
Les Johnson (14:07:16)
I did it as well. If they spent some of the money wasted on that survey working on gathering and reporting temperatures properly now that would make a difference. Shocking how our tax dollars are spent. And we voted for more!
Anthony,
I just noticed your inline comment above. It’s true that there were very few stations with CRN ratings of 1 or 2. However, when I extended the analysis to the approximately 50 rural stations with CRN ratings of 1, 2, or 3 the results were virtually unchanged.
I do not claim that my analysis is definitive. It is certainly much more robust than many of the negative claims made about GISTEMP around here.
—
jared:
My analysis was restricted to the USA lower-48 since that is the only region that has station ratings that I am aware of. Credit to Surface Stations for doing the ratings.
—
moptop:
You are mis-reading the graphs. Simple as that. There is very little difference between the CRN12R and GISTEMP trends.
If someone could be off assistance:
I’m trying to look at the difference between UAH located at http://vortex.nsstc.uah.edu/public/msu/t2lt/tltglhmam_5.2 and GISTEMP located at http://data.giss.nasa.gov/gistemp/tabledata/GLB.Ts+dSST.txt.
1. Are these the best sources to use?
2. It is my understanding that the base period for UAH is 1979 thru 1998 and GISTEMP is 1951 thru 1980. Is that correct?
3. In converting GISTEMP to base period 1979 thru 1998 I averaged to two ranges using GISTEMP numbers and subtracted the difference (0.238) from the GISTEMP anomaly. Does that make sense?
4. Doesn’t GISTEMP change their history on a regular basis?
What I want to see is a trend line on the difference between GISTEMP and UAH.
I’m sure it has been done many times before but it’s fun to get one’s keyboard dirty.
Re Steve McIntyre’s complaint about Gavin Scmidt —
Sheesh, could McIntyre get any more petty? This is the state of climate science and debate? McIntyre would do well examine his own blog for gratuitous insults before whining about any tossed his way.
A third for mosh.
Gavin (07:56:17) :
Your title is incorrect. I have no connection to GISTEMP other than the fact they work out of the same building.
From the April 2, 2006 New York Times: Dr. Schmidt runs a global climate model, called ModelE, out of the NASA Goddard Institute for Space Studies, a part of the Earth Institute at Columbia University.
I wonder whether Dr. Schmidt corrected the Times?
Harold Ambler:
GISTEMP is *different* than ModelE.
“Sheesh, could McIntyre get any more petty?”
The only pettiness I see is that of Schmidt. Mcintyre was trying to help a public servant to better do his job. I suppose McIntyre should have known that there is no help for this organization.
Harold Ambler (16:38:56) I think he means he’s not formally or organizationally directly connected to GIStemp. His reply is disingenuous, though, as Anthony demonstrates, and he’s certainly more connected to GIStemp than am I.
===============================
Well, okay, okay, guys, not to be petty or nothin’, but since GS and SM are present and have actually posted, we should not go too far in our personal commentary.
We would be better served by limiting ourselves to criticism rather than characterization.
Besides, I will snip you!
~ Evan
Agreed, Evan. Delete my last post, please. It crossed the border.
[REPLY – Done. Well spoken. And hats off. I wouldn’t have otherwise. I considered it teetering on the edge. (I even agreed with some of the points.) But best to maintain our decorum. ~ Evan]
Actually, I for one, am quite pleased Dr. Schmidt has ventured over here. Maybe this could be the start of a thaw in the bitter relations between the two camps.
That said, it would be nice if he could refrain from classifying everyone not in his camp as “deniers”. Many (most) of us don’t deny CO2 is a greenhouse gas, or that it has a effect on the climate. The main disagreements are either on the magnitude of the effect, or on the lack of openness of the methodologies.
We are skeptics. Yes, sometimes we err, but as is shown with this recent episode, so do the folks on the RC side of the fence.
Mike “sonicfrog” Alexander.
(geology school drop-out)
@Bill M,
I would be a tad snarky too – if I had to spend my professional life defending a pile of rotten garbage.
Yes.
And I would also highly encourage Dr. Schmidt to allow Anthony to post over on RC without getting zapped. It’s not as if he doesn’t have a tremendous amount to contribute.
He’s such a good man. He’s been so fair, impartial, and reasonable for such a long time, and he gets such poor return for it. Makes me mad.
As I an effective member of the peanut gallery. I can say that their vacancy is assured.
I’m an idiot.
Mr Gavin, we are equal.
Nobody can be so arrogant.
FM
[REPLY – Brace up. We’ll win the day on the merits. ~ Evan]
I probably should not have been sarcastic. Direct, in the blogosphere and everywhere else, is typically better.
That said, Dr. Schmidt, whose salary is paid by American tax dollars, has written extensively at Real Climate, fluently commenting on GISS and GISTEMP as though they were part of one larger entity, which they are. The statement “I have no connection to GISTEMP” is a surprising one.
For background on Dr. Schmidt’s own conflation of the two when it served his purposes, see http://www.realclimate.org/index.php/archives/2007/08/1934-and-all-that/
In terms of what the two have in common, GISS and GISTEMP, the list is short but important: Both funded by U.S. taxpayers, both run by James Hansen, both part of NASA, both in New York City, both in same building. Dr. Schmidt’s comments on Real Climate make it clear that the people at GISTEMP are no strangers to him.
All that said, Anthony’s headline change makes sense.
There is no way even this corrected data is correct. It is way, way off.
Look at the GISS Zonal temp anomaly map. The south pole has an anomaly of -0.5 and the north pole has an anomaly of +3.5C with a general gradient going up as one moves from south to north.
http://data.giss.nasa.gov/work/gistemp/NMAPS/tmp_GHCN_GISS_1200km_Anom10_2008_2008_1951_1980/GHCN_GISS_1200km_Anom10_2008_2008_1951_1980_zonal.gif
UAH sat data only has a 0.17C difference between the hemispheres whereas GISS would be 1.25C or so.
JohnV
Does your graph take into account the fact that NOAA’s “QC” has located and corrected many stations in “cool pools” while missing the one next to a burn barrel and the other 70 percent (so far) poorly sighted stations. (see NOAA’s website for QC reference)
evanjones (17:31:30) : Yes.
BUT I would also highly encourage Dr. Schmidt to allow Anthony to post over on RC without getting zapped.
I agree with you Evan. furthering science is all about debating points and getting to the evidence, not covering ones own point of view based on what one believes. In science, everyone is a skeptic 😉
[REPLY – So true. ~ Evan]
Evan, If you feel I have crossed the line feel free to remove or snip as you wish.
Mike Bryant
[REPLY – Better so. I feel your pain, and I hate to do in your prose. But . . . better so. It speaks well of you that you are so decent about it. ~ Evan]
“”Cassandra King (14:35:46) :
Thanks for the input and corrections George, “”
Cassandra, one of the problems here is that a lot of us don’t know who on earth we all are, so it is easy to talk down (or up) to somebody else. For all I know, you may have four PhDs in Astrophysics; Thermodynamics, Molecular Spectroscoipy, and Cosmic Radiation; well I don’t have any of those just a BSc in Physics and Maths plus 48 years as a practising physicist in industry.
But back to the GISStemp. Clearly the original data is some sort of thermometer reading/s from some set of formal measuring stations; all of which I know nowt about; but Anthony has been snooping in on some of these places, and some are like gold rush town horseshoe foundries.
But suppose you measure a temperature (of the ground) somewhere; bearing in mind, that 73% of the ground is actually the ocean. According to physical theory, every body at a temperature above absolute zero, radiates electromagnetic radiation energy, which will result in cooling that body. That radiation is limited by a theoretical absolute maximum radiation of a type known as “Black Body Radiation”, which I can tell you has nothing to do with Eartha Kitt. BB radiation is one of the miracles of modern Physics, and the first correct calculations of its properties was the very start of the era of quantum mechanics and quantum physics.
It turns out that a good deal of the earth, particularly the oceans actually radiate relatively closely to the Black body limit. What we know about BB radiation is that the Total energy radiated per second, is proportional to the fourth power of the absolute temperature (Kelvins).
So here we have the very first problem with GISStemp. We measure a temperature; say on your front doorstep; but the heat energy being lost to the rest of the universe from that step depends on the 4th power of that temperature; not on the temperature itself. So what purpose is served by averaging temperatures.
Now the so-called mean Global temperature is about 15 C or about 288K, and we know that the BB radiation at that temperature is about 390 Watts per square meter. The coldest earth temperatures, at Vostock Station in Antarctica get down to -90C which is 183 K, so the maximum radiation rate can only be (183/288)^4 x 390 or 63.6 Watts per square meter. At the other end, in tropical hot deserts, the air temperatures can get to about +60C, and the ground even hotter, and that is 343 K, so the radiation rate can get up to (343/288)^4 x 390 or 784.6 Watts per square meter. So that is a range of 12.3 times from coldest to hottest temperature locations. Clearly the very hottest places on earth are actually cooling the planet fastest, and the polar regions are doing very little to cool the earth, and the hot deserts can be cooling the earth at double the rate corresponding to the global mean temperature.
So you can see that averaging all the points on earth to get an average temperature is not very useful for determining whether we are heating up or cooling down.
When it comes to the influence of Green House Gases such as CO2, the problem gets a lot more complicated.
Not only does the theroy of Black Body Radiation specify the total radiation being emitted from an ideal black body, but it completely specifies the spectrum of wavelengths that are emitted. I should add that a “Black body” in this sense is a body that completely absorbs ALL EM radiation that falls on it, from the longest wavelength radio waves to the shortest wavelength gamma rays and even cosmic radiation. We can make a laboratory gizmo that very closely approximates a black body; they typically are very well thermally insulated cavities that optically trap radiation that enters their “aperture”. At a particular temperature, that aperture emits a radiation pattern which we can completely describe with very high precision.
The deep oceans, because they absorb virtually all the radiation that strikes them, are fairly good approximations to a black body.
The other wonderful property of a BB, is that the spectrum of the emitted radiation is completely and very accurately described by Max Planck’s BB radiation law; it is one of the most accurate physics theories we have, and the wavelength at which the emission is maximum is inversely proportional to the absolute temperature.
So the sun at about 6000 K emits a spectrum that peaks at about 0.5 microns wavelength in the green region. At 300 K, which is close to the earth temperature (288), the emitted spectrum peaks at 20 times that for the sun or 10 microns, and it is about 10.1 microns at 288 K, and that is the mystery Infra Red radiation that GHG warming is all about.
Well at 343K (60C) the peak wavelength is about 3000/343 or 8.75 microns, while at 183K it peaks at 3000/183 or 16.4 microns.
Well CO2 does its thing at about 14 microns or so, and actually absorbs well from about 13.5 to 16.5 microns, so CO2 is more effective in the polar regions than in the tropics. On the other hand Ozone absorbs at 9-10 microns so it slams even tropical IR emissions, but CO2 is much less effective. This spectral peak shift is called the Wien’s displacement law, and the other important point is that the value of the radiation rate at the spectral peak actually varies as the 5th power of the absolute temperature, whereas the total radiation varies as the 4th power.
Then you have to consider that over the oceans, you get evaporation, vertical convection in the ocean water, conduction between water and air, and vertical convection of the heated air, a swell as the thermal radiation. Over a hot dry desert you dont’ get any of the evaporative cooling effects, so the relation between temperature and heat energy emission is quite different than for the ocean, and different again over a tropical rain forest or alpine meadow.
So you see, the temperature tells you nothing much of value and the average temperature tells you even less.
And if all of this is what you wrote your PhD thesis on; please accept my humble apologies; hopefully it is useful for somebody else who may be lurking here.
But that is why I am not too anamoured with GISStemp. And there are other problems with it which are almost too gory to talk about.
George
Anthony,
Thanks for putting this blog together. I’ve alerted my brother to its existence and he will be doing a couple of station surveys for you during the holidays when he heads to Alice, Texas. I look forward to seeing his results on your surfacestations.org page.
‘Many (most) of us don’t deny CO2 is a greenhouse gas, or that it has a effect on the climate.’
Really? One wouldn’t think so reading the articles and comments on this blog, which I do regularly. In fact, many times it leads me to believe that mainstream climate science is one big lie. I find it hard to believe that so many smart people are completely wrong about AGW, but then again I find it hard to believe as well that people are more worried about billions of dollars and millions of lives potentially wasted due to AGW-policy, when billions of dollars and millions of lives are being wasted as we speak in places such as Iraq or Afghanistan.
This blog has more influence than people think. The bigger the influence, the greater the responsibility. I truly hope AGW is non-existent but if it isn’t I wouldn’t like to be in the shoes of a Mr Watts or McIntyre, ethically speaking.