Temperature is such a simple finite thing. It is amazing how complex people can make it.
– commenter and friend of WUWT, ossqss at Judith Curry’s blog
Sometimes, you can believe you are entirely right while simultaneously believing that you’ve done due diligence. That’s what confirmation bias is all about. In this case, a whole bunch of people, including me, got a severe case of it.
I’m talking about the claim made by Steve Goddard that 40% of the USHCN data is “fabricated”. which I and few other people thought was clearly wrong.
Dr. Judith Curry and I have been conversing a lot via email over the past two days, and she has written an illuminating essay that explores the issue raised by Goddard and the sociology going on. See her essay:
http://judithcurry.com/2014/06/28/skeptical-of-skeptics-is-steve-goddard-right/
Steve Goddard aka Tony Heller deserves the credit for the initial finding, Paul Homewood deserves the credit for taking the finding and establishing it in a more comprehensible
way that opened closed eyes, including mine, in this post entitled Massive Temperature Adjustments At Luling, Texas. Along with that is his latest followup, showing the problem isn’t limited to Texas, but also in Kansas. And there’s more about this below.
Goddard early on (June 2) gave me his source code that made his graph, but I
couldn’t get it to compile and run. That’s probably more my fault than his, as I’m not an expert in C++ computer language. Had I been able to, things might have gone differently. Then there was the fact that the problem Goddard noted doesn’t show up in GHCN data and I didn’t see the problem in any of the data we had for our USHCN surface stations analysis.
But, the thing that really put up a wall for me was this moment on June 1st, shortly after getting Goddard’s first email with his finding, which I pointed out in On ‘denying’ Hockey Sticks, USHCN data, and all that – part 1.
Goddard initially claimed 40% of the STATIONS were missing, which I said right away was not possible. It raised my hackles, and prompted my “you need to do better” statement. Then he switched the text in his post from stations to data while I was away for a couple of hours at my daughter’s music recital. When I returned, I noted the change, with no note of the change on his post, and that is what really put up the wall for me. He probably looked at it like he was just fixing a typo, I looked at it like it was sweeping an important distinction under the rug.
Then there was my personal bias over previous episodes where Goddard had made what I considered grievous errors, and refused to admit to them. There was the claim of CO2 freezing out of the air in Antarctica episode, later shown to be impossible by an experiment and the GISStimating 1998 episode, and the comment where when the old data is checked and it is clear Goddard/Heller’s claim doesn’t hold up.
And then just over a month ago there was Goddard’s first hockey stick shape in the USHCN data set, which turned out to be nothing but an artifact.
All of that added up to a big heap of confirmation bias, I was so used to Goddard being wrong, I expected it again, but this time Steve Goddard was right and my confirmation bias prevented me from seeing that there was in fact a real issue in the data and that NCDC has dead stations that are reporting data that isn’t real: mea culpa.
But, that’s the same problem many climate scientists have, they are used to some skeptics being wrong on some issues, so they put up a wall. That is why the careful and exacting analyses we see from Steve McIntyre should be a model for us all. We have to “do better” to make sure that claims we make are credible, documented, phrased in non-inflammatory language, understandable, and most importantly, right.
Otherwise, walls go up, confirmation bias sets in.
Now that the wall is down, NCDC won’t be able to ignore this, even John Nielsen-Gammon, who was critical of Goddard along with me in the Polifact story now says there is a real problem. So does Zeke, and we have all sent or forwarded email to NCDC advising them of it.
I’ve also been on the phone Friday with the assistant director of NCDC and chief scientist (Tom Peterson), and also with the person in charge of USHCN (Matt Menne). Both were quality, professional conversations, and both thanked me for bringing it to their attention. There is lots of email flying back and forth too.
They are taking this seriously, they have to, as final data as currently presented for USHCN is clearly wrong. John Neilsen-Gammon sent me a cursory analysis for Texas USHCN stations, noting he found a number of stations that had “estimated” data in place of actual good data that NCDC has in hand, and appears in the RAW USHCN data file on their FTP site
From:John Nielsen-Gammon Sent: Friday, June 27, 2014 9:27 AM To: Anthony Subject: Re: USHCN station at Luling Texas
Anthony –
I just did a check of all Texas USHCN stations. Thirteen had estimates in place of apparently good data.
410174 Estimated May 2008 thru June 2009
410498 Estimated since Oct 2011
410639 Estimated since July 2012 (exc Feb-Mar 2012, Nov 2012, Mar 2013, and May 2013)
410902 Estimated since Aug 2013
411048 Estimated July 2012 thru Feb 2014
412906 Estimated since Jan 2013
413240 Estimated since March 2013
413280 Estimated since Oct 2012
415018 Estimated since April 2010, defunct since Dec 2012
415429 Estimated since May 2013
416276 Estimated since Nov 2012
417945 Estimated since May 2013
418201Estimated since April 2013 (exc Dec 2013).
What is going on is that the USHCN code is that while the RAW data file has the actual measurements, for some reason the final data they publish doesn’t get the memo that good data is actually present for these stations, so it “infills” it with estimated data using data from surrounding stations. It’s a bug, a big one. And as Zeke did a cursory analysis Thursday night, he discovered it was systemic to the entire record, and up to 10% of stations have “estimated” data spanning over a century:
And here is the real kicker, “Zombie weather stations” exist in the USHCN final data set that are still generating data, even though they have been closed.
Remember Marysville, CA, the poster child for bad station siting? It was the station that gave me my “light bulb moment” on the issue of station siting. Here is a photo I took in May 2007:
![marysville_badsiting[1]](http://wattsupwiththat.files.wordpress.com/2014/06/marysville_badsiting1.jpg?quality=83)
It was closed just a couple of months after I introduced it to the world as the prime example of “How not to measure temperature”. The MMTS sensor was in a parking lot, with hot air from a/c units from the nearby electronics sheds for the cell phone tower:
Guess what? Like Luling, TX, which is still open, but getting estimated data in place of the actual data in the final USHCN data file, even though it was marked closed in 2007 by NOAA’s own metadata, Marysville is still producing estimated monthly data, marked with an “E” flag:
USH00045385 2006 1034E 1156h 1036g 1501h 2166i 2601E 2905E 2494E 2314E 1741E 1298E 848i 0 USH00045385 2007 797c 1151E 1575i 1701E 2159E 2418E 2628E 2620E 2197E 1711E 1408E 846E 0 USH00045385 2008 836E 1064E 1386E 1610E 2146E 2508E 2686E 2658E 2383E 1906E 1427E 750E 0 USH00045385 2009 969E 1092E 1316E 1641E 2238E 2354E 2685E 2583E 2519E 1739E 1272E 809E 0 USH00045385 2010 951E 1190E 1302E 1379E 1746E 2401E 2617E 2427E 2340E 1904E 1255E 1073E 0 USH00045385 2011 831E 991E 1228E 1565E 1792E 2223E 2558E 2536E 2511E 1853E 1161E 867E 0 USH00045385 2012 978E 1161E 1229E 1646E 2147E 2387E 2597E 2660E 2454E 1931E 1383E 928E 0 USH00045385 2013 820E 1062E 1494E 1864E 2199E 2480E 2759E 2568E 2286E 1807E 1396E 844E 0 USH00045385 2014 1188E 1247E 1553E 1777E 2245E 2526E -9999 -9999 -9999 -9999 -9999 -9999
Source: USHCN Final : ushcn.tavg.latest.FLs.52i.tar.gz
Compare to USHCN Raw : ushcn.tavg.latest.raw.tar.gz
In the USHCN V2.5 folder, the readme file describes the “E” flag as:
E = a monthly value could not be computed from daily data. The value is estimated using values from surrounding stations
There are quite a few “zombie weather stations” in the USHCN final dataset, possibly up to 25% out of the 1218 that is the total number of stations. In my conversations with NCDC on Friday, I’m told these were kept in and “reporting” as a policy decision to provide a “continuity” of data for scientific purposes. While there “might” be some justification for that sort of thinking, few people know about it there’s no disclaimer or caveat in the USHCN FTP folder at NCDC or in the readme file that describes this, they “hint” at it saying:
The composition of the network remains unchanged at 1218 stations
But that really isn’t true, as some USHCN stations out of the 1218 have been closed and are no longer reporting real data, but instead are reporting estimated data.
NCDC really should make this clear, and while it “might” be OK to produce a datafile that has estimated data in it, not everyone is going to understand what that means, and that the stations that have been long dead are producing estimated data. NCDC has failed in notifying the public, and even their colleagues of this. Even the Texas State Climatologist John Nielsen-Gammon didn’t know about these “zombie” stations until I showed him. If he had known, his opinion might have been different on the Goddard issue. When even professional people in your sphere of influence don’t know you are doing dead weather station data infills like this, you can be sure that your primary mission to provide useful data is FUBAR.
NCDC needs to step up and fix this along with other problems that have been identified.
And they are, I expect some sort of a statement, and possibly a correction next week. In the meantime, let’s let them do their work and go through their methodology. It will not be helpful to ANYONE if we start beating up the people at NCDC ahead of such a statement and/or correction.
I will be among the first, if not the first to know what they are doing to fix the issues, and as soon as I know, so will all of you. Patience and restraint is what we need at the moment. I believe they are making a good faith effort, but as you all know the government moves slowly, they have to get policy wonks to review documents and all that. So, we’ll likely hear something early next week.
These lapses in quality control and thinking that infilling estimated data for long dead weather stations is the sort of thing happens when the only people that you interact with are inside your sphere of influence. The “yeah that seems like a good idea” approval mumble probably resonated in that NCDC meeting, but it was a case of groupthink. Imagine The Wall Street Journal providing “estimated” stock values for long dead companies to provide “continuity” of their stock quotes page. Such a thing would boggle the mind and the SEC would have a cow, not to mention readers. Scams would erupt trying to sell stocks for these long dead companies; “It’s real, see its reporting value in the WSJ!”.
It often takes people outside of climate science to point out the problems they don’t see, and skeptics have been doing it for years. Today, we are doing it again.
For absolute clarity, I should point out that the RAW USHCN monthly datafile is NOT being infilled with estimated data, only the FINAL USHCN monthly datafile. But that is the one that many other metrics use, including NASA GISS, and it goes into the mix for things like the NCDC monthly State of the Climate Report.
While we won’t know until all of the data is corrected and new numbers run, this may affect some of the absolute temperature claims made on SOTC reports such as “warmest month ever” and 3rd warmest, etc. The magnitude of such shifts, if any, is unknown at this point. Long term trend will probably not be affected.
It may also affect our comparisons between raw and final adjusted USHCN data we have been doing for our paper, such as this one from our draft paper:
The exception is BEST, which starts with the raw daily data, but they might be getting tripped up into creating some “zombie stations” of their own by the NCDC metadata and resolution improvements to lat/lon. The USHCN station at Luling Texas is listed as having 7 station moves by BEST (note the red diamonds):
But there really has only been two, and the station has been just like this since 1995, when it was converted to MMTS from a Stevenson Screen. Here is our survey image from 2009:
Photo by surfacestations volunteer John Warren Slayton.
NCDC’s metadata only lists two station moves:
As you can see below, some improvements in lat/lon accuracy can look like a station move:
http://www.ncdc.noaa.gov/homr/#ncdcstnid=20024457&tab=LOCATIONS
http://www.ncdc.noaa.gov/homr/#ncdcstnid=20024457&tab=MISC
Thanks to Paul Homewood for the two images and links above. I’m sure Mr. Mosher will let us know if this issue affects BEST or not.
And there is yet another issue: The recent change of something called “climate divisions” to calculate the national and state temperatures.
Certified Consulting Meteorologist and Fellow of the AMS Joe D’Aleo writes in with this:
I had downloaded the Maine annual temperature plot from NCDC Climate at a Glance in 2013 for a talk. There was no statistically significant trend since 1895. Note the spike in 1913 following super blocking from Novarupta in Alaska (similar to the high latitude volcanoes in late 2000s which helped with the blocking and maritime influence that spiked 2010 as snow was gone by March with a steady northeast maritime Atlantic flow). 1913 was close to 46F. and the long term mean just over 41F.
Seemingly in a panic change late this frigid winter to NCDC, big changes occurred. I wanted to update the Maine plot for another talk and got this from NCDC CAAG.
Note that 1913 was cooled nearly 5 degrees F and does not stand out. There is a warming of at least 3 degrees F since 1895 (they list 0.23/decade) and the new mean is close to 40F.
Does anybody know what the REAL temperature of Maine is/was/is supposed to be? I sure as hell don’t. I don’t think NCDC really does either.
In closing…
Besides moving toward a more accurate temperature record, the best thing about all this hoopla over the USHCN data set is the Polifact story where we have all these experts lined up (including me as the token skeptic) that stated without a doubt that Goddard was wrong and rated the claim “pants of fire”.
They’ll all be eating some crow, as will I, but now that I have Gavin for dinner company, I don’t really mind at all.
When the scientific method is at work, eventually, everybody eats crow. The trick is to be able to eat it and tell people that you are honestly enjoying it, because crow is so popular, it is on the science menu daily.
This is an excellent opportunity to do a meaningful analysis on the efficacy of the estimates. Are they accurate? Are they representative? How do they differ from the “real” temp?
I originally posted this over at CE.
Would you bet your house on the accuracy of a temperature reading prior to the use of properly sited digital stations? No. Whilst many stations are individually good many more have a string of associated problems. Even the good ones have probably been substantially adjusted or there is data missing and interpolation has taken place
I wrote about some of the myriad problems with taking accurate temperatures here.
http://wattsupwiththat.com/2011/05/23/little-ice-age-thermometers-%E2%80%93-history-and-reliability-2/
The further back in time the more potential for problems there are. Thermometer accuracy, accuracy of readings, calibration, time of day, recording a true max and min, use of appropriate screens, there are many and varied ways of messing up a temperature. If you really want to try to get to the REAL temperature of a historic record then you need to spend millions of Euros and several years examining 7 historic European temperature records as Camuffo did..
The result is a 700 page book which I have had to borrow three times in order to read it properly
http://www.isac.cnr.it/~microcl/climatologia/improve.php
Do all temperature readings -especially the historic ones-get such five star analysis? No of course not. We should treat them all with caution and remember Lambs words about them that ‘we can understand the tendency but not the precision.’ Some will be wildly wrong and misleading, some will be good enough. Do we know which is which? I doubt it.
I have no doubt that temperatures have ranged up and down over the centuries as there is other evidence to support this. Do we know the global temperatures to tenths of a degree back hundreds of years? Of course not. Do we know a few regional examples of land temperatures to an acceptable degree of accuracy. Yes, probably. Do we know the ocean temperature to a few tenths of a degree back to 1860. No, that is absurd.
Have temperatures been amended from the raw data? Yes. Has it been done as part of some deliberate tampering with some of the record, rather than as a scientific adjustment for what are considered valid reasons? I remain open to the possibility but am not a conspiracy theorist.
Someone like Mosh-who I trust- needs to keep explaining to me why the past records are adjusted. With this in mind it needs clarification as to why the readings from the famous 1987 Hansen hearing differ in part to the ones Giss then produced . I am sure there must be a valid reason but as yet no one has told me what it was.
It is absurd that a global policy is being decided by our governments on the basis that they think we know to a considerable degree of accuracy the global temperature of land and ocean over the last 150 years.
Sometimes those producing important data really need to use the words ‘ very approximately’ and ‘roughly’ and ‘there are numerous caveats’ or even ‘we don’t really know.’
tonyb
Nick, just stop. You are in a hole and yet you just keep digging. Just stop.
One might consider making a contribution to Goddard’s tip jar.
“You can’t make a clean estimated signal out of a bunch of muddied signals, ever.”
Then there’s no point in discussing analysis, is there? But it is the job of NOAA and USHCN to interpret the data, as best they can, even if you think it is worthless. And I think at Luling they did everything right. They picked up a problem, quarantined the data, and got the best estimate available with the remaining data.
“Now its well past your bedtime is Australia. Maybe that is why you aren’t thinking clearly”
When it’s afternoon in California, the sun is over the Pacific somewhere. It’s 8am here.
REPLY: Right you are, I thought you’d been up all night based on your commentary elsewhere. I also thought you lived in Perth. Obviously not.
Estimating data is the issue, and again when you use let’s say the six nearest stations, and statistically as we have shown at least 80% of them are unacceptably sited, resulting in a warm bias (and that’s not just my opinion that’s from Leroy 99 and 2010, and NCDC’s use of that to setup USCRN), that means your signal is going to be biased, full of the mud from the other stations.
It renders the idea of a useful estimate pointless.
And if you are too obtuse to see that, then yes, there’s nothing else to discuss -Anthony
Nice Post Anthony, I am very happy you admitted you were wrong about what you said after truely looking at Steve Goddard claimed.
Now I would hope you work with Steve Goddard because he was the original whistle blower and one who dug up the findings.
Going to be interesting.
BTW Sunshinehours blog discovered the other day that July 2012 was not the hottest month after all. Data has been changed recently.
http://sunshinehours.wordpress.com/2014/06/25/noaa-usa-july-1936-maximum-temperatures-top-3-are-1936-1934-and-1901/
http://sunshinehours.wordpress.com/2014/06/25/noaa-usa-july-1936-is-back-on-top/
new ones also so many he is uncovering.
http://sunshinehours.wordpress.com/2014/06/28/ushcn-2-5-omg-the-old-data-changes-every-day-2/
http://sunshinehours.wordpress.com/2014/06/28/ushcn-2-5-omg-the-old-data-changes-every-day-updated/
Can of worms has been opened.
This all must not just be confined to the US temp records, how far has it spread. world wide temp records?
On using absolute temperatures vs anomalies in the context of Low Frequency Information Content.
I accept the problem with using Absolute Temperatures, such as deg C above zero, or even deg K. when you are trying to compare stations at sea level and those as 3000 m, those at 30 deg North and othose at 45 deg N latitude. And yes, missing data using these absolutes temps creates spurious anomalies like those found in Marcott and perhaps the Goddard hockey stick.
From a low frequency preservation issue, the is no loss of information of each station uses it’s own baseline to establish a zero-base absolute temperature anomaly. The key is that the baseline MUST NOT CHANGE over time. Keep that criteria and the low frequency information (climate change) is preserved. This is like a simple tare measurement when weighing samples in a lab. If you do it right, you measure the tare before and AFTER the procedure.
My problem with the BEST process is they change the baseline for each station on criteria that results in chopping the station records into segments much too short to preserve the climate signal sought. If they feel the need to chop a station record into segments as short as 10 years, then the station is useless for the purpose of climate monitoring. It is as if they can discern changes in the tare of the beaker just by looking at the string of samples weighed. Madness.
Thinking you are improving the data by manipulating it is the worst form of confirmation bias. At best, tare adjustments add uncertainty to the measurement. But if you don’t measure the tare, don’t assume it changes. Take your thumbs off the scale
Anthony, I don’t see why you aren’t “sure why [I] can’t see” you’re addressing “the difference between raw and estimated data in his graph.” This post never claims to address that difference. It never claims to explain why he got the graph he got. There isn’t a single word about quantifying the effect of the problem you highlight. This post does nothing to show the problem highlighted in it actually explains the difference Goddard highlighted.
In fact, it seems unlikely this issue does explain the difference. Steven Goddard used a graph from 1999. At that point, GISS only had data for a fraction of its stations for 1990 and on. Data for thousands of stations were missing from their data. It would be no surprise if missing data in the US regions caused the results to be distorted.
When comparing results from what a decade or more apart, things like code/version changes and data availability are far more likely culprits for differences in results than this bug. As such, there’s no reason to posit a causal link between the bug you discuss and the things Goddard said.
REPLY: The missing stations aren’t the issue, that’s been well known for some time. The fact that estimated data from long-dead and missing stations is being produced is news. The idea was that this infilling was designed to fix occasional lost data, not do a wholesale replacement of weather station data for stations like Marysville that have been closed since 2007. -Anthony
Integrity: you can’t buy, borrow or steal it.
I like integrity.
Steven Mosher says (June 28, 2014 at 1:36 pm) “if you split a station where there is no actual move it has no effect“. That should be “little effect” not “no effect”. After all, if you put in a station move after each measurement, you end up with nothing, so putting in moves does have some effect.
Please note, this comment is a very minor nitpick, there are much more important issues on this thread. I hope that something seriously good comes of the whole exercise (apart from a seriously good quotable quote about scientists eating crow).
Nick Stokes:
I write to congratulate you on your fortitude and to commend you for the honour you display by ‘standing your ground’.
As is clear from my comments to you on the other thread, I think you are profoundly mistaken about the validity of the various methods used by you and others to determine GASTA (global average surface temperature anomaly). But that disagreement does not blind me to the courage you are displaying here.
You have risen in my esteem, and I hope others are also observing the respect you deserve for your contributions to this discussion.
Please continue your contributions.
Richard
For those watning a visual aid, here is a map of Illinois TMAX USHCN for May 2014 show raw, TOBS and Final. There are a bunch of stations without raw,
https://sunshinehours.wordpress.com/2014/06/28/ushcn-2-5-omg-the-old-data-changes-every-day-2/
Might now be a good time to instate Real Science on your blog-roll, even if it goes in the political climate section?
Just to clarify.
The analysis Anthony refers to for Kansas, has looked at mean temperatures at every USHCN station in Kansas for January 2013.
The USHCN Final dataset has adjusted UPWARDS every single station, bar one, by an average of about 0.5C.
This is in addition to cooling historic temperatures. E.g. temperatures for 1934 have been reduced by about half a degree as well.
http://notalotofpeopleknowthat.wordpress.com/2014/06/28/ushcn-adjustments-in-kansas/
There are also 8 out of 29 stations which have “Estimated” numbers.
Does Nick Stokes really believe these are all due to faulty sensors?
MUCH ADO ABOUT SOMETHING!
At world-famous U of Michigan School of Engineering, which I completed by the way, we learn rules about data. The first rule is that the data always have accuracy only to a certain level, known as “significant digits.” The second rule is that, if you need better data, buy a better instrument. Calibration back to NIST is key as well.
BEST and NCDC and GISS and CRU, and all of the rest of climate record keepers, any attempt to “improve” on raw data by “adjusting” will always be met with contempt by engineers, who in general make our living by getting RESULTS!
Thermometers typically are used to report WEATHER. Attempting to generate CLIMATE signals is futile at best. Gridding adds no information whatsoever, but lets the gridders claim their results are “Global.” Thermometers of course should be located where the air temperature is representative, otherwise the data are spurious. Averaging from nearby stations is ludicrous, merely claiming information you do not have. Extrapolating across hundreds of kilometers is even more ludicrous.
Adjusting historic records on a monthly basis would have gotten me thrown out of school. My hero Professor Brown chides “Climate Science” for the complete and total absence of error bars, and his words are significant.
Average Temperature is itself a very dubious concept, as temperature is defined as the average kinetic energy of the molecules of the mass whose temperature is being recorded. What mass is being discussed, exactly?
The problem I have with NCDC is that it is bad enough they use “estimated” data but then they go back into the record and retroactively change it. If you look at the NCDC database at the temperature for some date in the past and then look in the database in a few years time, you will find it has been changed. It seems that another problem with using estimated data is that those estimates are re-calculated every month. For example, this is how the database has changed for two dates over time:
http://climate4you.com/images/NCDC%20Jan1915%20and%20Jan2000.gif
Mike Jonas, this is no nitpick.
It is a statement that makes my jaw drop in disbelief.
If Steven Mosher, and the rest of BEST, believe that, then that explains why scalpel has turned into a Cuisinart, mincing temperature records willy-nilly, because they believe an unjustified slice does no harm!
With an unjustified slice:
* What was one trend has now been made into two sequential trends with an offset.
* An offset whose significance is ignored because, “it’s a station move – therefore a new station.”
* Low Frequency content is lost. Higher frequency (weather) information is weighted more heavily.
* The uncertainty in the result rises greatly (just make slice at different points and see the differences)
Confirmation Bias is the Blue Plate Special today. Another serving of crow on order.
Nick, you are obviously a bright guy, but mon ami, you wear blinders and can’t see anything except your own keyboard. In the Anthony’s post starting 6-26-14, a poster listed a URL to a city with a very similar temperature chart as Luling, Texas. Anthony screwed up on Steve Goddard and admitted it like a man. I suspect that you are young otherwise you would have learned to not be such a know-it-all and think that only you knows something. Just a word to the wise from someone who has been there.
Nick Stokes says:
June 28, 2014 at 3:00 pm
Nick, they couldn’t have used San Antonio which is one of the near stations as it also has Estimated values from May 2013 to current month.
I found 3 in first 10 stations in the Texas zip file have estimated data for 2012-2014.
Yes 30 % on a small sample.
This post definitely earns a Watts’ Best tag in my library.
Well done and well worth remembering.
Oh, I think I see the problem. This post talks about a Polifact story which criticzed Steven Goddard’s claim about adjustments to a temperature record. In doing so, it said things like:
I, perhaps naively, thought that meant this post was about what that story covered. As such, I pointed out this post fails to show Goddard was right, in any way, about what he was quoted as saying in that story. I failed to notice this post was actually saying Goddard was right about an entirely different issue when it said:
Goddard was right to point out this bug existed. That just has nothing to do with the Polifact story this post portrays him as being right in.
So yeah, my bad. Sorry I didn’t realize you were giving Goddard credit in reference to one issue because he was right on a completely different issue. I guess I’m just bad at spotting incoherence.
REPLY: I have some crow I haven’t been able to choke down yet, happy to share 😉 See the link to the “40% fabrication” at the top of the post. I agree though, a lot of material here, and if you haven’t been following the issue, easy to get sidetracked. – Anthony
In 2007 I started monitoring a few GISS stations. Amundsen-Scott, Vostok, Halley, Davis in Antarctica. Then a few more over the years. It was just to check some of the claims of data changing. Also to see if I could plot the temps over time. Some small changes at first, but nothing of concern.
In 2008 I added more stations such as Nuuk, to check more claims. It is easy to save the temp files for each station. Picking a few stations at random points around the GISS globe, up to about 12.
Goddard was right. Past data points were changing. Usually small amounts on a monthly basis.
Some station data did not change at all, but only the new monthly data were added.
In 2009 I was saving monthly data for 15 stations.Every month the most recent data is added to the end of each file. But, scanning past data it became apparent that some historical station data was being “revised” almost completely. That means that a time plot shifted by enough to notice on a temperature plot. Usually the “new” plot showed that the past became cooler. Small amounts, maybe one tenth of a degree. When I got to 20 stations, I noticed that one or two sets of data were being extensively revised every month. At the end of 2009 there was a sudden revision of more stations by larger amounts. Almost always the data before 1960 was getting cooler.
By 2010 I was up to 30 stations, plus the Antarctic four.
Akureyri, Bartow, Beaver City, Concordia, Crete NE, Ellsworth, Franklin, Gothenburg, Hanford, Honolulu, Hilo, Jan Mayen, Kodiak, Kwajaalein, La Serena, Loup City, Minden, Nantes, Nome, Norfolk Island, Nuuk, Red Cloud, St. Helena, St. Paul, Steffenville, Thiruvanantha, Truk, Wakeeny, Yakutat, Yamba.
Downloading 30 sets of temperature data to save. Then putting all data into an excel file to make plots. Then comparing time plots to see where individual data points had changed. It became clear that a regular revision of about 10 percent of all GISS stations was taking place on a monthly basis.
Every month in 2010, three of the 30 stations has changes to past data points that were visible by plotting. Looking at every point for every station is not possible, but scanning at random shows that many stations have small changes. 2011 was about the same, but at the end of 2011 was another substantial change to many stations. In early 2012 my 10 year old hard drive died. I had a separate backup, but it was found to be virus infected to the extent that I could not save all data. I still have the drive and can open some files but can’t copy files to another drive.
In 2012 GISS began making larger changes to more past data. Some stations were showing monthly changes that could only be described as “erratic” with some large shifts for a couple months, followed by data returning to values from months earlier. Sometime in December 2012 was a very large change in past data for many stations. Some stations showed data that stopped in 2007 or 2008. In March 2013 those stations were suddenly “restored” and resumed showing complete data up to March 2013.
Every month GISS changes past data by small amounts to some historical data. Some stations are being selected for larger revisions at random times. For some stations, portions of data from decades ago vanishes completely. The numbers are replaced with “999.9” indicating no data. Months later those same past data points re-appear, just as they were before, sometimes with small adjustments. If what is happening to a sample of 30 stations is any indication of the entire GISS data set, then I’d say that there is an Orwellian plot to manipulate the past. There is no record of these past changes, it would be impossible to verify or reproduce those changes. Over the years, the past keeps getting colder.
I know of less than a dozen sites that would post a lead story such as this. It is very refreshing, kudos! Carry on!
The problem Mr. Bastardi, and I don’t want to get into a big back and forth, is that, for many of us, this is much closer to “even a blind squirrel stumbles across a nut sometimes”, than an occasional error. Goddard’s analysis in this case was shown to be faulty multiple times. It was Homewood’s analysis that caused this issue to be taken seriously
Anthony, you are to be commended for your integrity. Not everyone can admit to mistakes.