GISS "raw" station data – before and after

I’ve been following this issue a few days and looking at a number of stations and had planned to make a detailed post about my findings, but WUWT commenter Steven Douglas posted in comments about this curious change in GISS data recently, and it got picked up by Kate at SDA, which necessitated me commenting on it now. This goes back to the beginning days of surfacestations.org in June 2007 and the second station I surveyed.

Remember Orland? That nicely sited station with a long record?

Note the graph I put in place in June 2007 on that image.

Now look at the graph in a blink comparator showing Orland GISS data plotted in June 2007 and today:

NOTE: on some browsers, the blink may not start automatically – if so, click on the image above to see it

The blink comparator was originally by Steven Douglas. However he made a mistake in the “after” image which I have now corrected.What you see above is a graphical fit via bitmap alignment and scaling of the images to fit. This is why the dots and lines appear slightly smaller in the “after” image.  I don’t have the GISS Orland data handy at the moment from 2007, but I did have the GISS station plots from Orland from that time and from the present, downloaded from the GISS website today. If I locate the prior Orland data, I’ll redo the blink comparator.

I believe this blink comparator representation accurately reflects the change in the Orland data, even is the dots and lines aren’t exactly the same thickness.

Douglas writes in his notice to me:

It appears that RAW station plots are no longer available, although NASA GISS (Hansen et al) do not say it in this way. Here is the notice on their site:

Note to prior users: We no longer include data adjusted by GHCN and have renamed the middle option (old name: prior to homogeneity adjustment).

I don’t know about the “renamed” option, but the RAW data appears to be NO LONGER AVAILABLE.

Here’s a detailed blink comparison of Orland. All their options now give you an “adjusted” plot of some kind. The “AFTER” in this graph show the “adjustments” to Orland.

Here is what the GISS data selector looks like now, yellow highlight mine, click to enlarge:

Above clip from: http://data.giss.nasa.gov/gistemp/station_data/

Here is the “raw” GISS data plot of Orland I saved back in 2007:

Click for full sized

And here is another blink comparator of Orland raw -vs- homogenized data posted by surfacestations.org volunteer Mike McMillan on 12/29/2008:

click for full size

And here is the “raw” GISS data for Orland today, please note the vertical scale is now different since the pre-1900 data has been removed, the GISS plotting software autoscales to the most appropriate range:

click for source image from NASA GISS

Source:

http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=425725910040&data_set=0&num_neighbors=1

And it is not just Orland, I’m seeing this issue at other stations too.

For example Fairmont, CA another well sited station well isolated, and with a long record:

Here is Fairmont “raw” from 11/17/2007:

click for full size

And here is Fairmont from GISS today:

click for source image from NASA GISS

Source:

http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=425723830010&data_set=0&num_neighbors=1

This raises a number of questions. for example: Why is data truncated pre-1900? Why did the slope change? The change appears to have been fairly recent, within the last month. I tried to pinpoint it using the “wayback machine” but apparently because this page:

http://data.giss.nasa.gov/gistemp/station_data/

is forms based, the change in this phrase:

Note to prior users: We no longer include data adjusted by GHCN and have renamed the middle option (old name: prior to homogeneity adjustment).

Appears to span the entire “wayback machine” archive, even prior to 2007. If anyone has a screen cap of this page prior to the change or can help pinpoint the date of the change, please let me know.

It is important to note that the issue may not be with GISS, but upstream at GHCN data managed by NCDC/NOAA. Further investigation is needed to found out where the main change has occurred. It appears this is a system wide change.

The timing could not be worse for public confidence in climate data.

I’ll have more on this as we learn more about this data change.

UPDATE1 from comments:

GISS also just started using USHCN_V2 last month. See under “What’s New”:

http://data.giss.nasa.gov/gistemp/graphs/

“Nov. 14, 2009: USHCN_V2 is now used rather than the older version 1. The only visible effect is a slight increase of the US trend after year 2000 due to the fact that NOAA extended the TOBS and other adjustment to those years.

Sep. 11, 2009: NOAA NCDC provided an updated file on Sept. 9 of the GHCN data used in our analysis. The new file has increased data quality checks in the tropics. Beginning Sept. 11 the GISS analysis uses the new NOAA data set. ”

Share


Sponsored IT training links:

Worried about N10-004 exam? Our 640-802 dumps and 70-680 tutorials can provide you real success on time.


Advertisements

  Subscribe  
newest oldest most voted
Notify of
007
Alan S. Blue

This is one of the most infuriating aspects of “the data” in these discussions.
“The data is available!” But there are ongoing revisions without any apparent version control strategy. It’s like a Monty Python skit or Lucy with the football.

Doug

I always assume ‘adjustments’ are political.
Not unlike inflation numbers, leaving out the cost of food and fuel.

timetochooseagain

Hide a decline.
(not the decline but another one)

George Turner

Well this just illustrates the well-established and documented trend that late-19th and early 20th century temperatures have been plummeting for years, at ever increasing rates. Soon the ministry of truth will start photoshopping snowflakes into old historical photographs, such as the Wright brother’s flight at Kitty Hawk.

Anthony,
Very interesting. I was noticing some many of these changes (where your archived plots on surfacestions.org do not match the current GISS plots) when I was gathering the GISS data for StationLab. It was on my list to ask you about, if you were interested in my StationLab.
I’m glad this has come to light!
Regards,
Daniel Ferry

The Revenge of the “Un-Data”, starring James Hansen and Gavin Schmidt.
Anthony,
Don’t you wish you had placed more of the old data “in a mayonaise jar on Funk & Wagnall’s porch”?

I always find it laghable how they pretent that they have made adjustment to imporve the accuracy, but when analyzed it is clear that they have done just the opposite. All these examples (as well as many previously discused relating to the surfacestations audit) show a lowering of historic tempetures, and an elevation of modern temperatures. Of course, when trying to remove UHI effect from the record, that is exaclty opposite of what should be done.

GISS also just started using USHCN_V2 last month. See under “What’s New”:
http://data.giss.nasa.gov/gistemp/graphs/
“Nov. 14, 2009: USHCN_V2 is now used rather than the older version 1. The only visible effect is a slight increase of the US trend after year 2000 due to the fact that NOAA extended the TOBS and other adjustment to those years.
Sep. 11, 2009: NOAA NCDC provided an updated file on Sept. 9 of the GHCN data used in our analysis. The new file has increased data quality checks in the tropics. Beginning Sept. 11 the GISS analysis uses the new NOAA data set. ”
REPLY: I had suspected as much, this is a likely source of the change, thanks for providing that notice. – Anthony

Good work.

Getting Hot

It would have been fine if the oil industry would quit funding 6th graders to make charts that are deflecting the focus away from CO2.

CGCO2

“We no longer include data”
Is that an English language synonym for “deleted”?

NK

TO: Alan S. Blue (12:40:26) :
Monty Python and Lucy with the football, excellent analogies.
I’ve had it. GISS and CRU CAN’T be THIS incompetent, they KNOW that over the last 20 years, that as the gatekeepers of the ‘data’, their continuous massaging the numbers through ‘adjustments’ and dumping ‘raw’ data records allows them to effectively manufacture their own data to match up with their own models. It IS a scam, it’s not just incompetence. This is positively Orwelian, but this is really happening.

Steve

Anthony,
How on earth do the AGW team expect any credibility.
This needs wide distribution.
Or, maybe we just need to start prep for the overrule of Lisa’s (Obama) EPA co2 ruling.
Steve

David

Ok, new version, same data. Why is the data incompatible with the version instead of the other way around? Is there a legit reason pre-1900 data showing more warmth has to be truncated from the record because of the version switch? I don’t get how that is an explanation, I guess.

Jim

I think it’s time we got some lawyer with a conscience to file a class action suit for us. This hiding of data that we in the US paid for is criminal – or it should be if it isn’t. These people should be fired AND prosecuted.

Gino

I think this thread might require a sticky to go with the smoking gun.

Arnold

I made a download of raw data on 9-12-09 for the dutch data. This is now no longer available, so it seems that this adjustment was made yesterday or today. The data that is available now is really different.
I’ll try to make a graph to show the difference.

Michael

[snip waaaayyy off topic]

Honest ABE

Oh, off-topic but I have a question, have oxygen and nitrogren been increasing in the atmosphere?
How much O2 is in the ocean compared to CO2?

Jim

I guess the Wayback machine wouldn’t work to get the old data since it was stored on a server.

David

And while I am on this, is there somewhere that one can acquire the station data that includes Tmin and Tmax? All I have found so far is the monthly means. I would like to look at data on a daily basis, but I can’t find it. Does it exist?

Bill P

Back in 2007 Douglas Keenan was planning to file a fraud claim against Dr. Wei-Chyung Wang and his co-author for a paper they had written on rural / urban temperatures in China. The paper allegedly showed that there was no difference between temps in cities and in the rural villages, but had been unable to produce any clear data on the stations used. Phil Jones was the co-author. Here’s a letter from Ben Santer to Jones.
From: Ben Santer
To: P.Jones@xxxxxxxxx.xxx
Subject: Re: FYI
Date: Tue, 24 Apr 2007 09:57:34 -0700
Reply-to: santer1@xxxxxxxxx.xxx
Cc: trenbert@xxxxxxxxx.xxx, mann@xxxxxxxxx.xxx
Dear Phil,
Sorry about the delay in replying to your email – I’ve been out of my
office for a few days.
This is really nasty stuff, and I’m sorry that it’s happened to you. The
irony in this is that you are one of the most careful and thorough
scientists I know.
Keenan’s allegations of research misconduct, although malicious and
completely unfounded, clearly require some response. The bottom line is
that there are uncertainties inherent in measuring ANY properties of the
real-world climate system. You’ve probably delved deeper than anyone
else on the planet into uncertainties in observed surface temperature
records. This would be well worth pointing out to Mr. Keenan. The whole
tenor of the web-site stuff and Keenan’s garbage is that these folks are
scrupulously careful data analysts, and you are not. They conveniently
ignore all the pioneering work that you’ve done on identification of
inhomogeneities in surface temperature records. The response should
mention that you’ve spent much of your scientific career trying to
quantify the effects of such inhomogeneities, changing spatial coverage,
etc. on observed estimates of global-scale surface temperature change.

Michael K

Will someone with some scientific credentials PLEASE get on television with some charts that lay people can understand that exposes this DATA manipulation. The alarmists keep beating the drum that the temperature is rising at an alarming rate and no one is countering with understandable criticisms and evidence that the data was mysteriously adjusted upwards. This will make a huge difference in the credibility of their base line argument.
There are many good points to focus on, but pick one such as this and drive it home.

Ryan O

As part of this, the GHCN data that feeds GISS/CRU/NOAA is not truly “raw” either. Note that the GISS station selector yields multiple sources for various locations. You can get the monthly data for those various locations in the GHCN files, but there is no archived daily data in the GHCN files. The daily data has it combined.
This is important, because monthly means calculated using the archived daily data are DIFFERENT than what appears in the monthly file – sometimes by as much as 2 degrees C.
Note that the GHCN data has 2 versions, raw and adjusted. These comments apply to the RAW version. In other words, the raw version is not truly raw, either.

3x2

Is it possible that the raw data is being withheld or is it gone—either trashed or manipulated to the point of being unrecognizable and useless?
REPLY: No it is still intact at NCDC, on paper forms, with transcription data also available – Anthony

Ed Scott

From Clayton Cramer’s blog: Pasteurized Data
All this talk about how the CRU “homogenized” the data to remove inconsistencies brought forward a comment from a reader. He pointed out that the data wasn’t just homogenized; it was also pasteurized: heat was added until the data was completely sterile!

The “blink comparator” is the skeptics’ “nuclear option” for fighting revisionist temperature history.
Apparently the “global temperature record” is now also considered to be a “living document”.

Manfred

This does not look good. Is there any other plausible explanation than just a NOAA and/or GISS sabotage of efforts to verify their temperature records ?
http://wattsupwiththat.com/2009/07/05/pielk-sr-responds-to-ncdcs-talking-points-about-surfacesations-org/

Evan Jones

UDHCN1 did have some downward trend adjustments. But twice as many upward. The effect was to change the US station average (equally weighted) from +0.14C per century (raw) to +0.59C per century (full FILNET adjustment).

Jeff

I’ve started doing some research on what sort of adjustments are done when the data is homogonized … mainly with Pennsylvania stations … so far about half the stations show a step ladder of adjustments … i.e. greater adjustments down in the early years 1900 ish stepping up to zero adjustments for the last decade …
__
|

|

|

keep in mind these adjustments are down … if they were adjusting for UHI then we would expect the most recent adjustments to be the highest …
seems like a neat trick to get the slope to be steeper … don’t raise the near term temps, just lower the older temps …

Robert of Ottawa

Haha. Data not included.

Jeff

tried to illustrate the stepladder … failed miserably …

Ray

You don’t teach an old monkey new “tricks”. Let’s not hide the decline. Just remove the negative slope by removing the data… that’ll do the “trick”.
Oh, and don’t forget to delete the emails too.

Arnold

I did download the data on 09-12-09. But i downloaded data that was spliced together allready. So i don’t know anything with regard to this adjustment.
So disregard my previous statement.

theBuckWheat

Someone has lots of ‘splainin’ to do.

Mark Buehner

If you are looking for another ‘odd’ set of stations- check out Valencia and Castellon-Almazora in Spain. They are 30 miles apart and both about 2 miles from the Med. But Valencia is a big city and Castellon a much smaller town.
Three things- the unadjusted data are in close agreement up until about 1972, at which point they diverge by over a degree, with Valencia much warmer (urban effect?).
Secondly if you look at the adjusted data, they are suddenly in much closer agreement after 72- but it is Castellon that has been adjusted UP, not Valencia down.
Finally- if you look at Valencia by itself between 1900 and 1950, you will see that they have adjusted those temps downward a full degree across the board.
Now there may be very good reason for these adjustments (which obviously should be published- neither of the stations appear to have moved, at least according to the historical change db), but at first blush it would appear Valencia has an urban heat issue, and instead of correcting it NOAA instead corrected the nearest neighbor upwards to match. Moreover, a large adjustment to Valencia’s temps in the first half of the century have created (surprise) a hockey stick that doesn’t exist in its neighbor, and much less so even with the potential urban heating uptick.
(Also check out Alicante-Cuidad, the next town south, which also was adjusted way down in the 50s to produce a similar hockey stick. I’d really like to know why Valencia supposedly jumped a degree in temperature when none of her neighbors show this in their unadjusted data).

Jack Simmons

I like my temperature data the same way I like my sushi: Raw.

Bill DiPuccio

It has come to the point where we need to go back and dig up the paper records to be sure the information has been faithfully copied. Are images of station forms available? Or, has the paper conveniently dissolved?

Bart Nielsen

Great work, Anthony. The self-referential nature of “climate science” as practiced by the AGW proponents (using tax money) is appalling. It appears that reality is being “adjusted” to fit the models, which of course, yield the desired outcomes. The individuals and organizations engaging in apparent data manipulation, data destruction and secrecy need to be held accountable.

Retired Engineer

“renamed” or just “reamed”.
We’ve been had.

An NOAA blink comparison: raw vs “adjusted” temperatures: click
[may take a few seconds to load]

wobble

Jeff (13:12:58) :
“”seems like a neat trick to get the slope to be steeper … don’t raise the near term temps, just lower the older temps …””
Are you getting a sense of the timeframe of a possible pivot point? In other words, is there a date in which many prior adjustments seem to be up and many later adjustments seem to be down?

Kevin Kilty

Have we reached a point where to audit the basic data of this debate credibly we must return to paper records and carefully deliberate each of the needed adjustments in public; while at the same time reviewing the quality of those paper records from the standpoint of station maintenance and UHI effect? Are we at this point?
I wish I were retired already and had the time to begin such an undertaking.

The Iconoclast

Anthony,
Is there an archiving effort? We need snapshots of the evolution of the datasets over time if at all possible.
Also I have lost the URL to the PDFs of the paper forms. Can someone please post it? These need to get copied and secured in case they too disappear down the memory hole, or worse.
REPLY:
Station B91 PDFs here: http://www7.ncdc.noaa.gov/IPS/coop/coop.html
Transcribed here: http://cdo.ncdc.noaa.gov/dlyp/DLYP

Michael

Copenhagen climate change summit: The world is COOLING not warming says scientist Peter Taylor … and we’re not Prepared
http://www.dailymail.co.uk/sciencetech/article-1234515/Copenhagen-climate-change-summit-The-world-COOLING-warming-says-scientist-Peter-Taylor—prepared.html

NK

To: Kevin Kilty (13:38:58) :
That’s what Anthony and Steve M. have been doing for a few years. Those are huge undertakings and we all owe them, big time.

stan

Someone in Congress needs to make a big show of asking for full transparency of all data, code and adjustment explanations. After all, the science is settled. We just want to see what we paid for.

Invariant

Anthony: watch this video…
[ http://www.youtube.com/watch?v=aUtzMBfDrpI ]
REPLY: Yes I’ve seen it. – A