UPDATE: 8/22/12 9AM The problem has been solved, GISS responded to my complaint -Anthony
Like the erroneous graph at California Governor Jerry Brown’s climate denier slam site, here’s another one of those things that I’ve been sitting on for about a week, waiting for somebody to fix it. Since they haven’t, and I’ve given adequate time, I suppose it is time to bring this latest GISS miss to the global attention of everyone.
Last week during my email group exchanges, somebody (I forget who) pointed out this graph from NASA GISS:
Source: http://data.giss.nasa.gov/gistemp/graphs_v3/Fig.D.gif (click to see yourself)
That is part of the GISTEMP graphs page here: http://data.giss.nasa.gov/gistemp/graphs_v3/
I chuckled then, because obviously it is some sort of data error, and not worth reporting since I figured surely those RealClimateScientists would notice in a day or two and fix it. Nope. But still there a week later? Now it is newsworthy.
That “off the charts” Figure D image has been around on this highly cited NASA GISS page, apparently unnoticed, since August 13th, 2012, here’s the proof in the image info:
I decided I’d have a look at the tabular data they offer, to my surprise, what I discovered was an “unprecedented” value in the dataset, larger than the hottest years of 1934, 1998, and 2006:
http://data.giss.nasa.gov/gistemp/graphs_v3/Fig.D.txt
Year Annual_Mean 5-year Mean 1930 0.1060 0.1156 1931 0.9860 0.2346 1932 -0.0360 0.5856 1933 0.6520 0.5716 1934 1.2200 0.4072 1935 0.0360 0.3868 1936 0.1640 0.4110 ... 1997 0.1330 0.5700 1998 1.3020 0.6248 1999 1.0630 0.8214 2000 0.6920 0.9284 2001 0.9170 0.8046 2002 0.6680 0.7124 2003 0.6830 0.7560 2004 0.6020 0.8304 2005 0.9100 0.8824 2006 1.2890 0.7766 2007 0.9280 0.6926 2008 0.1540 0.6276 2009 0.1820 0.5006 2010 0.5850 0.8220 2011 0.6540 * 2012 2.5350 *
Wow. 2.53°C ? I thought maybe the very warm, and warmest to date this year, July 2012 was the issue causing this. But, we know that can’t be right, because NOAA tells us in their July State of the Climate analysis:
The average temperature for the contiguous U.S. during July was 3.3°F (1.8°C) above the 20th century average, marking the warmest July and all-time warmest month since national records began in 1895.
So, I’m not sure where they come up with 2.53°C since NASA uses NOAA’s data, and one month shouldn’t skew half a year so much, but that is what seems to be happening. Plus they have the 2.53C in the annual mean column, which as we know isn’t complete yet, since 2012 is not complete.
GISS makes no direct caveat about presenting monthly data in the section on Figure D, though by inference, they possibly suggest it in the “five year running mean”, but aren’t clear if that is a monthly or annual calculated running mean.
Source: http://data.giss.nasa.gov/gistemp/graphs_v3/
Even so, if that running 5 year mean is using monthly data rather than annual data, updating one part of an annual graph with monthly data (for the annual mean as seen in the tabular data) can be very misleading to the public, and as we know, that page at GISS is used worldwide by media, scientists, and advocates. Therefore, it is very important to present it accurately and not mix monthly data and yearly data types without explanations of any kind.
I wanted to look in the Wayback machine to see what the Figure D graph said earlier this year, like maybe up to June, but to my surprise, GISS apparently prevents that public page from being indexed by the Wayback machine. In fact, they seem to have prevented a lot of content from being indexed and stored since 2005, see the dates:
http://wayback.archive.org/web/*/http://data.giss.nasa.gov/gistemp/*
In fact if you look at this graph of plots
http://wayback.archive.org/web/*/data.giss.nasa.gov/gistemp/
…and then try to go to the GISTEMPS graphs page, you get a lot of this:
I find it troubling that the publicly funded NASA agency GISS would block archiving of such an important global resource. This is not cool, guys.
Fortunately, Steve Goddard archived the GISS figure D image on January 29th, 2012, right after the year 2011 was updated with annual data:
So clearly, the effect is in 2012 data to date, but why would they plot monthly data to date on a graph depicting annual values?
This brings up some points.
1. The current US data Figure D graph compiled by GISS for 2012 is clearly erroneous the way it is presented.
2. The Figure D graph at GISS is clearly being updated with incomplete annual data, since this update showed up on the GISS website on August 13th, 2012. The graph portrays annual data. No mention is given of monthly data. This is wrong and misleading.
3. As before, as I pointed out to Governor Browns office, (now corrected) if I made a dumb mistake like this in a time-series, plotting incomplete months and presenting it as annual data, Tamino and his followers would “rip me a new one” (his words).
4. Why do I have to be the one to keep pointing these things out? Doesn’t the Governors Office and NASA’s Goddard Institute of Space Studies have any quality control procedures for the climate data they present to the public? Apparently not.
5. Why does GISS block the archiving of such important resources like the global temperature data they produce by such public domain services like the Wayback machine? Could it be they don’t want inconvenient comparisons like this one below to be made with their graphs?
Inquiring minds want to know.
h/t to Art Horn for the reminder today.
UPDATE: Shortly after this piece published, I emailed Dr. Gavin Schmidt of NASA GISS:
From: Anthony
Date: Tuesday, August 21, 2012 12:44 PM
To: Gavin.A.Schmidt@nasa.gov
Subject: courtesy note
Dear Dr. Schmidt,
I doubt you’ll credit me when you fix this, or even acknowledge receipt of this message, but I’m informing you of the error anyway.
http://wattsupwiththat.com/2012/08/21/climate-fail-giss-is-presenting-2012-us-temperature-as-off-the-chart-while-preventing-older-data-from-being-archived/
Best Regards,
Anthony Watts
UPDATE2: Commenter Jim P. points out 2012/08/21 at 1:50 pm
Anthony, there’s no error. It’s just the chart doesn’t extend high enough for this year.That’s the data for the year to date, not July.
As you can see from this NOAA chart: http://www.ncdc.noaa.gov/temp-and-precip/time-series/index.php?parameter=tmp&month=7&year=2012&filter=ytd&state=110&div=0. The mean temperature for the year-to-date is 56.4F, or 13.6C. The normal is 52.2F, or 11.2C. The departure is 2.4C or close to what GISS is reporting.
REPLY: Yes, I see, thank you. But, presenting monthly year to date data, in a graph labeled annual mean data, with no caveat at all, is most certainly wrong and misleading. I’d be excoriated by the climate community at large for presenting an annual mean graph with incomplete data for a year like that, so why should they get a pass for being sloppy like the California Governor? – Anthony
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
My guess is that they are ‘projecting’ how hot the rest of the year will be from the firist 7 months. So the hot month pushes up the average and gets replicated for the rest of the year.
In effect finding the average monthly anomaly and mulitplying by 12. Thus the one hot month will get counted (almost) twice. Of course, if there was a record cold month they’d make sure it did not get counted twice.
oldfossil says:
August 21, 2012 at 2:01 pm
http://www.giss.nasa.gov lists 149 personnel, of whom 30 are NASA affiliated. That makes me feel optimistic that the data is fundamentally accurate.
>>>>>>>>>>>>>>>>>>>>
Then I am guessing that you have never worked in a large organization where senior management has taken a position on some matter that is completely divorced from reality. The phrase “don’t shoot the messenger” was invented for a reason, which is that senior management routinely shoots those brave enough to speak truth to power. Being a whistle blower is no easy thing to do, particularly if you have very specialized job skill sets which aren’t applicable anywhere else, and that describes the researchers at NASA/GISS very very well.
I’m reminded of the ancient king who has a dream in which each of his teeth falls out one at a time until he has none left. Disturbed, he calls for a wise man to interpret the dream. The first wise man tells him that his life will be a nightmare in which he is forced to watch as each and every member of his family dies. Enraged the king has the wise man beheaded. The second wise man, assuming that by confirming the prediction of the first wise man the king would accept reality, was similarly dealt with. The third wise man thought carefully and then offered this advice.
“Congratulations. The gods have awarded you a life of extraordinary longevity, you shall surpass all your relatives in this regard.”
Speaking truth to power is a delicate matter, and while organizations for the most part don’t behead anyone who tells them something they don’t want to hear, they do fire them, and unemployment for someone with a very narrow skill set is rather intimidating. There a reason why a long list of former NASA employees has raised the alarm regarding the antics of GISS in general and Hansen in particular. They are all retired. Pretty good indication that there is something amiss and that current employees fear to speak out.
@Oldfossil:
It all depends on who hired whom, who controls the budget of whom, who agrees with Green Whom, and who wants to keep the paychecks coming in….
We’ve already has the “hottest ever” for about 2 decades yet it’s colder in my garden than during most of that time and tomatoes are getting harder to mature, not easier…
Also, there is a fundamental error of thought in talking about “GISS Data”. GIStemp just takes in GHCN and USHCN data and does some some (slightly bizarre IMHO) transformations on it and spits out the result. GISS doesn’t really have any “data” only pasteurized (past your eyes? 😉 processed data food product… ( In the USA, “fake cheese” is labeled “cheese food product” when it contains little that is actually cheese…)
More detail than any normal human being would want here:
http://chiefio.wordpress.com/gistemp/
including source code and results of my ‘code review’ on it, various runs and examinations, and how to run your own personal version if you so desire. It runs on the 2009 (or so) code and data (GHCNv2) and I’ve not had the stability of stomach to force myself through it again to do a ‘re-port’ of the newer code. (If someone wants to pay me to do it, I will, but “just out of curiosity”? … well, I’ve satisfied my curiosity… it’s crap. IMHO of course.) Some of the code clearly dates from about the 1970s-80s and is written in very old FORTRAN so it’s not like the thing changes very fast.
Most of the “changed data” results from processes built into the code where changed input data results in changed output of greater / different degree. So, for example, to “fill in” missing data, the stations used for each filling in depend on which stations are ‘nearest’ to the hole. Drop some data from a station that is presently “near”, and a different set of stations will be used to fill the hole in the first location. Creative control of when data / stations are missing can work wonders, IMHO. So, over time, the GHCN has fewer stations. During the baseline period, most data is actually fairly local. Over time, more of the grids / boxes will be filled with “nothing”, so fabricated – and from stations ever further away over time as there are more dropped stations.
Another? UHI adjustment is done in a similar way. Changes in stations and / or data in GHCN / USHCN will result in changed UHI adjustment results.
There’s more at the link.
The bottom line is pretty simple: GIStemp is a “NEVER the same result twice” product. Only if the data is fixed at a point in time will you get a stable result. And GHCN is a constantly updating data set ( in V3 it looks like they are starting to do a bit of ‘check point’ and date stamping – I hope…) so every month your will get a slightly different result. (Sometimes even from the “same” dataset during the month as updates flow in…)
This also means that EVERY adjustment / change done “upstream” at NCDC to GHCN and USHCN will have a ‘ripple of change’ through GIStemp and can significantly shift the output (as stations jockey for position in the “adjust and modify” processes inside GIStemp. (A station must have 20 years of data to be used at all, for example, so a station that did NOTHING with 19 years of data may, in the next year, be changing the results of stations 1200 km away…)
It’s just an exorcise is instability of output, fundamental to the way it is designed. As to the question of “Malice or Stupidity?” – well, that gets to motivation and I can not address that. One could have maliciously chosen to exploit that behaviour, or one could be simply a bit too dense to realize it’s a dumb thing to do and “sucked their own exhaust” or as we say in programmer land “believed their own BS”. Others will need to address that question.
Having year to date data points on annual graphs also happens on station charts from GISS as well.
I have been tracking changes because I wanted to know the latest figures. It surprised me that they (seem to be?) constantly changing data from the past. This can’t be something that is a secret, can it? http://www.changedetection.com/log/gov/nasa/giss/data/glb2_log.html
They are very naughty boys and their fibs are being found out: http://endisnighnot.blogspot.co.uk/2012/03/giss-strange-anomalies.html
You think GISS is bad? Try the Aussie BoM or New Zealand’s NIWA for shonky temperature data and data-filing habits! These are supposed to be professional, technically advanced and completely reliable scientific bodies responsible for looking after national temperature records. Instead they behave like tin gods and petty tyrants doing Primary school science projects. They should be sacked the lot of them!
I knew about the no wayback copy back in 2008.
When I noticed the sudden change in 2009 just before the Copenhagen conference, I decided to D/L the FigD every month. Much to my chagrin, I had a disc crash & haven’t been able to afford to recover the data. (It was about the only data I had been remiss about backing up too!). 🙁
DaveE.
FYI: I put together plots of annual average U.S. temperature (with observation counts, 2012 partial) and average July U.S. temperature (again, with observation counts). See http://blog.qtau.com/2012/08/average-us-temperature-in-ghcnv3.html
Hope they help put things in perspective 😉
Anthony
While you are on a roll correcting various misleading temperature charts put forth by taxpayer funded entities, perhaps you can address the following one that has been annoying me for the last 6 months or so.
On the NOAA National Climatic Data Center website, Global Surface Temperature Anomalies page;
http://www.ncdc.noaa.gov/cmb-faq/anomalies.php
the chart shown;
http://www.ncdc.noaa.gov/sotc/service/global/global-land-ocean-mntp-anom/201001-201012.gif
ends at 2010, and thus excludes 2011 data, which showed a significant drop, as illustrated by this somewhat comparable chart from the MET Office;
http://www.metoffice.gov.uk/hadobs/crutem3/diagnostics/global/nh+sh/annual_bar.png
and by NCDC’s data found here;
ftp://ftp.ncdc.noaa.gov/pub/data/anomalies/annual.land_ocean.90S.90N.df_1901-2000mean.dat
with last decade listed below for easy reference:
2001 0.5325
2002 0.5931
2003 0.6033
2004 0.5614
2005 0.6355
2006 0.5803
2007 0.5728
2008 0.4946
2009 0.5766
2010 0.6346
2011 0.5081
Regardless of the reason why this chart hasn’t been updated, oversight or otherwise, it is not appropriate for a taxpayer funded entity to provide an outdated chart on such an important subject.
@ansgarjohn No secret. You are looking at two different datasets Your URL redirects.
http://data.giss.nasa.gov/gistemp/tabledata_v2/GLB.Ts+dSST.txt
http://data.giss.nasa.gov/gistemp/tabledata_v3/GLB.Ts+dSST.txt
No doubt it’s part of the transition from GHCN-M V2 to V2
http://data.giss.nasa.gov/gistemp/updates_v3/V3vsV2/
I mean No doubt it’s part of the transition from GHCN-M V2 to V3
Brent Hargreaves says:
August 21, 2012 at 3:05 pm
They are very naughty boys and their fibs are being found out:
Here is how I see this Arctic issue. With the circumference of Earth being about 40000 km, the distance from 82.5 to 90 would be 7.5/90 x 10000 = 830 km. So the area in the north NOT covered by RSS is pir^2 = 2.16 x 10^6 km2. Dividing this by the area of the earth, 5.1 x 10^8 km2, we get about 0.42% NOT covered by RSS. (It just seems to be the north pole that is the issue with low ice, etc.) The anomaly for RSS for 1998 was 0.55, while it was 0.476 for 2010. The anomaly for GISS for 1998 was 0.58, while it was 0.63 for 2010. The relative difference between 0.074 lower and 0.050 higher is 0.124. If it is assumed that this 0.124 is due to 1/230 of the area of the earth, then that part above 82.5 degrees must have been 230 x 0.124 = 28.5 degrees warmer in 2010 than in 1998. There is no way that this was the case. And applying Hadcrut3 statistics to the issue gives about the same results (27.6 degrees). Then there is the issue of why this polar amplification had such a huge affect in 2010 but virtually no effect in 1998.
(P.S. You are welcome Theo!)
Just The Facts says:
August 21, 2012 at 4:48 pm
ends at 2010, and thus excludes 2011 data, which showed a significant drop
Good point! Then there are both Hadsst2 and Hadcrut3 that have not been updated so WFT can use them since March. Hadcrut3 is being replaced by Hadcrut4. But that only goes to the end of 2010 as well. So how can I tell the time period that Hadcrut4 would show a flat slope if it were up to date? Below is how I got around this problem and if I can help you do something similar to answer a different question, I would be happy to try to help.
Hadcrut4 only goes to December 2010 so what I did was get the slope of GISS from December 2000 to the end of December 2010. Then I got the slope of GISS from December 2000 to the present. The DIFFERENCE in slope was that the slope was 0.0049 lower for the total period. The positive slope for Hadcrut4 was 0.0041 from December 2000. So IF Hadcrut4 were totally up to date, and IF it then were to trend like GISS, I conclude it would show no slope for at least 11 years and 8 months going back to December 2000. (By the way, doing the same thing with Hadcrut3 gives the same end result, but GISS comes out much sooner each month.) See:
http://www.woodfortrees.org/plot/hadcrut4gl/from:2000/to/plot/hadcrut4gl/from:2000.9/trend/plot/gistemp/from:2000/plot/gistemp/from:2000.9/to:2011/trend/plot/gistemp/from:2000.9/trend
E.M.Smith, you wrote:
You didn’t mention that 1. the interpolation and extrapolation is done only up to a defined distance, above this distance the area is not filled, but marked as not covered, 2. that the uncertainty from lack of coverage is estimated in the analysis. It can be seen as green error bars:
http://data.giss.nasa.gov/gistemp/graphs_v3/Fig.A2.gif
You also didn’t mention that the robustness of the analysis to changes in the number of stations has been tested and published in the respective scientific publications.
You also didn’t mention that, in addition to the analysis with an extrapolation up to 1200 km, the analysis is also done with a more restricted extrapolation up to 250 km, and also for meteorological stations only.
That is not fully correct. Changes in the NCDC adjustments to the GHCN data don’t have a ripple effect on the GISS analysis, since the scientists at GISS who do the analysis use the unadjusted GHCN data and apply their own adjustments to account for the urban heat island effect.
Again, you also didn’t mention that the robustness of the results to larger changes, for instance when the version of the USHCN data are updated, is being tested.
Do you have any examples where changes in the data input changed the results of the GISS analysis statistically significantly on a global or larger regional scale?
What is your suggestion, anyway? That no updates to older data sets should ever be done, even if there are newer versions available? Or new data should not be included in the analysis, as they become available?
“…Anthony, there’s no error. It’s just the chart doesn’t extend high enough for this year.That’s the data for the year to date, not July.
As you can see from this NOAA chart: http://www.ncdc.noaa.gov/temp-and-precip/time-series/index.php?parameter=tmp&month=7&year=2012&filter=ytd&state=110&div=0. The mean temperature for the year-to-date is 56.4F, or 13.6C. The normal is 52.2F, or 11.2C. The departure is 2.4C or close to what GISS is reporting…”
And as I pointed out on an earlier post, NCDC shows a 20th century average of 60.4F, or 15.8C. Using that goalpost, GISS’s year-to-date of 56.4 is still a full 4 deg F below the 20th century average.
Using NCDC’s average, and GISS’s data, 2012 will end up well below the 20th century average.
No worries, then.
May have mentioned this before: if at any time you run across something on the net that you would like to make CERTAIN doesn’t disappear under the erasing efforts of a diligent Winston Smith someplace you can preserve it for posterity with iCyte. Here’s an example: I just cited the NASA graph page and it is stored at:
http://www.icyte.com/saved/data.giss.nasa.gov/611581
Now if by any chance the problem was corrected/erased and someone tried to deny it ever existed, all you’d have to do is offer that link as evidence. Here’s an example of how I’ve used it in the past when I felt I was being libeled by James Repace in his efforts to portray me as a tobacco industry “mole”:
http://www.icyte.com/saved/www.smokefreedc.org/538500
My defense comment never DID make it out of invisible “moderation,” but several months later, after more publicized attacks on this, the site *did* remove Mr. Repace’s comment and replaced it with a short (and probably made-up) generic antismoking comment dated around the same time (If you click on the little “S” near the upper right of iCyte screens you can see what the page in question currently looks like.)
While this tool may not be thought to be necessary on something as “official” as a page from NASA, I wouldn’t be too sure of that. I’ve seen some fairly official stuff fiddled with in the past. The attacks on “Climate Deniers” will probably escalate as you manage to get more people to actually examine the data and listen to your arguments … so prepare your defenses well!
– MJM
As A says, it’s wrong! YTD and and Mean are NOT the same thing. period.
GISS – Grant Income Supported Storytellers
Jim P. says:
August 21, 2012 at 1:50 pm
Now wait a moment.
You and GISS and G. Schmidt are telling us that you don’t use autoscaling in GISS graphs, but that you happily draw lines way above the caption if a measurement value goes up enough?
Can you please point me to other GISS measurement values and published graphs that behave like that. It must happen regularly as GISS doesn’t use autoscaling.
Jan P Perlwitz says:
August 21, 2012 at 5:41 pm
“What is your suggestion, anyway? That no updates to older data sets should ever be done, even if there are newer versions available? Or new data should not be included in the analysis, as they become available?”
How did you cool down 1934 so much? That’s your masterpiece.
Looking at the html source and the excluded data (see: http://www.ncdc.noaa.gov/robots.txt) it appears the graphs are being rendered by a cgi script and thus not archived. This may be unintentional on the part of noaa but it still has the effect of preventing the charts from being archived. The robots.txt file in a web site’s root sets the bot policy of the web site for those bots that respect it.
As for showing month-to-date data (7/12 of year) mixed with annual data provides a ridiculous result. Good catch.
I love it how Dr. Jan Perlwitz of NASA GISS can just completely ignore the problem pointed out by this post and instead worries about E.M. Smith’s comment of “not fit for purpose” about the GISS 30 year old spaghetti code.
You can tell a GISS man, but you can’t tell him much.
DirkH says:
August 21, 2012 at 9:11 pm
Jan P Perlwitz says:
August 21, 2012 at 5:41 pm
“What is your suggestion, anyway? That no updates to older data sets should ever be done, even if there are newer versions available? Or new data should not be included in the analysis, as they become available?”
————————————————————————————————————–
DirkH: How did you cool down 1934 so much? That’s your masterpiece.
============================================================
Good question. Just how do you come up with a “newer version” of something somebody wrote down on a peice of paper before you were born? Wave a magic pencil at it?
If you don’t know the details of the individual sites, you have zero justification in changing the numbers in the records.
If they don’t show a hockey stick, so be it.
Well at least you folks HAVE temperature data from 2012 to complain about. I wanted to do some comparisons of temps in the Canadian prairies during the dirty 30’s to this year, but the Environment Canada site has no data after 2007. WUWT?