NASA GISS Version 3 vs. Version 2, using HadCRUT.4 Version differences as a baseline
Guest essay by David Dohbro
Recently the climate blogosphere has uncovered the effects of adjusting past and present United States’ land-temperature data as measured by the United States Historical Climatology Network (USHCN) and how that possible affects the temperature records as well as our understanding and knowledge of historic temperatures (References 1-3).
To that extend it is prudent to also look at the effects of temperature adjustments on a global scale. Here the National Aeronautics and Space Administration’s (NASA) Goddard Institute for Space Studies (GISS) GLOBAL Land-Ocean Temperature Index (GLOTI) previous Version 2 and the most recent Version 3, which includes the month of June 2014, are compared against each other (4, 5). GISS was updated for various reasons, which I won’t detail here.
In addition, early last year the collaborative product of the Met Office Hadley Centre and the Climatic Research Unit at the University of East Anglia, HadCRUT.4 Global Surface Temperature Anomaly data set, was update from version 4.1.1.0 to 4.2.0.0 (6, 7). The difference between these two versions is negligible and shows no temporal trend (7). I also won’t elaborate on the changes between these two versions, as that’s a whole different discussion.
Namely, I simply wanted to assess if and how adjustments have affected the new NASA’s GLOTI data record (Version 3) versus the previous Version 2, and how those differences compare to the differences of the HadCRUT.4 versions’ update. However, since GISS Version 2 was replaced by Version 3 in November of 2011, the GISS comparison can only be made for data available till then. First, annual differences were assessed and then monthly differences. Differences are in this case simply the data from GISS Version 3 minus the data from GISS Version 2 (either annually or monthly). Then these differences in GISS are compared with the differences in HadCRUT.4
Below is a graphic showing the Version 3 GLOTI-values in red squares and the Version 2 GLOTI-values in blue diamonds.
On average, the difference between Version 3 and Version 2 GLOTI annual-averages data is 18% +/- 143%, ranging from -1280% (year 1947) to 629% (year 1960). However, since it is hard to discern the difference between the two Versions in an absolute value format, a better representation between Version 3 and Version 2 on an annual basis, is by plotting the actual annual differences (see below).
It follows from these two graphs that Version 3 and 2 do not produce the same annual GLOTI values; otherwise all annual differences would be 0. Instead, it can be observed that currently
1) From 1880 to the mid-1890s, Version 3 has higher annual GLOTI values (warmer) compared to Version 2 up to a difference of almost 0.10ºC.
2) From the mid-1890s to late-1960s, Version 3 has lower annual GLOTI values (colder) compared to Version 2.
a. In addition, from the mid-1890s to ~1910 the Version 3 annual GLOTI values become increasingly less compared to Version 2 (up to a difference of 0.12ºC)
b. While from ~1910 to late-1960s the difference –in general- becomes smaller
3) From the late-1960s to late-2011 Version 3 annual GLOTI values are higher (warmer) than Version 2’s reported values, with an increase till 1999 and a slight decrease since.
In simpler terms: the late 19th century is (made) warmer in Version 3, whereas the early to mid 20th century is (made) cooler in Version 3 especially at the beginning of the past century; compared to Version 2. In addition, the last quarter of the 20th century till the most recent past is also (made) warmer in Version 3 compared to Version 2.
These differences can also be assessed in more detail on a monthly scale. Below is the same plot as the previous graph but instead of annual-averages data, the actual monthly data values are used.
The above plot shows –of course- the same temporal patterns as the annual plot, but now it can be observed that some monthly values differ by over 0.20ºC, with most differences in monthly values between -0.10ºC and 0.10ºC. In addition, the green line shows the 0-value (no difference between both Versions); the blue, red and orange lines are simple linear regressions for the periods 1880-1910, 1910-2011, and for all data, respectively using Instat+ (8). The regression lines are plotted to exemplify the observations made from the annual differences.
As mentioned, the HadCRUT update from version 4.1.1.0 to 4.2.0.0 resulted in negligible differences and shows no temporal trend (7). As a check, I have plotted the HadCRUT.4 differences between versions myself below, which is identical to what is shown on their website, except for a few outliers that are not shown, but which don’t affect the general pattern (7).
It is obvious that the change from HadCRUT.4.1.1.0 to version 4.2.0.0 has not lead to any changes in reported GSTA or its temporal trend. This is in very stark contrast to the differences between GISS’ Version 2 and 3, which shows a strong cooling of the past and strong warming of the present in Version 3 vs. Version 2. These obvious differences in GLOTI-values require attention and understanding of their causes, especially in light of the fact that HadCRUT does not exhibit any significant differences in GSTA-values between versions.
The problem with trying to change the past is that we won’t understand the present and will be unable to predict the future correctly. Hence, policies and regulations based on analyses of adjusted past data and aimed at addressing supposed future climate will be ineffective.
References
3) http://notalotofpeopleknowthat.wordpress.com/2014/07/09/analysis-of-ushcn-dataset/
4) http://data.giss.nasa.gov/gistemp/tabledata_v2/GLB.Ts+dSST.txt
5) http://data.giss.nasa.gov/gistemp/tabledata_v3/GLB.Ts+dSST.txt
6) http://www.metoffice.gov.uk/hadobs/hadcrut4/
7) http://www.metoffice.gov.uk/hadobs/hadcrut4/data/versions/HadCRUT.4.2.0.0_release_notes.html
8) http://www.reading.ac.uk/ssc/n/n_instat.htm
=============================================================
NOTE: this essay as originally posted was missing figures 3 and 4, which somehow did not copy from the MS Word document, but the surrounding text did. I didn’t get notice of this error until after 7PM tonight due to being distracted by other issues. The figures are now updated. -Anthony
Missing the monthly graph, unless the intent was to point to a link?
The problem with trying to change the past is that we won’t understand the present and will be unable to predict the future correctly.
====
exactly….since the “best” they can do is extend a trend line……you can’t predict anything when your frame of reference is a wonky trend line to begin with
An editor is needed here. …..Lady in Red
“To that extend it is prudent …”
Uh, “To that extent…”
“The problem with trying to change the past is that we won’t understand the present and will be unable to predict the future correctly.”
——————————-
Not exactly….
The problem is that you can NOT predict the long term future of chaotic systems at all. It is no help to us that characterizing the past at best indicates the probable boundaries of the future. These futures possess a known probability of exceeding the past boundaries. For example, based on the past, we have the probability of any year exceeding it’s past max or min or whatever boundary of 1 divided by the observed period. Any year has a 1 in 100 possibility of exceeding the 100 year flood based on 100 years of observation.
So we have a non-zero probability of new record values, and near certainty that the future behavior will not behave as has the mean of past observations.
… models always forecast too warm
… historical data is always adjusted cooler in the past and warmer in the present
Are there any exceptions to this ? There must be somewhere.
This is slightly off topic but along the same line: The recently released data from the USRCN shows the US has cooled by 0.4 C in the last ten years. Is there a graph that shows the comparison with the USHCN over the same period?
So, best is to forget about the past and look only at recent history
There is no man made global warming, not even earthly influences
see graph below the minima table.
http://blogs.24.com/henryp/2013/02/21/henrys-pool-tables-on-global-warmingcooling/
I’m glad I did not live at the beginning of the 20th c. , it’s getting colder by the day 😉
Mary Brown says:
July 17, 2014 at 10:54 am
… models always forecast too warm
… historical data is always adjusted cooler in the past and warmer in the present
====
No Mary, you don’t understand. Older data was biased warm and has to be corrected.
Every year the old data is a bit older, so each year it needs correcting a bit more.
You’re new at climatology, aren’t you ? 😉
@ Greg Goodman –
Believe what you described is “Climastrology”, isn’t it?
The corollary is also true:
Climate models are calibrated against the climate record. This means that they will necessarily bias the future warm. As we go forwards the this future warming bias will be reduced. So each year they will need to reduce the warming in the models to compensate.
By the time we get to 2100 the models will be bang on target.
I think you need read M. Mann’s new book: ” Climate whores, dispatches from the front line”, he explains how all this works.
Nice work. A good in-depth look at this issue.
Fleshing out a lot of what Steve Goddard does, for those who can’t read a simple graph!
Hence, policies and regulations based on analyses of adjusted past data and aimed at addressing supposed future climate will be ineffective.
Depends what is meant by “ineffective”. I’m sure with Gavin forcing an severe upward trend into the future from the 1900s the data will be really effective
Sorry about the unintended missing monthly GISS-value differences plot, and the missing HadCRUT.4 differences plot. You can find these two figures here :https://www.dropbox.com/s/rdw9ete3m4kf13j/figure%202%20and%20figure%203.pdf
I hope this helps!!
It is even worse than that, HADCRUT 4 is a manipulated data set, maing current warmer and past colder.
http://climate4you.com/images/HadCRUT4%20GlobalMonthlyTempSince1979%20With37monthRunningAverage.gif
Greg Goodman said:
“I’m glad I did not live at the beginning of the 20th c. , it’s getting colder by the day ;)”
To support UNFCCC they will have to cool the beginning of the 20th c by another 4 deg C by year 2100?
🙂
Who controls the past controls the future. Who controls the present controls the past.
George Orwell
For some Orwell’s 1984 is not warning about the nature of dictatorship but an instruction manual of how to behave.
And by ‘lucky chance ‘ all these adjustments always work in favour of ‘the cause’ and those that profit from it , never against. Climate ‘scientists’ are wasting their time , they should be at Vegas with that ‘luck’ they be multimillionaires .
Stop! My grandparents just froze to death before my parents were born!
now we know where Enron’s accounting department ended up.
When you adjust the data, it isn’t science.
Greg Goodman says, at 11:18:
“I think you need read M. Mann’s new book: ” Climate whores, dispatches from the front line”, he explains how all this works.”
I’m thrilled to hear about the new book.. He may get a Pulitzer for it, or even a second Nobel prize, this time for literature!
Rather than only comparing only versions of estimates of temperature data, you should compare estimates with actual measurements too.
Dave L. says:
July 17, 2014 at 12:37 pm
When you adjust the data, it isn’t science.
Not quite. When you adjust data it ceases to be data and becomes either information or results.
philip lee ,unfortunately climate science does not work like that .there would be no point making a comparison to the raw data,as in climate science the raw data is always incorrect .
you would be comparing raw data against the very finest adjusted data government money can buy, no raw data is worthy of comparison to that created by the finest minds in climate science 😉
“cooling of the past and strong warming of the present”
What a coincidence.
Thats exactly what happened in Australia when our bureau of Meteorology adjusted our historical temperature records.
And in New Zealand same thing and in England same thing and now America gosh golly same thing………
Anybody noticing the wood through the tree’s here?
People are not stupid, they just need prodding into action against these fraudsters.
These “adjustments” are being physically and systematically made around the world to enhance the fraud.
They are NOT accidental.
Read here and see if you can identify any other “accidental coincidences”.
http://joannenova.com.au/tag/australian-temperatures/
Dave L. says:
July 17, 2014 at 12:37 pm
When you adjust the data, it isn’t science.
Adjustment is never an ideal step, but quite often is a legitimate step. We broke a 100-meter tape once during field survey work – losing the first 12.7 cm of the tape. With no spare we continued to work, but every measurement had to be “adjusted” to yield a proper distance – we also adjusted the tape, trimming it so an even 15 cm was missing, which made “adjustments” easier. At the same time we also had to take into account tape sag, which increased with distance, slope inclination, and tension on the tape, which were considerably more difficult to track.
The problem is not adjustment, but vaguery in what needs adjustment, why it needs adjustment, and which parts of the record really need it.
Duster on July 17, 2014 at 3:40 pm
Dave L. says:
July 17, 2014 at 12:37 pm
…
The problem is not adjustment, but vaguery in what needs adjustment, why it needs adjustment, and which parts of the record really need it.
The problem with adjustment is when it is done by social animals such as humans which have a clear socio-economic interest in the direction and outcome of the adjustment.
http://joannenova.com.au/2014/06/australian-bom-neutral-adjustments-increase-minima-trends-up-60/
Duster, here’s but one of the many thousands of “adjustments” I’m talking about.
That have been made right through our historical temperature record here in Australia.
With zero explaination.
And if you think these “adjustments” are accidental or just local you really do need to read up a bit more.
The link helps you to get a handle on just what the fraudsters are about.
Our adjustments are not the result of a broken tape.
“The most extreme example that Ken found of data corruption was at Amberley, near Brisbane, Queensland, where a cooling minima trend was effectively reversed,”
[Deleted the “duped” text. .mod]
Opps trying to get the link to the bottom and posted twice.
Sorry about that.
Adjustment is never an ideal step, but quite often is a legitimate step.
Absolutely (and highly regrettably) true. I have adjusted data, and dropped stations from the record for moves, for TOBS, and adjusted for MMTS conversion. Both times, the trend was warmer than when I started, too, I might add. But the idea is to get it right.
Don’t worry, we’ll also present the data for the stations we had to drop. And, of course, all the raw data. Although I haven’t done a final rating review of all of the ones we had to drop, since we had to drop them anyway. I can see myself going back and trying to do it.
I can see myself having to go back and do that someday. It never really ends. We are just lucky to have a reliable 30-year stretch that shows a clear warming signal that enough stations for a representative sample.
And that brings us back to oversampling. NOAA has done that, and it is for that reason there are enough eligible stations to be representative. And our final “adjustment”, also to address a criticism from 2012 on gridding, we will grid using NOAA’s “9 Areas” (and weight for area). We will, of course, provide the straight averages, as well.
To NOAA’s credit, they oversampled. Whatever else they did wrong, they did do that. That took a lot of time and effort by a lot of volunteers. And whoever has been fixing up the USHCN metadata deserves a raise.
Had to learn how to do seasonal forecasting of sales figures for a customer recently, and in reading the literature on how to do it, I discovered that you could not have any missing data points. It reminded me at once of the “adjustments” in the climate world–filling in missing data points are they? Isn’t there a way to fill them in so that they are neutral to the outcome of the forecast? Aren’t there any more sophisticated tools available to scientists than the students’ version of Excel that I am using for forecasting? I sincerely hope that science is a bit more sophisticated than that.
“Absolutely (and highly regrettably) true. I have adjusted data, and dropped stations from the record for moves, for TOBS, and adjusted for MMTS conversion. Both times, the trend was warmer than when I started, too, I might add. But the idea is to get it right.”
Problem is you cannot trust the metadata without a cross check.
For example. The record may show NO move, but the actual agency that controlled the site ( not noaa) will show a move.
That is why you have to cross check for the following
1. undocumented changes in TOB
2. in documented changes in instrument
3. un documented stations moves
You can do this with break point analysis. its simple. you WITHHOLD the metadata.
you then run your breakpoint code. the breakpoint code will predict
changes at time a, b, c, d . you then look at your metadata and see how many you corrrectly
identify.
Trusting metadata is probably the worse mistake one could make. I hope you didnt trust it.
in other words just because a record fails to identify a change does not mean there wasnt a change
you need cross checks. from other metadata sources and from the time series itself
REPLY: “The record may show NO move, but the actual agency that controlled the site ( not noaa) will show a move.”
Mosher, this is ridiculous. All the COOP stations are managed by NOAA, local NWS WSFO’s there is no other “agency” with station metadata. Don’t make stuff up. – Anthony
“Mosher, this is ridiculous. All the COOP stations are managed by NOAA, local NWS WSFO’s there is no other “agency” with station metadata. Don’t make stuff up. – Anthony”
With Coop stations yes. But even there you cannot simply TRUST the metadata without a cross check.
years ago when your access was cut off to NOAA metadata I did an FOIA of noaa to get the mails surrounding the B-91 and the metadata. The mails indicate that the station history and metadata wasnt
accurate. So as with everything one cant simply trust the records. In short. metadata has been cleansed, adjusted, “fixed” reconciled. Dont assume its perfect. It may be. but assuming its perfect will lead to mistakes.
So t least we can agree on that. Trusting metadata with no cross check isnt exactly safe.
[Mosher adds in an email: Other stations Anthony. I’m speaking in general. But I hope you cross checked the records.
Don’t trust them with out a cross check. ]
@Steve Mosher
The NOAA TOBS data comes right off the B-91 or B-44 (etc.) forms. I don’t see how you can get a more reliable record than that. The originals are available as PDF.
We prefer to find the cleanest set of stations we can possibly find and adjust as little as possible. Break points can occur naturally. Unless you have a specific reason for that break point there is insufficient cause to adjust. And if there is, we do not adjust — we drop.
evanmjones says:July 17, 2014 at 5:20 pm
“Absolutely (and highly regrettably) true. I have adjusted data, and dropped stations from the record for moves, for TOBS, and adjusted for MMTS conversion. Both times, the trend was warmer than when I started, too, I might add. But the idea is to get it right.”
Hundreds of times?
And every instance to give an appearance of temperature trending up!
They “adjusted” the temperature “trend” down pre 1950.
They then “adjusted” post 1950 “trend” up and dramatically so after 1970.
Viewed as a graph line the affect is as dramatic as it was intended to be and gives the desired affect for the global warmist “adjuster”.
Any one for hockey?
They then discarded pre 1910 temperature records where temperatures were warmer stating that “they were unreliable”.
A bit like Mann and co. removing the medieval warm period.
Those same unreliable pre 1910 Australian temperature records the IPCC has no qualms about using.
What it is clearly showing us is that more than half of the alledged 1.03 of a degree rise in Australia’s temperature of the last century can be clearly sheeted home to “adjustments”.
And the scientists wonder why us untrained skeptics question them.
Believe it or not some of us can read.
Here’s a tip if you want to get it “right”.
Stop mucking with the raw data.
Here’s a tip if you want to get it “right”.
Stop mucking with the raw data.
I really wish that were true. It would make things so much easier.
(We know how NOAA went wrong with their adjustments.)
I understand the adjustments and I think most are correct. I do suspect, however, that the “favorable” ones are done with enthusiasm while the urban heat islands are neglected or partially botched. The adjustments are kind of like when the Detroit Tigers moved from Tiger Stadium to Comerica Park in 2000. Tiger Stadium was a sluggers park. Comerica is a spacious pitchers park. Did pitchers suddenly improve when they moved to Comerica? No. Add to this the drop in slugging with the end of the steroid era and you have a statistics nightmare…or dream, depending on your outlook. Sometimes the game changes. Sometimes, the venue changes. Sometimes, anthropogenic home run enhancement occurs (steroids). Separating time of day, urban heat islands, site changes, natural climate change and human induced change is tricky. Check. Recheck. Be skeptical.
If you need complex statistics to squeeze out a few tenths of a degree in a human lifetime, then it’s probably not much to worry about.
the line of red squares 1910-1940 is just too perfect to be true
“I’m glad I did not live at the beginning of the 20th c. , it’s getting colder by the day ;)”
Yeah. Hehehe it is strange how the past kepps getting colder.
” Lady in Red says:
July 17, 2014 at 10:28 am
An editor is needed here. …..Lady in Red ”
Why? Most of us had no trouble reading and understanding, so what was your problem? This is a blog, not a professional journal, so why does everything need to be perfect – assuming it isn’t.
Don’t worry about it. The massive La Nina s that are coming; the Antarctic Ice increases; and the Arctic Ice recovery will destroy the Global Warming via CO2. I just hope that it doesn’t snow in Kansas in July destroying crops!!
Mary Brown says: July 17, 2014 at 9:36 pm
I understand the adjustments and I think most are correct.
Have you looked at the latest series of TOBs posts by Steve Goddard?
They may change your mind about correctness.
http://stevengoddard.wordpress.com/2014/07/16/another-smoking-gun-that-tobs-is-crap/
http://stevengoddard.wordpress.com/2014/07/16/putting-the-tobs-scam-to-bed/
http://stevengoddard.wordpress.com/2014/07/16/tobs-even-worse-than-it-seems/
http://stevengoddard.wordpress.com/2014/07/16/another-smoking-gun-that-tobs-is-crap/
http://stevengoddard.wordpress.com/2014/07/17/us-summer-afternoons-cooling-for-120-years-tobs-has-no-effect/
http://stevengoddard.wordpress.com/2014/07/18/another-nail-in-the-tobs-coffin-2/
And then there is UHI
http://stevengoddard.wordpress.com/2014/07/17/climate-joke-of-the-day/
A C Osborn says:
“Have you looked at the latest series of TOBs posts by Steve Goddard?
They may change your mind about correctness.”
He has some interesting stuff on his pages, especially the historical newspaper stories, but I don’t trust his computation skills so I generally don’t look at that kind of data on his site.
Mary Brown:
At July 18, 2014 at 11:02 am you say
it is amazing what one does not see if one chooses not to look.
And it is amazing what AGW-advocates choose not to see; e.g. multiple data adjustments, MWP, the ‘pause’, etc.
Richard
for Richard…
I’m a skeptic. And I’m skeptical of Steven Goddard.
I’m skeptical of urban heat island treatment.
I’m skeptical of the other adjustments but I don’t think they are nearly as bad as Steven G would have you believe.
evanmjones says:
July 17, 2014 at 5:20 pm
I come from a different area of computing so forgive me if I ask for somethiing you may not have considered. Have you published the verification and validation testplans and results? Have they also been published for the homogenization adjustments? I am sure that if these were published along with the original and modified data, that people would have a lot more confidence. Indeed, I would expect that the testing information would be a standard deliverable with the output. But then as I said I come from an area of computing that getting it right is a fundamental requirement.
I can say this much: We think we have discovered what is going on with adjustment and why homogenization is failing.
We adjust only for MMTS conversion (we wish that were not necessary, but it is). We will provide the raw data for comparison. We dropped stations for TOBS bias and moves (this has much the same results of adjustment, but without any actual adjusting). We will provide the data for the stations we dropped as well, so as to forestall concerns about cherrypicking.
I had better not go into any more detail at this time.
I’m skeptical of the other adjustments but I don’t think they are nearly as bad as Steven G would have you believe.
As we found in 2012, failure to consider microsite bias, compounded by homogenization, appears to be the largest problem with the surface record.
Yeah. Hehehe it is strange how the past keeps getting colder.
Seems reasonable. I used to walk two miles to school uphill both ways in the snow. And the older I get the colder I remember it.
Jones, Mosher, all other “data adjusters,”
You cannot make a silk purse out of a sow’s ear. Error bars? Error bars, anyone? Bueller? Anyone? Metrology? Validation and verification as Ian W also requests? Calibration records? These thermometers are installed to report Weather, and generating Climate reports to the hundredth or even thousandths of a degree is simply lying.
I don’t see how self-respect is possible when one’s job is to present a plausible Big Lie to the public. The BEST people do it, but the best people never do…
Hence, policies and regulations based on analyses of adjusted past data and aimed at addressing supposed future climate will be ineffective.
It bears repeating: It depends on what is defined as the desired effect. The effect they desire may be quite different from the one we might expect. I’m not so much a climate skeptic as I am a skeptic of governmental benevolence.
You cannot make a silk purse out of a sow’s ear.
I hate adjustments more than even you. But there is no way to avoid adjusting for MMTS conversion (it is not a large adjustment). But that is just one adjustment. We wish we could avoid it. We can’t. We do avoid all of the other obnoxious, presumptive adjustments. We drop the moved and TOBS-biased stations rather than adjusting. The result is much better than a sow’s ear. But we can’t avoid it entirely.
I see no way to get away from MMTS conversion adjustment, and believe me I have tried.
We will provide the raw data results, also, of course.
I only trust my own data
http://blogs.24.com/henryp/2013/02/21/henrys-pool-tables-on-global-warmingcooling/
which proves there is no man made warming
but
there is natural cooling
coming up
Apparently USA is mostly affected