GISS Hockey-Stick Adjustments

You need to make a short animated movie out of that. Two frames per month. Watch the line dance.

http://floridamemory.com/fpc/JJS/JJS0078.jpg

Do they have an explanation of why they are doing this?

Thanks Walter. Man made. Global Warming. To order.

Does anyone have GISS versions that predate 2005?….
http://stevengoddard.wordpress.com/2014/07/03/giss-data-tampering-worse-than-it-seems/

This is the earliest I have come across in numeric data. Global temperatures 1880-1993, Global first, NH second, SH third.
http://cdiac.ornl.gov/ftp/trends93/temp/hansen.612
Abstract: This data set represents temperature changes over the past century
(1880-1993) calculated from surface air temperatures published in the
“World Weather Records” and the World Meteorological Organization’s
(WMO) Monthly Climatic Data for the World, supplemented by monthly
mean station records available from NOAA’s Climate Analysis Center.
At each gridpoint, data from nearby stations are combined to form an
estimate of the temperature change (Hansen and Lebedeff (1987)).
Data is presented as temperature anomalies relative to a reference
period of 1951-1980 in degrees Celsius for the globe, Northern
Hemisphere, and Southern Hemisphere.
The citation for this dataset is:
Wilson, H., and J. Hansen. 1994. “Global and hemispheric temperature
anomalies from instrumental surface air temperature records”,
pp. 609-614. In T.A. Boden, D.P. Kaiser, R.J. Sepanski, and F.W. Stoss
(eds.), Trends ’93: A Compendium of Data on Global
Change. ORNL/CDIAC-65. Carbon Dioxide Information Analysis Center, Oak
Ridge National Laboratory, Oak Ridge, TN, USA

So it is clear that this is not random, this is “history re-engineering.” My question is: is this some lone ranger weather guy on a political crusade? Or is there some new algorithm that is faulty? Or is there a dedicated group (in which why has there been no leak?). Or has this been achieved through policy decisions at a higher level, in which case those should be visible?

Have you tried http://www.climate4you.com ? They seem to have even older data and show a similar analysis for all major temperature sets on their website.

Possibly a representative from NASA could explain the reasons for the adjustments… from the witness stand… under oath.

This may reveal the deeper meaning of the term “anthropogenic global warming.”

MattN says:
> July 3, 2014 at 5:17 pm
>
> Do they have an explanation of why they are doing this?
Near the beginning of the post where I say
> GISS lists its reasons for adjustments at two webpages:
There are links to 2 webpages with their rationale…
http://data.giss.nasa.gov/gistemp/updates/
http://data.giss.nasa.gov/gistemp/updates_v3/
——————————————————————–
From this layman’s superficial reading of the above rational for alteration, the changes appear to be normal, necessary and benign. However, the changes would appear to reflect unfavorably on the question of the level of certainty regarding any temperatures, the level of certainty regarding the change in temperatures over time, and regarding the characterization of a “settled science”.

For those here who are not fluent in Swedish, I -since I am- can attest that the comments made by norah4you above, are spot on in their understanding of the sleigh of hand that has been going on.

GISS has already answered your questions at their Q&A site,
http://data.giss.nasa.gov/gistemp/abs_temp.html
Excerpt:
Q. If SATs cannot be measured, how are SAT maps created ?
A. This can only be done with the help of computer models, the same models that are used to create the daily weather forecasts. We may start out the model with the few observed data that are available and fill in the rest with guesses (also called extrapolations) and then let the model run long enough so that the initial guesses no longer matter, but not too long in order to avoid that the inaccuracies of the model become relevant. This may be done starting from conditions from many years, so that the average (called a ‘climatology’) hopefully represents a typical map for the particular month or day of the year.
It is not real, just the latest version of the GISS Fairy Tale for your entertainment!

cynical scientst said:
I doubt this is the work of one individual. To mess up this badly you require a committee
I second that opinion

Walter,
Thanks for the analysis. You are showing the local trends as discontinuous lines, but that’s only fair to GISS if they really have step-wise adjustment changes at the transition times. If they don’t, would it not would be fairer to present this as connected trend lines? Basically, you’d change from a fit on 6×2=12 variables to 6×1+1=7 variables.
Frank

“My question is: where are the politicians with the guts to acknowledge that the whole CO2 story has reached the end of the line, and that no more money should be wasted on this fairy story?”
When the practical consequences of ‘climate change’ policies start to hit the ordinary voter in his or her pocket in a significant and easily seen manner, politicians will gain the balls to face down the pro-AGW brigade. We have already seen this in the UK, where rising domestic energy prices are now a political hot potato. Politicians are still managing to hold two conflicting ideas in their heads, namely being against ‘fuel poverty’ and being pro-AGW at the one and same time, despite their policies to combat the latter actively causing the former. But the fact that they now have two ideas in there when before they only had one is itself progress. Slowly they will cotton on there are votes in reversing AGW policies and once one party breaks ranks others will follow. UKIP kind of have in the UK, but it’ll take one of the traditional parties to follow suit before the dam breaks.

w.r.t that last paragraph, I downloaded the data and see that it doesn’t include individual stations. With exceptions for closely ranked years, the same statement should also hold true for points in time. If month1/year1 is markedly warmer than month2/year2 when they were collected in 2005, that should also be true in 2014. Barring rounding differences, if we happen to find that month2/year2 is markedly cooler in 2005 and warmer in 2014, for any pair of months/years, then I think that should be a huge red flag, as it indicates that the size of an “adjusted degree” is not being held constant, even within a particular dataset.
If an “adjusted degree” is bigger for one station than it is for another, even during a particular data sweep, than I don’t see that how it conveys any useful information. On the other hand, if the inter-time-period ranking of months stays the same for all pairs at all times (except for rounding differences), that would offer some confidence in the adjustments.
If I get around to it, I may run this test, but it might be a while… Don’t hold your breath waiting for me.

Bill_W says:
July 4, 2014 at 5:06 am
John Harmon,
I’m not so sure the Congressional record can’t be changed. After all, congress-folk are allowed to record a speech at a later date and insert it into the record when they were not even present the day the debate took place. I hope that the actual date they made the recording is prominently displayed but I have not checked
==========================================================
If you can stomach watching and listening, congressional types nearly always end their spiel with:
“I reserve the right to REVISE and extend my remarks”. A convenient out for the times they get called out being for or against an issue. Just another way of saying “I was for it before I was against it.”

Changing the time of temperature observations at a given weather station makes a huge difference to the reading. Surely that is obvious. Further, the work trend has been for people to stay at work all day, rather than return home at lunch time to eat which used to happen. So older manual readings tend to be taken closer to the hottest part of the day, and newer manual readings tend to be taken early in the morning before going to work. Most recently automated stations presumably give you a full temperature profile throughout the day.
So there’s no way that you can get a sensible trend from readings at the same place with different “Time of Observation”, which forces you to adjust it and guarantees that the adjusted temperature is going to change with time as the knowledge of how to do it improves. Thus, to cope with the change in time you would expect to increase the trend over time for the majority of stations, though a minority may reduce, and you should have the data (“Time of Observation” in the paper record) available to do this. Exactly what adjustment you should then make is going to be a matter of doing analyses and developing expertise which would then let you create rules, which you might hope would get better over time.
The other factor which needs adjustment is the well-known UHI (“Urban Heat Island”) effect. This time you need to identify changes in individual station readings not reflected in readings in the same area (which need to include urban and rural stations), and mostly this ought to result in reducing the more recent temperatures for urban stations. The process need not be done by explicitly identifying urban stations – just to make adjustments for any station which suddenly starts to get out of line with the other stations in the region ought to be able to cope with any UHI effects and similar changes in either direction.
There are research papers and code available comparing “urban” with “rural” trends for the USA which determine whether there are any UHI distortions left in the data set, and the GISS temperatures come out pretty well on this analysis taking any one of four definitions of what “urban” might actually mean!

From Phil.
“””””…..The bottom line is that there is no “temperature data,” prior to satellite measurements (which suffer from their own issues) and the USCRN network, which only has about 10 years worth of data, IIRC, BEST, GISS, HADCRUT, etc. etc. notwithstanding. All of the temperature indices are essentially models (which may reflect the modelers’ biases and preconceptions more than the historical weather.)……”””””
And the top line is that almost concurrent with the satellite data era, is the era of those oceanic buoys, that have been measuring simultaneously, both the near surface (-1 metre) oceanic water Temperature, and the near surface/lower troposphere (+3 metre) oceanic air Temperatures, on which Prof. John Christy, et al, reported about Jan 2001 that those Temps are not identical (why would they be). And further more, they aren’t correlated, so all that 150 years of ocean water temps obtained haphazardly at sea, as proxies for oceanic lower troposphere air Temperatures, to mingle with the other 29% of the earth surface, that is solid ground, instead of water; is just plain junk. Total rubbish.
So lacking correlation, the correct air temps, are not recoverable from the phony baloney ersatz water Temperatures.
So I’m in agreement. No global Temperature data, pre circa 1979/80; so actually anything approximating global Temperatures have been getting measured for about one earth climate “interval” time (30-35 years). Wait till we have maybe five under our belt.

Richard Courtney says that the GISS guys should get only one shot at adjusting the data – once they have had that – no further changes. This would be a pretty stupid approach. Here’s how it would work….
The GISS team have already adjusted readings for Time of Observation but now realise that the new electronic mini-max thermometers installed in stations on average give a maximum temperature 0.4C lower and a minimum temperature 0.3C higher than the previous manual readings. But Richard’s rule says that they have already had their shot at adjusting, so its not permissible to make the new change.
And more scrutiny of the data has brought to light that a few guys were cheating in the 1920’s – they went on holiday without arranging a standin and decided the easiest approach was to record the same temperature for a week. Sorry guys – you’ve had your one adjustment and the adjusted temperatures are now fixed in perpetuity.
See http://reason.com/archives/2014/07/03/did-federal-climate-scientists-fudge-tem for further cases – despite the title whoever wrote that piece had done more than take a superficial view saying “adjustments are always bad – particularly adjustments in the direction we don’t like”.

People really ought to look into R. R is great for examining data. For example, it took me all of ten minutes (including the time to familiarize myself with some unfamiliar commands) to generate 94 images showing the changes between each iteration of the data set. I’ve uploaded that collection of images here. It takes only five or so lines of code to repeat the process.
I’m not drawing any conclusions on this topic. I haven’t spent the time. I just would like to see people move away from things like Excel.

Dear Mr. Dnes,
I discovered in March 2012 that NASA-GISS changed retro-actively their temperature data offered in March 2010. I had downloaded the 2010-data and put into my archives. Between March 2012 and January 2013 I evaluated the discrepancies between 2010-data and 2012-data for 120 stations. The results are dealt with in a comprehensive paper. Meanwhile a German version has been successfully peer-reviewed and will be published still this year. I prepared also an English Version and send it, among others, also to Mr. Watts but got no answer. I would like to send you that paper – perhaps we can cooperate – if I had your e-mail address. Please send it. Many thanks in advance and best regards
Friedrich-Karl Ewert

Peter says:
July 5, 2014 at 1:44 am
I was actually referring to all the temp dataset holders – which have been shown elsewhere to be erroneous (can’t remember the thread, but I had to point out that BEST di use adjusted data despite Mosh and others protestations to the contrary)
Returning to GISS in particular, I have no idea if GISS do adjustments ‘on’ adjustments themselves (but we know others do!) but my primary point was to say that with ALL the datasets, and remember that many of which are cross correlated to some degree as they share station data, etc – I have never YET seen a raw dataset, followed by an adjusted dataset COMPLETE with description of each and every ‘wave’ of adjustments (and I don;t mean a generic type description – I mean a station by station algorithmic and then manual QC check followed by careful checking of details (e.g. does the station still exist even though there is missing data! see recent wuwt thread! LOL) and recording of said findings and finally explanation of the adjustment made and why. Simply put – without such detail – any data made available or used in any scientific capacity (such as say, ;proving’ CAGW – lol again!) is scientifically invalidated as the scientific method is clearly being ignored.
When the harryreadme txt file was released it clearly shows that no one knows the situation with crutemp or hadcrut (cant remember which one) and hence the data is unverified – or more specifically UNVERIFIABLE in any way shape or form! (this of course, bearing in mind that Jones stated the raw data was not available anymore!).
I have no problem with adjustments – or more specifically ‘corrections’ being made – but it you cannot produce a full traceability trail for the ‘data’ – and I mean FULL – forget it…..without such traceability, the data and the data pushers are putting themselves up for ridicule!

Kev-in-UK said “However, when said data is used and upheld as scientifically valid – that changes the game – it must be reproducible and proven as valid. Frankly, to this day, I don’t think we have reliable data – and certainly not without questions as to its history or validity! Ergo, in almost any other scientific endeavour, the ‘results’ or ‘conclusions’ based on such data would be thrown out or at best held in very low esteem (think wagonload of salt here!)”
Kev,
Sure we’d all like perfect historical data with proof that is is valid, but for the historical readings we have got only what we have got, so have to make the best of it – you can’t go back in time to do individual station equipment quality checks. All you can do is to quality check your current equipment and then post-process and adjust the historical observations to see what you can get out of them.
“Throwing out” is not an option – we have no better data, and “esteem” should be expressed in more statistical terms.
Specifically, with post processing and adjustment of station readings you should not only emerge with a best estimate as to what the temperature actually was at that time, but also with a statistical calculation of the expected error in the readings. That error estimate can be used to tell you how much confidence to place in any results you get when you use the figures, and what the likely bounds of the results should be.
So maybe a scientist does a sound piece of research using the adjusted data, including a detailed error calculation, and produces a well-written paper containing, maybe a trend with error bars. What happens then? Well it probably gets published in a reputable peer-reviewed journal which only makes articles available online behind a paywall. Scientists and academics have access because their institutions subscribe, but general members of the public (who have often paid for the research in their taxes) would have to pay $20-30 dollars per throw to get at such stuff, and who can afford it, particularly since, despite reading the abstract, you don’t necessarily know until after you have bought a paper whether it really was worth it. Such is life!

RichardSCourtney said ‘The asserted “total waste of time” provides unquantified error to every analysis which uses the data set. No scientist can find that acceptable.
So, it IS “important to know what errors were present in every single one of the past versions of data”. And that can only be determined in falsifiable manner by knowing “exactly how correction of these errors has changed every single station reading”‘
My reading of your statement is that you are saying that you wish to know about every single station reading in every single version of a temperature data set because somehow it affects the errors present in the most recent version of the data set. If that is not what you are saying then you had better correct me.
Each new version of GISTEMP or UKHCN stands alone. The metadata documents all errors anomalies and identifiable inaccuracies which have every been discovered with the raw or source data and the code release for each version handles all of these. They are documented and handled not as fixes to a single weather station and/or single reading, but as a fix to a situation which is going to occur for multiple weather stations and/or readings. Further there will be an accompanying set of error data with each release of “best estimate” temperature data.
So we have a current list of situations handled by the current code. It is difficult to see how the subset of situations handled by previous versions of the code has any bearing at all on the best estimate temperature data or the error estimates for a current data set. Typically an old version would handle only a subset of the anomaly situations, or an inferior algorithm for a particular situation.
In other words once BEST (or whatever process you are using to validate a data set) had done comparisons with GISTEMP (or some other dataset) at a particular version and validated the processing, then there is no more to be said about any previous versions and no point in making comparisons between older versions pre-validation and newer versions post-validation.
Even in there is no validation process the presence or absence of differences between the current and past versions of a data set (particularly ones from 20 years ago) does not change the validity or error estimates for the current version, though they might indeed change the error estimates for the older versions.

Richard said “Scientific standards require publication of each and every change to each and every datum because the data has been changed and somebody who wants to use it may need to know the change; n.b. each and every change to each and every datum.”
The computer run to produce a specific version of an adjusted dataset always starts with the raw data and not with a previous version of the dataset. Thus there is no scientific or statistical requirement to document changes by station from previous versions of the dataset.
The documentation required is the following :
1) Raw input data
2) Specification of the processing which will be done on the raw data.
This will typically use language describing common conditions. For instance, it might describe how to handle missing readings from stations which would normally report. This is a generic process – there is no requirement to list in the specification all the days missing for each station and new data may come to light at some point which fixes the problem in later runs.
3) The computer code which does the processing.
Someone wishing to validate the process can read this code to look for problems and may wish to amend it to make it easier to check how the anomalies relating to certain stations were handled if it does not already write out interim files after each adjustment step. The code should clearly handle all the situations documented in the specification correctly. Another method of checking the code would be to write an independent set of code from scratch (which has been done for GISS adjustment code) and then just compare the two sets of output which should then be substantially the same, though not necessarily absolutely identical. It is very unlikely that the same bug will be present in two independently written versions of code although they might handle the same condition in very slightly different ways.
4) The adjusted output data
All this stuff is available for GISS temperature data.
No-one in their right mind is going to write a spec which includes a detailed set of processing requirements for every individual station – it is a total and utter waste of time as the information can be extracted from a computer run if necessary. It is possible that some conditions are in the spec and apply to only a single station and are documented as such, but this is likely to be a very small number.
The data on precisely what changes are made to each station reading during the different processing steps within a specific (normally the current) version of code are available from the computer run, if they are needed.
Now this is a definitive and complete set of information to tell an interested party exactly how the data adjustments have been done in a particular version for each an every station if necessary.
You seem to be suggesting that such validation should include looking at all previous data set versions , but clearly from the above this is not necessary in the process of validation in order to meet the requirements of the users, the science and the statistics.