GISS is Unique (Now Includes May Data)

davidmhoffer says:
July 5, 2014 at 3:37 pm
No it doesn’t. Everything up to 0.49999…. is rounded down
Keep in mind we are talking about temperature readings to the nearest tenth of a degree, or so I thought.

Mike T says:
July 5, 2014 at 3:57 pm
One small thing: “revert back” is a tautology. Revert means to go back to a previous state.
==========================
It is with the deepest regret that I must point out that “go back to a previous point” is a tautology, since you can’t go forward or sideways to a previous state.

As a statistician and meteorologist, much of what is going on here makes me feel like I am watching a horror show. The errors occurring on all sides here are frightening – here are a few of them:
1) If the measuring instrument changes, no amount of “infilling” or “adjustments” can ever change the fact that the data are bad – you cannot treat new and old as the same data series. If they are used in calculating a statistic, then the proper degrees of freedom of any adjustment method require deflation. On the flip side, if the changing data are used in a model as a proxy of an unobserved “true” state (such as often used in econometrics), then all statistics must be calculated with respect to this model, but usually model specification errors dominate to the point of making the inferences useless. In other words, one cannot use fake data from which to calculate an inference and not acknowledge that the data have been tampered with in the corresponding inferences.
2) With respect to the first point, the other option would be to throw out data series that have a changing measurement or introduced bias, but this then means that survivorship bias will dominate. In other words, are surviving “good” series in rural areas, urban areas, country/regional specific, etc?
3) The bigger horror is that average temperature anomalies are not really even measuring what has to be measured, namely, the combined total heat content found in the atmosphere/ocean system. Since even small relative changes in latent heat content dominate small changes in average temperature, this is not even measuring the right thing, much less handling distributional issues (e.g., net boundary layer heat content as opposed to a surface station temperature that will be influenced by land-use change effects on boundary layer decoupling, etc).
4) Point 3 was the tip of the “iceberg”, as the heat capacity and storage of the oceans is so massive compared to the atmosphere that even accounting for water vapor and latent heat roles in the atmosphere would still succumb to the much more massive and still mostly unmeasured oceanic heat content. In a nutshell, we do not even have a valid historical series in which we can measure such changes on the scale being attempted.
We do “know” that the earth has been both much warmer on average and much colder on average than present, but the ugly truth is that we do not yet have any usable data to give us a good estimate of total heat content changes in the modern era. Satellite measurements of total upwelling/down-welling radiation fluxes are a definite step in the right direction, as are deployment of much more extensive ocean buoy/temp sensors, but the rest of this looks like bad science and statistics to me. It’s nice to be able to ask if things are getting warmer or colder, but the real question is what is happening to the heat content and distribution. And that is not going to appear in any of these single number series, even ignoring their issues.

What if they do it the this way ?
I may be 0.001 to 0.444 is rounded down and 0.445 to 0.999 is rounded up.
That will give a bias too …
Worth checking ….

With my sincere apologies to our American friends.
Maybe this constant exaggeration of data as seen in GISS is an inherent characteristic of the always optimistic American psyche.
I have flown gliders, those big mostly German and Polish built FRP ones for just over 50 years now .
A number of American outfits over those 50 years have had a go at building gliders with the performance curves that are claimed to match or exceed the German and Polish built gliders and their performance curves, that is the sink rates and glide angles at different speeds, the “polar curves” as they are called which on paper the American designed and built gliders can match the best of the German and Polish aircraft .
[ Those best German gliders depending on class models now have best glide ratios of between 50 and 60 to one; ie 50 to 60 kilometres gliding distance in calm air for each kilometre of height; ie One kilometre = 3281.4 feet
A late model jet airliner has a glide ration close to and around 20 to one ]
But in the air it is a different matter altogether as the American designed and built gliders, despite their claimed performances derived from their “polar curves”, have never matched the German and Polish machines in performance.
And the reason is quite simple.
The Americans as do the European manufacturers of gliders design the glider using all the aerodynamic inputs required for best performance as well as for good handling in the air, a vital characteristic whether you spend six or eight hours driving hard over long distances of hundreds of kilometres or just lazing around for an hour or so casually enjoying flying the local airfield thermals .
When the glider’s claimed glide angles at different speeds, the performance characteristics seen in the “polar curve” of the glider is published, the Americans in the past have always use the modeled, the maximum theoretical performance curves in their advertising literature and in their technical specifications for the glider type.
As we all know there can be a lot of quite large differences between modeled and calculated performance and the real world performance in just about any field we would consider, especially climate and now temperature data manipulation..
The Germans use the measured in air performance of the very carefully built and finished prototype or preproduction models of a type for their performance curves.
The actual performance of their commercially built gliders are usually not quite up to their claimed performance simply because the precision built molds, built to less than a few tenths of a millimetre in accuracy in the wing aerodynamic profile which the wing is then molded in, deteriorate over a few tens of the numbers of wings produced and have to be re-profiled to regain the accurate aerodynamic wing profile.
So a glider type off the German and European production lines will rarely match the performance of the preproduction gliders but will generally come very close.
And those flown performance curves from the preproduction gliders are what are used in the advertising and technical information of the glider type to sell those gliders.
The Poles apparently grab a glider off the production line and measure the performance curves of the real world glider with all it’s small imperfections from a series built production, the glider the customers actually buy so you generally get the advertised performance you pay for from the Poles.
And so with the American GISS and the European HADCRU.
The Americans as usual figure that it is advertised world beating performance that sells and that has certainly been the case with the exaggerations of GISS in it’s temperature data .
And maybe that tendency to exaggerate somewhat in the American psyche is also seen in the gross corruption of temperature data by the American based NCDC before it ever reaches GISS and HADCRU for further severe massaging before being released for public consumption.
Mind you, if NCDC and either or both GISS and HADCRU were calculating and advertising the performance specifications for a glider, given their known ability to “adjust” global temperature performance and advertise those grossly hyped figures as the real deal I would now just assume that they were into advertising and selling gliders that had real world performances somewhat akin to that of a “concrete” glider as an alternative to selling the odd bridge or two.

Reminds us again which data set ‘Dr Doom’ hand his sticky little fingers all over , that is when they where not busy shutting windows and turning off ?

Steve McIntyre:

If I try to read GISS data in R (for example)
http://data.giss.nasa.gov/gistemp/tabledata_v3/GLB.Ts+dSST.txt
I am unable – it says “cannot open: HTTP status was ’403 Forbidden’”
I’ve examined a great deal of data and the apparent blocking of R access by GISS is very unusual – unprecedented even. Is this experienced by anyone else?

Yes. You need to use cURL instead. A bit of discussion here.
Here is a simple example for command line applications:
curl -o GLB.Ts+dSST.txt http://data.giss.nasa.gov/gistemp/tabledata_v3/GLB.Ts+dSST.txt
See for example rcurl for packages that implement this in R.

Regarding rounding used in ASOS observations, for a number of years, the instructions
have been to ALWAYS round UP a 0.5 degree reading. This means that if the reading
is below zero, you round the reading to the nearest higher whole degree, rather than
the nearest higher number absolute with the sign appended.
In other words, if the temperature reads -1.5 degrees, it would be rounded to -1
degree, rather than -2 degrees.
This is from the ASOS manual regarding measurements from before 2010.

Steve McIntyre says:
July 5, 2014 at 9:53 am
When I looked at GISS in 2007, they used a weird “two-legged” adjustment to individual stations that is not used by anyone else. I haven’t checked their recent methods to determine whether it’s still being used, but it probably is.

Yes, they still use the “two-legged” adjustment, where each of the legs can either go down (to remove “urban warming”) or up (to remove “urban cooling”). I think the rationale was as follows:
1. Urbanization bias is not necessarily “linear”, e.g., an area could undergo exponential growth during some period, while during other periods the rate of development might “slow down”. Therefore, Hansen et al. decided to allow a bit more flexibility in their adjustments than a single “linear” adjustment per station. Hence, the “bi-linear” adjustments…
2. They heard that, under certain conditions, urban development could introduce a slight “cooling” bias. So, they decided to let their automated adjustment program calculate adjustments that had either a negative or a positive slope.
Unfortunately, because of a number of serious flaws in their automated adjustment algorithm and the poor quality of the GHCN dataset, these adjustments are frequently unrealistic and/or nonsensical.
We provide a detailed discussion & analysis of these adjustments in our “Urbanization bias II. An assessment of the NASA GISS urbanization adjustment method” paper, which we have submitted for open peer review at our OPRJ forum: http://oprj.net/articles/climate-science/31
For our paper, we carried out a series of five comprehensive surveys of the adjustments over the period August 2010 until November 2011, when GISS switched to GHCN V3 and stopped publishing their intermediate calculations on their public ftp site. We also used some of the GISS files you have in your ClimateAudit data folder to calculate the adjustments for February 2008.
For each survey, we divided the station adjustments into 4 types, based on the slopes of the two “legs”:
Type 1 – Both slopes negative, i.e., remove “urban warming”
Type 2 – Both slopes positive, i.e., remove “urban cooling”
Type 3 – First slope negative; second slope positive
Type 4 – First slope positive; second slope negative
Types 3 & 4 comprise the majority of their adjustments. We call these “tag-team” adjustments. I think this is similar to the adjustments you referred to as “bi-polar”, although as far as I recall you found less “bi-polar” adjustments than we found “tag team” adjustments.
Here are the results of our surveys (see Table 1 of our paper):
Survey date | Type 1 | Type 2 | Type 3 | Type 4 |Total | Rural | Skipped |
Feb 2008 | 447 | 300 | 1342 | 1440 | 3529 | 2488 |250 |
Aug 2010 | 457 | 265 | 1191 | 1098 | 3011 | 3124 |164 |
Jan 2011 | 459 | 260 | 1184 | 1108 | 3011 | 3127 |176 |
Jul 2011 | 451 | 266 | 1182 | 1112 | 3011 | 3131 |180 |
Oct 2011 | 461 | 261 | 1181 | 1113 | 3016 | 3132 |177 |
Nov 2011 | 455 | 265 | 1177 | 1117 | 3014 | 3136 |177 |
After November 2011, NASA GISS stopped publishing the intermediate calculations (presumably as part of NASA’s “Open Data” project??? 😉 ). So, we were unable to check exactly how the breakdown of the four types changed when they switched to using GHCN V3.
But, as Nick Stokes points out, when they switched to using V3, they started using the homogenized version of GHCN.
As we discuss in our papers, the Menne & Williams, 2009 homogenization algorithm used for GHCN causes “urban blending”. That is, the trends of the most heavily urbanized stations are slightly reduced to match their less urbanized neighbours… but, the rural stations have their trends increased to better match their urban neighbours.
This means that, in the homogenized GHCN stations, the urbanization bias is blended (or “homogenized”) amongst all the stations – “rural” and “urban”.
The GISS adjustments explicitly assume that the “rural” stations have no urbanization bias. So, whatever the problems with the pre-2012 GISS adjustments, their post-2011 adjustments are likely nonsensical.

Let me fix that headline for you.
GISS is unique( May include Data)

You list how far back in the data sets the regression line is flat. Actually, you can push it back a bit further than that. If the slope of the regression line is less than the Standard Error, then the slope is not statistically significant and you can consider that line to be flat.

GISS reached some sort of a tipping point with its record May anomaly which pushed its statistically significant warming date forward by over two years from last month.
Hadcrut4 also set a record for May, however since it came so late, the statistically significant date had not been updated when I sent this off. It has now been updated and the time for statistically significance for Hadcrut4 only went up by one month to November 1996 where the CI is now -0.023 to 1.167. So Hadcrut4 still shows no statistically significant warming for over 17 years.