September 2013 Global Surface (Land+Ocean) Temperature Anomaly Update

As ever, pertinent and informative – thanks for your tireless effort Bob.
‘Hiatus denial’ is becoming increasingly untenable.

Two remarks.
* Why is GISS so out of touch with the other two? HADCRUT4 is 0.01C, GISS is 0.13C in one month? I was wondering why and how the Wunderground blog was trumpeting September as the 4th hottest ever — obviously they were using GISS, not HADCRUT4.
This makes little sense, given the [significant] overlap in the underlying data sources for the two, and just as neither one can [increase] without bound to conform to the GCMs via constant “adjustments” because their divergence from UAH/RSS LTT would give the game away even more than it has already been lost, GISS can’t produce warming record headlines relative to HADCRUT4 month by month without causing far too obvious a divergence between the two.
BTW, this is an even better explanation for the wider variance of GISS than the one you offer. If the GISS GASTA varies by tenths a a degree month to month, it is comparatively easy to pick months that “set records” (or come close) even though the trend is neutral. After all, they only get a headline in the media when they set a high temp record, not when it falls 0.1+C the next month to less than the 15 year trend. It is left as an exercise for the audience to compute what fraction of months they can discover as “records” of one sort or another simply by computing their number in such a way that its monthly variance is 0.1-0.2C.
* Sigh… Bob, I know that there is an overwhelming temptation to take three supposedly similar numbers and average them, but it simply is not justified in this or almost any related case in climate science. NCDC, GISS, and HADCRUT4 are three different computations of supposedly the same number, GASTA. They are based on heavily overlapping data. They use different computational methods to e.g. average or smooth the data over spatiotemporal gaps, and they “adjust” the data differently for poorly known or even unmeasurable things such as UHI. They therefore are in no possible sense “independent and identically samples drawn from a common distribution” (iid). Consequently their mean is a meaningless quantity, as is their mutual variance. The Central Limit Theorem does not apply, and neither the mean nor the variance can be taken as reflective of an underlying “true mean” GASTA.
Indeed, all that their difference indicates is that at least two, more likely all three are in error by some unknown amount. This problem is even more pernicious given their substantial data overlap. Given (say) 60 to 80% common data, the averages should be much closer than they are and have very similar noise/variance around an underlying mean. As you do note above, the differences in variance are startling and are actually profound evidence that something is seriously amiss.
I know how tempting it is to assert (or rather, hope) that an average might cancel some of the errors made by each of the three methods, but as long as they share data they are in no sense independent, as long as the are based on differing methodology the are in no sense identically distributed, and there is no theorem or sound argument I can imagine that errors in methodology are unbiased, zero sum white noise drawn from some trendless distribution. Indeed, it is almost certainly not the case that this is so — the REASON that they are able to continually “adjust” the data is that they can assert that they’ve discovered a methodology error with trend.
Again, it is left as an exercise for the audience to compute the probability of (say) 6 successive adjustments for supposedly trended method error that all happen to have the same sign and the same effect, to comparatively cool the past and warm the present, but that’s another story. At this point LTT has put a stop to that. GISS appears to be reduced to desperation — getting headlines at the expense of noise while waiting for a desirable trend in LTT that permits them to observe real trended warming once again. HADCRUT4, after the last round of adjustments, actually appears to be behaving comparatively consistently with NCDC and LTT (finally). Perhaps the era of adjustment driven warming is finally over.
I’m not convinced of this, though. The more I look at correlations from quantities that ought to be correlated with GAST (note well, absolute and not the anomaly) the similarity between the first half of the 20th century and the second is profound. Given that GAST is a completely unknown quantity, I’m wondering how much of the “anomaly” is an artifact. The only thing that prevents me from thinking that it is all is sea level as measured by tide gauges, but recent evidence that a significant fraction of SLR may be caused by subsidence that we are only just barely beginning to measure with universal GPS access is making me wonder even about that. SL SHOULD be the best thermometer we have (at least not susceptible to a UHI that could make the land record increase vanish overnight), and even here we have a pitifully short, pitifully spatiotemporally undersampled reasonably reliable record, especially at depth.
rgb

Just a quick aside…
It was more of a government ‘slowdown’ rather that ‘shutdown’. Even they stated that essential services were to be maintained. Of course if the others were nonessential why do we have them in the first place.

the reason is simple. The past was colder than previously thought and the present is warmer.
its not because there is a conspiracy ( we did land on the moon) its because the past was in fact cooler.
Dear Steven,
I didn’t say anything about a conspiracy. I made a very simple statement about statistics. Let us assume that nobody in the remote past or present was deliberately trying to make their numbers either “too warm” or “too cold”. Errors from all sources — miscalibration of thermometers, reading temperatures inconsistently, poor placement of thermometers, and so on would in general be assumed to be as likely to be too warm as too cold. This is as true in the modern era as it is in the remote past. If one is correcting the record in the past, then, it should be just as likely that the correction required be negative as it is positive. If one is correcting the record in the present, it should be just as likely as that the correction be negative as it is positive. On average, one would a priori assume that while any given error one discovers might alter the overall trend up, the corrections for many errors should be as likely to alter it up as to alter it down and (within the usual square root associated with a random walk) should be zero mean, trendless.
We can thus assert the null hypothesis: All the corrections made to computed estimates of GASTA are honest and unbiased. Our prior assumption (in a strictly Bayesian sense) is that corrections are a coin flip, equally likely to increase trend as decrease trend. We could go further and talk about an actual distribution of corrections within some probable range, but let’s keep it simple with coin flips. The data is then we flip a coin six times in a row and get heads every time — every correction creates more relative warming, some of them rather significant relative warming. Does the data support the null hypothesis?
Of course not. In fact, the p-value for getting 6 heads in a row is only 1/128, much less than 1%. The probability of getting 6 corrections that all increase the warming trend in a row is (given the prior of unbiased corrections equally likely to increase or decrease trend) sufficient to reject the null hypothesis and state that it is rather likely to be the case that the prior probability of the coin flip is not, in fact, 0.5, but instead that the probability of getting heads (or a systematic increase in trend) is indeed significantly more than 0.5.
Nicholas Nassim Taleb describes this circumstance quite clearly in his lovely book, The Black Swan where he has a scientists (like yourself) and a cab driver named Joe analyze the probability of flipping just such a coin and getting just such an improbable string of heads. The scientist is asked to compute the probability of the coin coming up heads on the next flip, and of course answers 0.5, because it is a two sided coin and the coin flips are independent trials, so it is just bad luck that it came up heads so many times in a row. Joe the cab driver replies quite differently: “It’s a sure thing you’ll get heads. It’s a mugs game — the coin is fixed.”
A Bayesian agrees with Joe. A Bayesian isn’t certain that the coin is fixed, not from only six flips, but is definitely lifting an eyebrow and thinking “Oh, really?” Then we could go down the line on many more flips in climate science where the bread lands butter side down, where “adjustments” or “errors” somehow always seem to increase the warming trend but never decrease it.
Finally, one really has to ask how you know this: “the past was in fact cooler”. Are you somehow gifted with second sight so that you just “know” precisely what GAST was (within the tenths of a degree Celsius necessary to support all of the vast range of conclusions that are being made about global warming and its causes) so that you can state that it was in fact cooler? Can you be very helpful in that case and tell me how much more cooler it will end up after GISS and HADCRUT4 etc make their next set of “corrections”? Would you explain how your knowledge apparently exceeds the stated/acknowledged probable errors (which are generally presented, BTW as being symmetric and hence suggest that even the creators of HADCRUT (for example) believe their uncertainty is unbiased and symmetric — except when it comes to making a correction that is supposedly a part of that error)?
Personally, I have to say that the notion of “correcting” a data set long after the fact is a process so fraught with the opportunity for error and bias that most angels quite rightly fear to tread there. We wouldn’t tolerate it for a minute in biological research or health care, because there we have ample direct evidence that people simply cannot avoid making all “corrections” to their data improve their confidence in some desired or profitable conclusion. Obviously you have tremendous faith that this does not happen in climate science, even though it obvious is just as likely to actually happen there as it is to happen anywhere else somebody isn’t forced to double-blind their data and not adjust it once they begin analysis. You persist in this belief even though the observed series of corrections is highly improbable given presumably symmetric errors as reflected in the symmetric error bars assigned to the curves. You, my friend, are playing a mug’s game, at least according to Joe the cab driver.
I’m just sayin’.
rgb

Steven Mosher says:
October 31, 2013 at 9:39 am
the reason is simple. The past was colder than previously thought and the present is warmer.
its not because there is a conspiracy ( we did land on the moon) its because the past was in fact cooler.
If you compute the temperature field correctly you will see this. Any and all improvements to GISS, CRU and NCDC will result in a colder past and warmer current period. Not because they are putting their thumbs on the scale, but rather because the past was in fact colder and the present is in fact warmer.
Wow this is low…
And because we landed on the moon we will see the relative value of 2010 getting warmer then 2002 in the future and because the past was colder we will see 2002 getting colder. Right Steven.

the reason is simple. The past was colder than previously thought and the present is warmer.
I’m glad that you know this, for a fact. I’m very interested to know how you know this, for a fact.
Additionally, even assuming that the past was cooler and the present is warmer, why wasn’t one adjustment enough? Why are there continuing adjustments every couple of years, with each of them making the past ever cooler and the present even warmer?

I could read RBG brilliant posts all day to be honest. Everyone interested in global warming climate change 7 maths and physics should read about “Climate as a random walk” as a model fit its much much better than anything else. This really simple WM Briggs post is brilliant
If you like that, you’ll like the work of Koutsoyiannis even better. It isn’t actually best fit by a random walk, but by a Hurst-Kolmogorov process of punctuated equilibrium. It isn’t clear that the steps are trendless and unbiased, although they could be — this is enormously difficult to determine from comparatively short stretches of data. Bob Tisdale’s SST data makes this overwhelmingly obvious, and he identifies ENSO events as a trigger for the jumps, but that still doesn’t yet predict the direction of the jumps or their magnitude (that is, the long term underlying trend in the local equilibria the jumps move between).
The mathematical model for this is probably a space of local attractors with complex poincare cycles moving the climate around them, and with events or just plain time evolution causing comparatively sudden “jumps” between attractors. The motion and structure of the underlying attractor space, however, is largely unknown. I don’t even think we know the dimensionality of the space. What we see in GASTA is at best a projection onto a single axis of a complex motion around and between locally stable points.
rgb