Huge Divergence Between Latest UAH & HadCRUT4 Temperature Datasets (Now Includes April Data)

Hugh,
I am not sure I understand your question – are you asking me to explain WHY they are different. I do not know, if I did I would try to write a paper about it.
Which periods from the UAH and RSS datasets do you think contain a “pause”. That is checkable.

Werner Brozek.
I have never seen a scientific study saying the satellite and surface temperatures should track each-other. If there is such a study someone can post a citation.
The UAH and RSS estimates do not always track each other, either. I do know the methods they use to allow for unsampled regions and data contamination differ.
I do believe if base lines are aligned correctly and a longer trend is computed then satellite and surface temperatures track each other better.

If there is such a study someone can post a citation. The UAH and RSS estimates do not always track each other, either.

See Paul Homewood’s comment below:
“For those who argue that surface and satellites don’t necessarily follow each other, remember what the Met Office had to say in 2013, when discussing their HADCRUT sets:
Changes in temperature observed in surface data records are corroborated by records of temperatures in the troposphere recorded by satellites””
http://wattsupwiththat.com/2015/06/09/huge-divergence-between-latest-uah-and-hadcrut4-revisions-now-includes-april-data/#comment-1958564
As for the tracking, that was true before UAH6.0. Now they more or less agree.

Werner Brozek.
The only reference I can find says this:
“Changes in temperature observed in surface data records are corroborated by measurements of temperatures below the surface of the ocean, by records of temperatures in the troposphere recorded by satellites and weather balloons, in independent records of air temperatures measured over the oceans and by records of sea-surface temperatures measured by satellites.”
In context he means the changes are “corroborated”. He doesn’t say they follow each other. So I am calling a straw man argument on this one.

That cannot possibly be true. It makes no sense.
Are you sure of that?

No. However, I’ve read a number of articles that are at least reasonably convincing that suggest that it is, some of them on WUWT or linked sites. One such article actually broke down the GISS UHI correction by station and showed that roughly a third of the UHI corrections decreased station temperatures, a third increased them, and a bunch in the middle showed no change. Sadly, I failed to link it and don’t remember who did it (but I have little reason to doubt what they stated). Indeed, articles openly published on the UHI effect by warmist sites directly state that it is the lack of any substantive change to the global anomaly upon applying the GISS correction that proves that UHI is a non-issue.
Other sites (again, referencing articles on WUWT in several cases by people who I believe to be both generally honest and reasonably competent) disagree:
http://www.greenworldtrust.org.uk/Science/Scientific/UHI.htm
or more recently:
http://wattsupwiththat.com/2014/03/14/record-daily-temperatures-and-uhi-in-the-usa/
Personally, I find it very suspicious that a correction that almost certainly should show some cooling effect ends up neutral when applied by GISS, and AFAIK HadCRUT just ignores UHI, and both warmist and skeptical sites seem to agree that the GISS correction is small, essentially neutral. NCDC disagrees, but claims that their (sometimes large) homogenization correction accounts for it and that it doesn’t explain CONUS warming over the last century. The WUWT article linked above (which admittedly focuses on the southeast) disagrees. So does the article on rural vs urban county warming in California. North Carolina (where I live) has been temperature neutral for over 100 years, with no significant trend.
That’s the odd thing about this “Global” warming. It’s not terribly global, even over a century of data, when you look at unadjusted data or data maintained by people who don’t use a model to turn it into a global product.
So no, I’m not certain, and yes, it is a shame because I’d love to be able to trust a global temperature record so that one could compare model predictions or forecasts to an objective standard. However, direct quotes of e.g. Phil Jones in climategate letters and Hansen’s obvious personal bias make it pretty clear that HadCRUT and GISS — their personal babies — have been built and maintained not by objective scientists but by strongly politically partisan scientists who have proven directly obstructive (in the case of Jones) when people interested in checking their data and methods attempted to do so. Yet simple red flags such as the obvious bias in corrections to the major temperature products over many version changes go ignored. If we are to believe them, all of the measurements made before 1950 or so caused daily temperatures to be overestimated, all of the measurements made after 1950 including contemporary ones (go figure!) cause them to be underestimated. Obviously, 1950 was a very good year for buying thermometers. Wish I had me one of them vintage 1950 mercury babies… then I’d know the temperature outside…
This is, to put it simply, extremely implausible. In particular it is difficult to understand how temperatures over the last three decades could possibly require additional upwards correction — from the 50’s or 60’s on (post world war II) weather reporting and weather stations have been built using proper equipment and paid, trained personnel and in much of that, electronic reporting that could not reasonably have a high or low systematic bias such as time of day.
The latest paper that is being skewered is simply the latest, clumsiest version of this. A major world meeting (G7) is going on to try to get agreement among the major world powers on global warming and the need to basically halt modern civilization in its tracks to prevent it at any or all human cost and at exactly the right moment a new paper appears that claims that the fact that there has been no statistically significant warming for at least 15 years (acknowledged even in AR5, which is not known for its objectivity on the issue) if not 18 to 20 years — basically the entire shelf life of the Assessment Reports themselves — is incorrect, that there really has been warming, and that the source of this warming is the underreporting of intake water temperatures on ships!
I’m still working on that one. Intake water drawn in by a ship has not been warmed by the engine in any ship that is under way. The water behind the ship might conceivably be warmed by the engine — if you sampled it at the precise spot where the cooling water was returned to the ocean and within seconds of doing so. After a few seconds, we have a simple mixing problem. Assuming cooling water with an exit temperature of 373 C (an overestimate) and ocean water at 273 C (an obvious underestimate) to get a 0.1 C temperature change in the latter we have to mix each liter of hot exhaust water with 1000 liters of ocean water. 1000 liters happens to be 1 cubic meter of ocean water. No matter how I mentally juggle the mixing of water in the wake of a major boat underway or during the laminar flow of water up from a forward intake to even a badly located temperature sensor (one e.g. sitting right on top of the engine itself) I don’t see much chance of a major warming relative to the true intake temperature (which is true SST). I absolutely don’t see how engine exhaust heat returned as water that is immediately mixed by the wake behind the boat is going to affect intake water temperatures in front of the wake, or for that matter behind the boat after the boat has travelled as little as a single meter. I’d want to see direct evidence, and I do mean direct, experiments performed on an ensemble of randomly selected actual ships underway on randomly selected parts of the ocean with reliable instruments in the sea in front of the intake valves compared to the intake temperature, before I’d even imagine such a correction to be globally accurate.
Worse, this paper argues that ARGO buoys are not measuring SST correctly, but corrected ship intake measurements do! Really? So we spent a gazillion dollars on ARGO to get systematically biased results? My recollection of ARGO is that it was motivated largely because ship-based data is highly unreliable and becomes more unreliable the further you go into the past until it is based on sampling buckets of water drawn up by hand using ordinary thermometers held up in the wind to take a reading by indifferent skippers along 19th century trade routes (giving us no samples at all in most of the ocean).
So what are the odds that this paper is a) apolitical; b) objective; c) timed by pure chance? I’d have to say “zero”. And we haven’t even gotten to the next actual climate summit. Expect yet another “adjustment” of the present and its nasty old inaccurate thermometers just in time to be able to claim that the hiatus is over.
The only problem is, these adjustments are going to serve no useful purpose but to increase the already glaring gap between surface temperatures and RSS/UAH (and the latest creates a new, bigger gap between it and even far-from-objective HadCRUT and GISSTemp, to add to their gap with RSS). At some point the scientists involved will have no choice but to address this gap, because there are simply too many ways to directly measure and challenge the individual temperatures at grid points to build the averages.
Also, the public dislikes being lied to or “fooled”. If the latest paper is torn to shreds in short order even by those that believe in global warming — as I think is not unlikely, as they are going head to head with all of the people with an interest in ARGO, so there is actual money and academic reputation on the line across a broad patch of scientists here who have used ARGO data and Hadley data at face value, all now being challenged — it is not at all unlikely that it will backfire.
I can hardly wait for September. Will we see HadCRUT4 and GISS magically sprout a 0.2 C jump in August (with RSS remaining stubbornly nearly unchanged)? Will ENSO save the day for warmists everywhere and convince China and India to remain energy poor for the rest of eternity, convince the US congress to commit economic hari kiri? Will past-peak solar activity start to have an impact through as-yet unverified means? Stay tuned, folks, and get out that popcorn!
rgb

Converting one to the other means making up numbers that were never a part of any observation anywhere by anybody. Sorta ike scotch mist !
g

I am curious what their analysis would show with the new UAH6.0 that is very close to RSS now. Would the pause still be there?

There is always a problem with estimating missing data.
Among other thing it dramatically increases your error bars.
The other is when the estimation is inherently invalid, such as taking land stations and stretching them across on ice covered ocean.

But I find the new found trust in the various global average temperature increase encouraging

It seems to me that most people here trust the satellites that show no temperature increase.

That said, it should be noted that no dataset agrees completely with any others.

True. For one thing, UAH goes to 85 S but if I am not mistaken, RSS only goes to 70 S. However there is good agreement despite this. Presumably 70 S to 85 S is not much different than the rest of the earth.

I am deeply suspicious of satellite data, as it doesn’t directly record temperature.

In the same way, does a mercury thermometer really record temperature or does it record the expansion of mercury in a glass tube that has certain markings on it? Different physical attributes can easily be quantified to represent temperature differences. I do not share this suspicion of yours although the mercury thermometer seems more straightforward to me. Nothing here is perfect, but we do the best we can with what we have.

TGOBJC, as I recall, the satellite readings of the lower troposphere (LT) are verified by balloon measurements of the LT, at least within measurement uncertainty.

Kriging per se isn’t necessarily questionable — it is just error prone. If you have to krige, it cannot “produce” data, it’s just a way of smoothing over missing regions not unlike a cubic spline or other smooth interpolation. The questionable part is strictly the way it affects precision. Kriged data cannot reduce error estimates as if it is an independent and identically distributed sample from some distribution. It by its nature smooths out any peaks or valleys in the kriged region, and is perfectly happy to make systematic errors.
For example, if one kriged missing data in Antarctica using sea surface temperatures from a ring around the continent, one would make a really huge error, would one not? Because the SSTs are all going to be in the vicinity of 0 C, causing a krige of 0 C across Antarctica in winter, which is a wee bit of a joke. Nor can one place a single weather station at the south pole and krige between it and the surrounding ring — this would produce a smooth temperature field out to the warm boundary and is again a joke — the temperature variation between the sea and the land is enormous and rapid. Similar considerations involve the Arctic — kriging either way across different surface and (especially!) across the arctic circulation and local phenomena that strongly affect local temperatures is going to smooth away reality and replace it with a number, but one that has to have a huge error bar added to it that contributes just as much error to the final error estimate as having no data there at all because you can’t squeeze statistical blood from a stone! Missing information is missing.
rgb

“””””…..Missing information is missing.
rgb…..”””””
Well Robert, I Kringe every time I think of Kriging.
I would venture (lacking a handy rigorous proof) that Kriging can only be tolerated, if NONE of the “Kriged” faux data points causes the Kriged signal to suddenly acquire faux signal component frequencies that are above the Nyquist bandwidth limit corresponding to the pre-Kriged sampled data.
As I have (in effect) asserted elsewhere “Kriged pictures look prettier; but they are still false representations of reality.
g

It would be … interesting to see annual ‘anomalies’ if years started in different months.

Yes, shifting to different 12 month intervals can give slightly higher values than the January to December averages. If you want to clearly see this, plot what you want on WFT, then take a mean of 12.

Yet globally all seasons are occurring all the time. However the difference between the global mean of each season is informative. In January the earth experiences plus 90 watts per sq meter insolation yet the atmosphere cools on average.

Yes, when you take a step back, warming rates of .01 deg C a year in the AGW era are insanely small. Yet, somehow, we are being told it will do all of this…
http://whatreallyhappened.com/WRHARTICLES/globalwarming2.html

But RGB, you didn’t really get to the point of my original post. Does a historical record of the wx model initializations make a valid climate database?
Data for the entire earth is collected, error-check, and smoothed into a 2m grid several times a day. This is a consistent, objective process and since it is not in the climate debate, it is “untampered”.
Seems to me, and apparently Dr. Maue, that the 2m GCM initializations are a valid historical record perhaps on par in accurate representation with satellites or land based measurements.

But RGB, you didn’t really get to the point of my original post. Does a historical record of the wx model initializations make a valid climate database?
Data for the entire earth is collected, error-check, and smoothed into a 2m grid several times a day. This is a consistent, objective process and since it is not in the climate debate, it is “untampered”.
Seems to me, and apparently Dr. Maue, that the 2m GCM initializations are a valid historical record perhaps on par in accurate representation with satellites or land based measurements.

No, I didn’t get the point, sorry. But that is a very interesting question indeed. Presumably one could almost instantly transform them into a global average temperature, further one that is “validated” to some extent by the success of the following weather predictions based on the entire smoothed field. Note well (also) that this would give one a consistent direct estimate of the actual global temperature, not the anomaly! Since I personally am deeply cynical about the whole “anomaly is more accurate than our knowledge of the average temperature” business, I’d be extremely interested in the result. It would also be very easy to subtract and generate an “anomaly” out of the timeseries of actual averages for comparison purposes.
One thing that people likely forget is that climate models produce actual global temperatures, not temperature anomalies! They have little choice! The radiative, conductive, convective, and latent heat transfers are all physics expressed not only in degrees, but in degrees absolute or kelvin. One has no choice whatsoever in their “zero”. This is why the fact that the model timesteps aren’t stable with respect to energy and have to periodically be renormalized (de facto renormalizing the temperature) is so unnerving. How is this even possible without a renormalization scheme that must be biased as it presumes a prior knowledge of the actual energy imbalance that one is trying to compute so it can be preserved while and indistinguishable numerical error is removed? In another thread (yesterday or the day before) I discussed this extensively in the context of solving simple orbital trajectory problem with 1-step Euler and then trying to remove the accumulating energy/angular momentum drift with some step-renormalization scheme. If one didn’t know that energy and angular momentum were strictly conserved by real orbits, how could one implement such a scheme?
So the other interesting thing is that these averages could be directly compared not to the “anomalies” of the GCM temperatures with their completely free parameter, the actual average temperature, that is already not in very good agreement across models(!) but to the actual average temperature as determined by the weather model initialization, computed with a reasonably consistent algorithm and, one might hope, with no dog in the race other than a desire to get next week’s weather as correct as possible, which will only work if the temperature initialization isn’t too far off from reality and maybe lacks a systematic bias. One could hope. At least a month to month, year to year time dependent systematic bias, since it may well be empirically that weather models perform better if the initialization temperature is slightly biased if they tend to over or under estimate cooling/heating.
Surely somebody has looked at this?
rgb

Absolutely awesome site, I bookmarked it. I’d even sign up for it, but $200/year is an insane price — weather underground is $5/year. I did look at the temperature product though, and it is a) very fine grained; b) as you say, quite different in structure from all of the temperature series. In particular, it totally removes the ENSO bumps most of the other records have. This all by itself is puzzling, but IMO quite possibly reasonable. The other really, really interesting thing about the record is that it is remarkably flat across the early 80s where HadCRUT and GISS show a huge warming. Indeed, the graph is very nearly trendless. 0.1 C/decade at most, with the odd bump well AFTER the 1998 ENSO that appears to end AT the 2010 ENSO.
It would be enormously interesting to compare this to RSS, but even there RSS has the 1998/1999 bump and the 2010 bump, because those were largely troposphere/atmospheric warming phenomena. This temperature graph appears to really track something else entirely. No wonder Joe Bastardi is skeptical (he seems to be affiliated with this site).
rgb

Thank you! If someone were so inclined, they could probably write a whole post on this topic. GISS and Hadcrut would love you if you could totally explain the discrepancy by concluding that both satellite and other data sets could still be correct.

Thank you! There is a lot here about changing humidity in the stratosphere, but I did not see anything to explain why satellite data for the lower troposphere should diverge from GISS.