New Compendium Paper on Surface Temperature Records

NOTE: An update to the compendium has been posted. Now has bookmarks. Please download again.

I have a new paper out with Joe D’Aleo.

First I want to say that without E.M. Smith, aka “Chiefio” and his astounding work with GISS process analysis, this paper would be far less interesting and insightful. We owe him a huge debt of gratitude. I ask WUWT readers to visit his blog “Musings from the Chiefio” and click the widget in the right sidebar that says “buy me a beer”. Trust me when I say he can really use a few hits in the tip jar more than he needs beer.

surface temp cover image

The report is over 100 pages, so if you are on a slow connection, it may take awhile.

For the Full Report in PDF Form, please click here or the image above.

As many readers know, there have been a number of interesting analysis posts on surface data that have been on various blogs in the past couple of months. But, they’ve been widely scattered. This document was created to pull that collective body of work together.

Of course there will be those who say “but it is not peer reviewed” as some scientific papers are. But the sections in it have been reviewed by thousands before being combined into this new document.  We welcome constructive feedback on this compendium.

Oh and I should mention, the word “robust” only appears once, on page 89, and it’s use is somewhat in jest.

The short read: The surface record is a mess.

Advertisements

  Subscribe  
newest oldest most voted
Notify of
Ruhroh

Anthony;
This is Huge!
Excellent work by all involved.
As a person who heard Richard Feynman deliver that famous Cargo Cult Science talk in 1974, I’m very sure he would be a huge fan of your work.
He loved to skewer stuffed shirt science, and you guys really di a great job on this one.
On behalf of my children, THANK YOU.
RR
REPLY: I appreciate that, but please thank Joe D’Aleo and in particular E.M. Smith, they did more work on this than I. – Anthony

Methow Ken

Downloaded; saved; scanned (full read not doable tonight).
Already I’ve seen enough to say:
Every member of the US Congress should be required to read this.

Mega congratulations to Anthony and Joe. This is the most important climate realist report written to date. A blockbuster. Kudos to all involved, including E.M. Smith, the surface station crew, and SPPI.
This report marks the tipping point. From here on, the AGW scare will steadily decline and disappear. The whole world is in your debt.

Peter of Sydney

Excellent piece of work. Certainly a far cry from the IPCC’s standard of research. I like to see the IPCC provide a response – a scientific one not their usual political spin.

Would you object to my quoting extensively from this in an upcoming article?
REPLY: No but since this is a large compendium, do try to attribute who said what. Ask if needed. -A

Clive

WOW!
And Thank you!!
Copy off to the Canadian Minister of Environment.
Cheers!
Clive
A grateful Canuck! ☺

Leon Brozyna

You have a gift for understatement:

“The surface record is a mess.”

Anthony, this will make good reading on my trip to SF. Joe is not one to fly off the handle, so it ought to be interesting to see to where this leads.

Anthony;
I found the linked website to be an extremely insightful distillation of the unstated assumptions that you are up against with this debunking effort.
I think that the first 3 are pivotal in this case;
1. Authority confers virtue.
2. Authority bestows wisdom.
3. Authority implies benevolence.
4. Authority creates wealth.
I expect that the non-sceptics that you might seek to reach will, without question, assume that
the NCDC is intrinsically virtuous, (more so than 2 guys on a website), wiser (than 2 guys on a website), and more benevolently trustworthy (than 2 guys on a website).
I don’t know how to get those unstated presumptions to be ‘in play’ for the folks you seek to reach, but perhaps it would be by explicitly discussing your pathway from initially sharing those presumptions, (as opposed to having rejected them from the git-go).
You might even note (along the way) in passing your surprise at the situation;
That despite your assumption of (NCDC) virtue, wisdom and benevolence, , you nonetheless have found contrary evidence.
Maybe this would be relevant if you find yourself reprising this Augean effort for a TV camera.
Just an idea from decades of marriage counselling…
RR

Andrew30

Great Work!
I’ve skimmed it twice and will do a full read tomorrow.
This just is:
“Andrew Weaver, probably Canada’s leading climate scientist, is calling for replacement of IPCC leadership and institutional reform.
If Andrew Weaver is heading for the exits, it’s a pretty sure sign that the United Nations agency is under monumental stress. Mr. Weaver, after all, has been a major IPCC science insider for years. He is Canada Research Chair in Climate Modelling and Analysis at the University of Victoria, mastermind of one of the most sophisticated climate modelling systems on the planet, and lead author on two recent landmark IPCC reports.”
http://network.nationalpost.com/np/blogs/fpcomment/archive/2010/01/26/terence-corcoran-heat-wave-closes-in-on-the-ipcc.aspx

Doug in Seattle

Anthony,
Do you still plan on releasing the journal paper you spoke about last summer? Or is this the one?
REPLY: Well this isn’t a journal paper, just a compendium of important issues discovered about the surface temperature record, and yes a full journal “peer reviewed” article is being worked on. – A

Strong stuff!
“…NOAA and NASA in the manipulation of global temperature records…”

Richard Wakefield

Yes, the surface temperature is a mess. I’ve gotten all of Southern Ontario’s data from 110 stations from Environment Canada and started to compile it, first to see how good the data is. It’s pathetic. Only 76 have data at all the rest from Ontario only have pecipitation data, no temperatures. The best coverage was in the mid 1980s with 72% data, but it drops off dramatically to a mere 10% today (up until the 1950’s it was less than 5%). This is because most stations started their data collection after the 1950s’, but for some reason many of them stopped data collection (budget cuts?). Only 10 stations are still collecting measurements and all of them started after the 1960’s. Only 4 have a long data range starting in 1900, but all of them ended before 2005.
What I have been able to get out of this data is very similar to what I found with the one location in Belleville (http://www.scribd.com/doc/25338819/What-Does-Averge-Temperature-Actually-Mean) The increase in the average mean temperature for all of southern Ontario is due to a narrowing of the variation. The number of days above 30C have dropped since the 1920s and the number of days below -20C has also dropped since the 1920’s. The length of the growing season is also increasing since the 1920’s.
Thus I have more evidence that this “warming” is nothing more than a narrowing in the extreme ranges of years temperatures to more moderate ranges today. British Columbia is next on my list.

Congratulations Anthony. I am pleased to see the fruition of your’s and others’ hard work. Keep it up, as you *are* much appreciated.

Stephan

Anthony I thought this was the one that needs an answer:
http://www.skepticalscience.com/On-the-reliability-of-the-US-Surface-Temperature-Record.html
Does your book cover this? or are they too thick to understand. He/they say you only show pics, they show data and their conclusion: there is no significant effect. I would doubt that very strongly…..

REPLY:
This was well along when the Menne paper came out, but I do touch on it in this compendium. I had to get this wrapped up before I could do any substantive replies here. I have a Paper with Pielke Sr. and others we are working on, and it is a fully detailed analysis. That will be the best rebuttal. – A

Tom G(ologist)

Cinderalla – The short story – The shoe fit
Just love stories you can sum up in a few words

JB

Just what the doctor ordered!
A copy is being sent to my MP.
Thank you!

George E. Smith

Well I just downloaded it and got it safely saved in my Climate file. Haven’t had time to do ther than look at the table of contents; but it looks like a substantial piece of work.
Very nice effort there Joe and E. M. Smith and Anthony.
I’ve already stated many times that I have little confidence in the surface data prior to the age of polar orbit satellites, and the oceanic buoys, which I date from about 1979/80.
So I am happy to read that you too feel that early record is highly contaminated.
Jolly good show chaps.

wayne

Gentlemen, hat’s off to you!

crosspatch

Anthony, is this true:

Global terrestrial temperature data are gravely compromised because more than three-quarters of the 6,000 stations that once existed are no longer reporting.

Or is it more a case of something like:
Global terrestrial temperature data are gravely compromised because more than three-quarters of the 6,000 stations that once existed are no longer having their reports included in the database.
I know that many of the stations that have been dropped from the GHCN are still there and still reporting, it is just that their reports are no longer included in the data.
REPLY: It is a combination of both, but I agree that could be worded for improved comprehension – A

REPLY: I appreciate that, but please thank Joe D’Aleo and in particular E.M. Smith, they did more work on this than I. – Anthony

So it seems to me that E.M. Smith deserves authorship credit along with Joe and you. Smith may not have written much of the text, but he was certainly a major contributor.
REPLY: Most certainly he is.

henry

You’re being mentioned over at Weather Underground;
http://www.wunderground.com/blog/JeffMasters/show.html
Primarily as an aside to a report given by Dr. Matthew Menne and co-authors at NOAA’s National Climatic Data Center (NCDC). In a talk at last week’s 90th Annual Meeting of the American Meteorological Society, Dr. Menne reported the results of their new paper just accepted for publication in the Journal of Geophysical Research titled, On the reliability of the U.S. Surface Temperature Record (http://www1.ncdc.noaa.gov/pub/data/ushcn/v2/monthly/menne-etal2010.pdf)
You really need to read Dr Jeff Master’s take on this paper.
At the end, he states:
“The surfacestations.org effort coordinated by Anthony Watts has made a valuable contribution to science, helping us better understand the nature of the errors in the U.S. historical temperature data set. In his talk last week at the AMS conference, and in the credits of his paper, Dr. Menne had some genuinely grateful comments on the efforts of Anthony Watts and the volunteers of surfacestations.org. However, as of this writing, Watts has made no mention on surfacestations.org or on wattsupwiththat.com of Dr. Menne’s study.”
Looking forward to your take on this paper.
REPLY: Been a little busy, see above, that’s my next project. Besides Menne’s paper was a surprise. I’ll explain in a future post. – A

JEM

It’s Chile, not Chili.
Just a quibble. Still reading and digesting.
REPLY: Damn spell checkers 😉

Anthony,
I downloaded a copy and sent one to Congressman McClintock and to California Assemblyman Dan Logue who is sponsoring and initiative to resend AB32, which calls for a California Cap and Trade scheme. I hope this will help them make the case the surface record is an unreliable mess, and any Cap and Trade bill would be based on faulty data.
Congratulation to you, Joe and E.M. Smith for a very interesting document.
I am please to hear that you are working on a rebuttal to the Menne paper. It see to be more of a cover their (you know what) paper.

Good work!
One relatively minor criticism: You have only one bookmark in the PDF, for “CASE 4: CANADA’S WEATHER NETWORK”. A fully-fleshed out set of bookmarks based on the TOC (and clickable links in the TOC itself) would make it much more reference-friendly.

Gary Crough

Thank you Joseph D’Aleo and Anthony Watts for all this good work.
From day 1 it seemed to me that a bunch of weather stations was a poor means of determining global temperature. They did not cover the whole earth; omitting whole oceans and focusing on urban locations. Still, prior to the satellite-based measurements they were the best proxy for global temperature and if used just to measure trends they could have been a very good proxy if:
· Measurements were taken from the same set of global positions each year
· Adjustments for creeping urbanization, equipment upgrades and equipment movements were made honestly
Sadly, neither condition was met.
I ignored land-based climate data in favor of RSS and UAH data. But evaluating claims about the “warmest decade” in 150 years requires one to look into surface station data. I knew it would be a mess and researchers having to make close decisions would be biased towards global warming … if only to make their jobs seem more important. But this report (and others) make this look more like fraud that simple bias.
Original sources claim they sent data into NOAA (or the Hadley CRU) and it was simply not used. If so, it should be possible for GISS etc. to re-do their calculations with all the data. Why not simply demand they do so? What excuse is provided for omitting existing data?
Alternatively, GISS could run their calculations using only the subset of stations (1500 ?) used in the latest calculations. Seems like a simple request since GISS has both the software and the data. This is the less attractive alternative as the existing stations are biased towards heat-island locations.
I have a question: Is it possible for a 3rd party to re-calculate the global temperature trend using data that was reported to NOAA but not used? If so, the output of such a project would trump anything done by GISS or Hadley CRU. As I understand it GISS is now responding to FOI requests (rather than face a court trial) and I would expect NOAA to do the same? If so, the software and data to create a much more honest climate trend should be in the public domain?

John F. Hultquist

Question, Anthony. Is the file for this document correctable and do you want to know of things? I was doing a real quick look-thru and found this:
page 62, 4th line under NONSENSE! there is a (the) that needs to be removed.
If you want to know of these please say so and I will make a list if I find any more – and I will read for them as well as reading for content. I’m impressed with the first look at the first 62 pages.
I also bought the paperback of Climategate and advise others to do so but I have read even less of it. Of course I’ve read a lot of both of these as these stories played out over the last weeks and months. It is great to see this stuff in one place and spruced up, though. Thanks to all.
REPLY: We plan to keep this as a living document, with version numbers as we update, so yes such things are welcome. We are depending on all of you for our “peer review”. Check your inbox for an email, send there. – A

John F. Hultquist

Ric Werme (21:36:17) : authorship credit
Not to try to refute your notion but a book is known by its authors and not by all the contributors or even the main ones. Consider the King James version of the bible. Did K. James write any of it?

Dave F

Getting ready to download this. I won’t get to read it for a while. At least not until the semester ends. I have too much reading to do before then to commit to 100 pages, but I am going to slowly leaf through in between cram sessions. If I find anything wrong, I will let you know, per your comment to John F. Hultquist (21:49:39) :, but it may be a month or two later than you hear from others.
You know, unless I get hooked and decide to skip an assignment to finish your paper. 🙂

geo

Wheee!
“Instrumental temperature data for the pre-satellite era (1850-1980) have been so widely, systematically, and unidirectionally tampered with that it cannot be credibly asserted there has been any significant “global warming” in the 20th century.”
I’d like to think I’ve done my part in my own small way for surfacestations.org, mostly in the upper midwest, but that one is a hard one to swallow. Almost as hard as “MWP? What MWP?” from the other side. Overstated? Yes, I can believe that easily. “Cannot be credibly asserted”? I look forward to a close reading of your arguement is as far as I can go right now.
I look forward to reading the report carefully to see if I am convinced this is the case. The rest of the “Summary for Policymakers” seems to me at least easily defensible from what I know up to know, but #1 is going to take some convincing.

rbateman

I shall read this, and hopefully I can get some ideas on making the most of the horrid mess my poor town’s temperature records are in.
And are they ever in a mess.
Thank you E.M Smith, Joe D’Aleo and Anthony Watts.

jorgekafkazar

[snip – for the same reason I don’t allow the Hitler parody videos here]

Norm in Calgary

Glad to see that there appears to be many Canadians on this blog. Is there anything comparable to the surface stations report on how Canadian surface stations are sited?

D. King

It is mind-numbing the level of deception that went
into this. I don’t know how you guys stayed sane.
Your methodical and relentless unraveling of this
should get you a medal. I suspect that we will see
other, more sophisticated methods of manipulating
the sensors themselves (sea ice).
Good work gentlemen.

Doug in Seattle

Thanks for your answer up above.
As for this paper today, great summary of the issues. It provides a much better background for the station drop out argument and the UHI presentation is well laid out in all its gruesome details. Nice final bit too using McIntyre’s “Hide it” piece.

Thank you, thank you, thank you!
Keeping the spotlight of public attention on the climategate iceberg is probably the most kind and loving action you could take on behalf of everyone living on planet Earth today.
With deep gratitude,
Oliver K. Manuel

Have just been skimming and this is a delight to read … First Mosher’s post and now this … how’s a part-time blogger ever to keep up, eh?!
This has been a *very* good day, today. Corcoran’s piece on Andrew Weaver jumping ship (Weaver must be feeling really silly about his attempt to bolster the CRU crew when he was getting media mileage from claim that his office had been broken into – without disclosing that there had been UVIC wide alert regarding potential theft).
Now the U.K. Times reports that John Beddington, U.K.’s chief scientific advisor has stated:
There is fundamental uncertainty in climate change, science tsar says
“The impact of global warming has been exaggerated by some scientists and there is an urgent need for more honest disclosure of the uncertainty of predictions about the rate of climate change, according to the Government’s chief scientific adviser. “John Beddington was speaking to The Times in the wake of an admission by the Intergovernmental Panel on Climate Change (IPCC) that it grossly overstated the rate at which Himalayan glaciers were receding. “Professor Beddington said that climate scientists should be less hostile to sceptics who questioned man-made global warming. He condemned scientists who refused to publish the data underpinning their reports. “He said that public confidence in climate science would be improved if there were more openness about its uncertainties, even if that meant admitting that sceptics had been right on some hotly-disputed issues.”
http://www.timesonline.co.uk/tol/news/environment/article7003622.ece

richard verney

Anthony.
I have quickly read your paper (which has obviously taken much work) with interest and there are many good points made. Hopefully, a copy will be submitted to the UK Parlimentary Committee which is looking into some of the issues arising from Climategate.
I don’t find Menne’s paper a surprise. If one is looking for a trend, it does not matter that station A is pure, whereas Station B is contaminated by noise provided that the contaimnation to Station B remains constant throughout the period over which the trend is being examined. In countries such as the US, most urban development/growth predates the period considered by Menne and hence when looking for temperature trends (rather than absolute accuracy in the temperature measurement), during the period considered by Menne, one would not expect to see substantial differences between good and bad sited stations, or between urban and rural stations. Materially, his paper does not deal with station drop out (which appers to have resulted in a warming bias), nor does his paper deal with homogenising the pre 1960 temperatures downwards. As such, it does not deal with the major flaws in the temperature records which your recent paper deals with.
Since global warming is not a global problem (the amount of warming and its effect are often dictated by microclimatic conditions/sensitivities), I would have thought that rather than seeking to create some artificial global temperature record, it would be more useful to compile individual records for each country (and if the country is large or has special and significant geological/ecological features then to compile a number of records as appropriate). When seeking to compile such a record, one would seek to identify station data that requires no form of adjustment/manipulation so that the record can be compiled exclusively on raw data (wherever possible). No doubt, the use of only well sited rural stations would give the most accurate temperature record.
An advantage of compiling temperature records in this manner is that one could more easily see whether various areas of the globe (northern, southern hemishpere, equatorial or polar) are warming at different rates which would give further insight into wher the warming anthropogenic.

Deadman

I thank you for your exhaustive work.
If you would like a comprehensive list of the pardonable typographical slips, grammatical errors and syntactical infelicities, please send me the appropriate e-mail address.

George Turner

Great work!
I found it interesting that most of the automated reporting stations for aviation only have to be accurate to plus or minus 0.9 F, and may be biased. If I was setting up an airport thermometer with a bias I’d be sure to make it read high, because reading low could be dangerous in terms of aircraft performance on takeoff (denisty altitude and such). Then we’d have an aircraft that shoots off the end of the runway, clipping trees in half, and a pilot who climbs out and starts measuring the trees’ growth rings to check airport thermometer calibration, since tree-rings can be read to a hundredth or so.. 😉
Then again, maybe we should go with Briffa’s post-1960 data!

George Turner

Oh, my point was that it might be interesting to talk to the people who set up the airport reporting stations to see if they make sure the thermometer reads accurate or high, but never low, for safety reasons, despite what the published specification says.

crosspatch

Another thing that SteveM noted about missing months in Siberia. Apparently many of the stations NOAA reported as having missing values for various months actually had monthly values that were available from other sources. I believe that was reported on this site as well. The discussion came up, if you remember, at about the time that there was a very odd anomaly in Russia that turned out to be a month where the previous month’s value was repeated in the NOAA data. A quick look at sources available online showed the correct value was available for the same station. Further checking showed that months “missing” values from NOAA’s data set were actually available.
This is important because a “missing” value allows a new “fill” value to be calculated to take its place. This “fill” value is never replaced even if a correct real observation is discovered. This, in turn, is important because these “fill” values are based on an average over time. It causes positive feedback and even greater warming bias. If this month is a warm month, than all other fill values in past years for this month get bumped up a little because this months warm observation increased the average over time. That is one reason why you can see current temperatures influencing the past.
The more of these fill values you have, the more you get the opportunity to influence the past. I find it odd that these calculated fill values are never replaced with actual observational data when it is discovered.

Paul Metcalf

Thanks for all of this. As a returning student to the halls of higher education I have already singled this subject out as one of my go to subjects for those dreaded term papers.
I’ve already managed to garner sideways glances from teachers at Butte and CSU.
After scanning this I can see that my good times are just beginning.
Thanks for all the hard work.

kwik

I have read it. Its really hard to accept, but what can you say?
One thing I think would be enlightening;
A flowchart of the data-path between the different organisations. And small explanatory boxes showing where data adjustments are done?
It will be easier for people that never heard of this before to understand where the “adjustments” happens, and who is using common adjusted data.
REPLY: I’ve been wanting to do a flowchart for quite awhile. -A

Konrad

O/T
I just missed out on getting in to see Lord Monckton in Sydney. The venue was filled to capacity and a large number of people had to be turned away with Lord Monckton’s apologies. Interesting times…

D. King (22:08:44): Your methodical and relentless unraveling of this should get you a medal.
The Nobel Peace Prize, for starters. NASA and NOAA have some explaining to do. I demand a full Congressional investigation. We cannot trust our scientific institutions any longer. Without immediate Congressional action to repair them, those agencies are useless and should be defunded.
On a related note, tomorrow (Jan 27, 2010) the Security and Exchange Commission (SEC) plans to issue a new rule requiring corporations to explain how they are “alleviating global warming.” The “interpretive release” will have the effect of force of law with no hearings, taking no testimony, and without statutory authority to exercise jurisdiction over global warming “abatement.”
The SEC failed (with disastrous consequences for the entire world) to rein in investment banks and their credit default swaps that undermined the financial sector worldwide in 2008. The SEC failed to respond to repeated warnings about the Madoff Ponzi scheme, resulting in $65 billion in losses to investors. Now the SEC has turned their defective attentions to the global warming hoax, not with the intention to protect investors from the fraud, but just the opposite — to force corporations to further the hoax.
As a result of the SEC’s ill-considered action tomorrow, corporations will be subject to civil lawsuits and criminal penalties if they do NOT participate in the greatest hoax in history.
http://tinyurl.com/ycnkfm4

A big compendium of nonsense here. I’ll try to make a start.

More than 6000 stations were active in the mid- 1970s. 1500 or less are in use today.

This just propagates a misunderstanding of what GHCN is. 6000 stations were not active (for GHCN) in the 1970’s. GHCN was a historical climatology project of the 1990’s. V1 came out in 1992, V2 in 1997. As part of that, they collected a large number of archives, sifted through them, and over time put historic data from a very large number in their archives.
After 1997, it was decided to continue to update the archive. But it wasn’t possible to continue to regularly update monthly all the sites that had provided batches of historic data to the original collection. That’s a different kind of operation. They could only, on a regular basis, maintain a smaller number. This notion of a vast swag of sites being discontinued about 1992 is very misleading. 1992 is about when regular reporting started.

It is only when data from the more southerly, warmer locations is used in the interpolation to the vacant grid boxes that an artificial warming is introduced

A constantly repeated, way-off meme. Firstly, there’s little quantification of such a drift. But the main thing is, all the GMST calcs are done with anomaly data. Station temps measured with respect to their own mean over a period, or at most, at their own supplemented with some nearby station data. It doesn’t matter if stations are replaced with other stations of higher mean.
What could matter is if stations are replaced by others with a higher warming trend. And that’s where this argument gets really silly. The stations with higher warming trend are at higher latitudes. Shifting stations away from the poles (to whatever extent it may have happened) would have a cooling trend, not warming.

Interestingly, the very same stations that have been deleted from the world climate network were retained for computing the average-temperature base periods

Misunderstanding of how anomalies are actually calculated underlie a lot of the argument about station shifts. They do not calculate a global average and then subtract it. The basic method is the Climate Anomaly Method, which NOAA uses. Each station has an anomaly calculated with respect to its own average.
Gistemp uses the same method, but applied to grid points (Sec 4,2), rather than individual stations. Again, this is very little affect by any general drift in stations – the grid points don’t move.

Great work, I have a particular gripe with Salinger and his manipulation of NZ temperature data. He is a witness for Mighty River Power in the Called -In Turitea wind farm, in Palmerston North. The AGW lobby are out to sacrifice this city to their global warming god ( note I used a small g ). Salinger is totally discredited and has been outed here.
http://www.palmerston-north.info

UK Sceptic

O/T but worthy of a mention. The warmistas are getting very, very desperate.
http://www.thestar.com.my/news/story.asp?file=/2010/1/27/nation/20100127092338&sec=nation

I’ve been waiting impatiently for this. Well done to Anthony, Joe, E. Michael Smith and Willis and others for exposing the climate liars and thank you for your dedication to the search for truth.