I was working on another project related to the CRU emails and came across this email from Dr.Phil Jones. I was stunned, not only because he was dissing another dataset, but mostly because that dissing hit many of the points about problems with the NASA GISS products we’ve covered here on WUWT and at Climate Audit.
Here’s the email with my highlights added. Email addresses have been partially redacted.
The original email can be seen at this link:
http://www.eastangliaemails.com/emails.php?eid=1042&filename=1254850534.txt
Here’s the thing, we’ve seen the problems with CRU’s temperature series in the code already. If Dr. Jones is aware of those problems, and he thinks GISS is inferior, well then, wow, just how bad is GISS?
I thought this statement was quite telling:
Their non-use of a base period (GISS using something very odd and NCDC first differences) means they can use
very short series that we can’t (as they don’t have base periods) but with short series it is impossible to assess for homogeneity.
One thing about GISS that has bothered a lot of people – the base period they use for calculating temperature anomaly is for 1951-1980. See it listed here on the GISTEMP page. No other data sets use that period. Critics (including myself) have said that by using that period, it makes this graph’s trend look steeper than it would if the current 30 year period was used.
In the past couple of years we’ve seen two significant errors with NASA GISS that had to be corrected after they were discovered through the work done here at at WUWT and Climate audit. Public errors have not been found in CRU products during that time, because the data an code have been withheld.
To the credit of NASA GISS, they have been more transparent than CRU on data, stations used, and code.
Here are some of the relevant posts on WUWT where we address issues found with the NASA GISS temperature products:
How bad is the global temperature data?
And now, the most influential station in the GISS record is …
NASA GISS: adjustments galore, rewriting U.S. climate history
Absence makes the chart grow fonder
A comphrehensive comparison of GISS and UAH global Temperature data
Getting crabby – another missing NASA GISS station found, thanks to a TV show
More on NOAA’s FUBAR Honolulu “record highs” ASOS debacle, PLUS finding a long lost GISS station
Weather Station Data: raw or adjusted?
GISS Divergence with satellite temperatures since the start of 2003
Divergence Between GISS and UAH since 1980
GISS’s Gavin Schmidt credits WUWT community with spotting the error
GISS, NOAA, GHCN and the odd Russian temperature anomaly – “It’s all pipes!”
Corrected NASA GISTEMP data has been posted
A new view on GISS data, per Lucia
The Accidental Tourist (aka The GISS World Tour)
Rewriting History, Time and Time Again
Why Does NASA GISS Oppose Satellites?
How not to measure temperature, part 52: Another UFA sighted in Arizona
How not to measure temperature, part 51.
NASA’s Hansen Frees the Code !
Does Hansen’s Error “Matter”? – guest post by Steve McIntyre
1998 no longer the hottest year on record in USA
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
One thing about GISS that has bothered a lot of people – the base period they use for calculating temperature anomaly is for 1951-1980.
That is probably the best choice, because temperature was relatively stable from 1950-1980.
Here is the link to what he claims about the archival process of cliamte data:
http://www.chron.com/commons/readerblogs/atmosphere.html?plckController=Blog&plckScript=blogScript&plckElementId=blogDest&plckBlogPage=BlogViewPost&plckPostId=Blog%3a54e0b21f-aaba-475d-87ab-1df5075ce621Post%3ade6fb89a-bf98-4503-a332-344214aa9a9b
I think this is a significant point to understand. I would appreciate any help in clarifying it.
Please check this out at Rajendra Pachauri’s blog
Award of Honorary Degree by University of East Anglia “Doctor of Science, honoris causa” on 17 July 2008, University of East Anglia, UK
Ecotretas
Paging Phil Jones! The Cracker Jack Company is calling and they want your PH d. back, they claim your are damaging the brand.
I’m going to stick my neck out defend the guys who produced those long-term climate records. As I understand it they use three different approaches:
– CRU tries to use a sub-set of stations (c 4000) with long records.
– NCDC uses a larger number of stations (c 7000), some of which have shorter records, to estimate grid temperatures.
– GISS also uses a large number of records but works on difference over time, assuming that stations in or close a particular grid will all warm or cool at a similar rate.
If these three methods give similar results for underlying trends that in itself gives the results a degree of confidence.
Most climate data currently used comes via a system operated under the auspices of the World Meteorological Organisation. Because of their need for consistent long records the CRU asked for and obtained additional data directly from meteorological services. It is a fact, regrettable but true, that many Met services charge for data and under these circumstances it is logical they would place an embargo on onward transmission of data.
As anyone who has worked with long-term climate records knows (and I have done this in more than 30 countries in 4 continents) it is very difficult to sort the wheat from the chaff. You do double-mass plots, inter-station correlation (to detect unreported infilling), sometimes you can access metadata, but it is never easy. It is possible that that if the CRU or others found data which gave the results they expected they checked less carefully than if results were unexpected. It is possible that their methods of allowing for the heat island effect were deficient. But we should at least recognise that they did a necessary, unglamorous and often thankless task and try to build on what they have done rather than rubbish it.
“They also impose some urbanization adjustment which is based on population/night lights”
This seems to be refering to GISTemp but they say:
http://data.giss.nasa.gov/gistemp/
“The GHCN/USHCN/SCAR data are modified in two steps to obtain station data from which our tables, graphs, and maps are constructed. In step 1, if there are multiple records at a given location, these are combined into one record; in step 2, the urban and peri-urban (i.e., other than rural) stations are adjusted so that their long-term trend matches that of the mean of neighboring rural stations. Urban stations without nearby rural stations are dropped.”
No mention of population/night lights.
Which is right, or was he not refering to GISS?
Also UHI is not I think just a long-term trend issue, it is a diurnal issue and seasonal issue. If the data is used to determine day/night changes, i.e nights warming faster/slower than days, or winter/summer changes, winters warming faster/slower than summers then perhaps it is best just to drop the urban/peri-urban stations altogether.
I sometimes wonder if weather stations tend to be positioned in places that are going to have troubled future temperatures. Perhaps we should cluster around them to keep warm.
I cannot see anyway of properly accounting for station-drift, whether it is UHI, the paint-peeling off, or any other cause. Sudden changes, new equipment, resiting, etc., should show up as jumps in the record and they can be looked for, but even so it might be better to just say that the records should be dealt with as two seperate weather stations. Really long term records are critical to preserving the low frequency components (long-term trends) but they need to be free of any discontinuities or drifts. Just because a record has been kept at a particular site for 100 years does not necessarily mean that it is free from discontinuities and can be treated as a continuous record.
A point for much debate occurs when stations are corrected for drift, discontinuities and perhaps both. How should this be done? Are discontinuties corrected because they are apparent from the data or because the station history documents a change in the station/environment? I don’t know what criteria they use. When is a drift a drift not a trend? It is a nightmare.
More to the point, I would think that given some of the known problems, it should be able to predict from the mix of various different lengths of weather station record just how much low frequency information is likley to be reliably maintained. My guess would be that it would be shorter than the length of the longest records and perhaps much shorter. That would imply that the recovered long-term trend should be viewed with much caution.
Perhaps this would not be a bad thing, in fact it might be a good thing. A lot of the well publicised verification of the models relates to their ability to account for the low frequency component of the recovered temperature record. It might be better to assume that the historic long-term trend is very uncertain and validate the models on how well they account for the wiggles like the 1910s-1940s wiggle, the 1940s-1970s wiggle, the 1970s-1990s wiggle and the current 2000s wiggle. I think we know that they are not too hot on the 1910s-1940s wiggle, and suspicion is mounting on the 2000s wiggle. I believe that we can have more confidence in the record on a decadal basis than on a centenial one, so perhaps that is a better test of the models overall performance than just getting the trend right. Even the simplest of models can get the trend right given a view basic assumptions. Or by regressions against known factors.
Obviously some short term events like volcanoes do cause the models to move in the correct general directions and they should match the record quite well on those (perhaps better than they actually do), but as I said they do seem to struggle to account for the 1910s-1940s warming. Here I refer to GCMs, simpler models can fare better, you just have to multiply the effect of TSI by a large factor. AFAIAA only simple models with high values of PDO effect match the 2000s wiggle, but as of yet it is probably too short a discepancy to be very damning.
If there has been any tuning of the models to get the trend right they may have made themselves hostages to fortune, simply because the recovered record is subject to revision and more importantly may simply be wrong. If they have been tuned to an incorrect trend then they are likley to diverge quite badly as the decades pass, or they will have to be retuned.
I am not saying that they do tune the models deliberately or even unintentionally just that it might be safer to focus on how well they get the more evidentially sound wiggles right rather than the trend.
Alex
Are there any definitive papers or text books that describe the use of Tree Rings as Temperature Proxies in climate science? Just wondering what the text books say about it.
Speaking of data, realclimate is frenetically unlocking the cellar after the corpses were found:
http://www.realclimate.org/index.php/archives/2009/11/wheres-the-data/
“The climate science community fully understands how important it is that data sources are made as open and transparent as possible….”
It just gets funnier.
More Giss v Hadley
http://www.eastangliaemails.com/emails.php?eid=926&filename=1224176459.txt
From: Michael Mann
To: Phil Jones
Subject: Re: Why are the temperature data from Hadley different from NASA?
Date: Thu, 16 Oct 2008 13:00:59 -0400
Cc: Judith Lean , Yousif K Kharaka
thanks Phil–this all makes sense. I’ll be intrigued to hear more about how the melting sea ice issue is going to be dealt with. no question there is a lot of warming going on up
there.
hope to see you one of these days,
mike
On Oct 16, 2008, at 6:52 AM, Phil Jones wrote:
Hi Mike, Judith and Yousif,
Mike has basically answered the question. The GISS group average surface T data into 80 equal area boxes across the world. The UK group (CRU/MOHC) grid the data into 5 by 5 degree lat/long boxes, as does NCDC. These griddings don’t allow so much extrapolation of data – no extrapolation beyond the small grid box. The US groups also calculate the globe as one domain, whereas we in the UK use (NH+SH)/2. This also makes some difference as most of the missing areas are in the SH, and currently the NH is warmer than the SH with respect to 1961-90. Our rationale for doing what we do is that it is better to estimate the missing areas of the SH (which we do by tacitly assuming they are the average of the rest of the SH) from the rest of the SH as opposed to the rest of the world.
The Arctic is a problem now. With less sea ice, we are getting SST data in for regions for which we have no 1961-90 averages – because it used to sea ice (so had no measurements).
We are not using any of the SST from the central Arctic in summer.
So we are probably underestimating temperatures in the recent few years. We’re working on what we can do about this. There are also more general SST issues in recent years. In 1990, for example, almost all SST values came from ships. By 2000 there were about 20% from Buoys and Drifters, but by 2008 this percentage is about 85%. We’re also doing comparisons of the drifters with the ships where both are plentiful, as it is likely that drifters measure a tenth of one degree C cooler than ships, and the 1961-90 period is ship-based average.
New version of the dataset coming in summer 2009.
All the skeptics look at the land data to explain differences between datasets and say urbanization is responsible for some or all of the warming. The real problem is the marine data at the moment.
Attaching a recent paper on urbanization and effects in China.
Cheers
Phil
At 22:08 15/10/2008, Michael Mann wrote:
Hi Judith,
Its nice to hear from you, been too long (several years??). My understanding is that the differences arise largely from how missing data are dealt with. For example, in Jim et al’s record the sparse available arctic data are interpolated over large regions, whereas Phil an co. either use the available samples or in other versions (e.g. Brohan et al) use optimal interpolation techniques. The bottom line is that Hansen et al ‘j05 I believe weights the high-latitude warming quite a bit more, which is why he gets a warmer ’05, while Phil and co find ’98 to be warmer.
But Phil can certainly provide a more informed and complete answer!
mike
p.s. see you at AGU this year??
On Oct 15, 2008, at 5:03 PM, Judith Lean wrote:
Hi Yousif,
Many apologies for not replying sooner to your email – but I’ve only just returned from travel and am still catching up with email.
Unfortunately, I am simply a “user” of the surface temperature data record and not an expert at all, so cannot help you understand the specific issues of the analysis of the various stations that produce the differences that you identify. I too would like to know the reason for the differences.
Fortunately, there are experts who can tell us, and I am copying this email to Mike Mann and Phil Jones who are such experts.
Mike and Phil (hi! hope you are both well!), can you please, please help us to
understand these differences that Yousif points out in the GISS and Hadley Center surface temperature records (see two attached articles).
Many thanks, for even a brief answer, or some reference.
Judith
On Oct 8, 2008, at 1:50 PM, Yousif K Kharaka wrote:
Judith:
I hope you are doing well (these days OK would be good!) at work and personally. Can you help me to understand the huge discrepancy (see below) between the temperature data from the Hadley Center and GISS? Any simple explanations, or references that I can read on this topic? I certainly would appreciate your help on this.
Best regards. Yousif Kharaka
Yousif Kharaka, Research Geochemist Phone: (650) 329-4535
U. S. Geological Survey, MS 427 Fax: (650) 329-4538
345, Middlefield Road Mail: [1]ykharaka@xxxxxxxxx.xxx
Menlo Park, California 94025, USA
—– Forwarded by Yousif K Kharaka/WRD/USGS/DOI on 10/08/2008 10:42 AM —–
Yousif K Kharaka/WRD/USGS/DOI
10/06/2008 02:07 PM
To “Dr David Jenkins”
Subject
Why are the temperature data from Hadley different from NASA? [18]Link
David and all:
One advantage (or great disadvantage if you are very busy!) of membership in GCCC is that you are forced to investigate topics outside your areas of expertise. For some time now, I have been puzzled as to why global temperature data from the British Hadley Centre are different from those reported by NASA GISS, especially in the last 10 years.
GISS reports that 2005 was the warmest year (see first attachment) on record, and that 2007 tied 1998 for the second place. The Hadley group continues reporting 1998 (a strong El Nino year) as having the highest global temperature, and then showing temperature decreases thereafter. The two groups report their temperatures relative to different time intervals (1951-1980 for GISS; 1961-1990 for Hadley), but much more important is
the fact that GISS data include temperatures from the heating Arctic that are excluded by others (see second attachment). If you are interested in the topic of sun spots, the 11-year irradiance cycle, and solar forcing versus AGHGs, see the first attachment for what NASA has to say. We may need help on this complex topic from a “true climate scientists”, such as Judith Lean!
Cheers. Yousif Kharaka
Yousif Kharaka, Research Geochemist Phone: (650) 329-4535
U. S. Geological Survey, MS 427 Fax: (650) 329-4538
345, Middlefield Road Mail: [19]ykharaka@xxxxxxxxx.xxx
Menlo Park, California 94025, USA
On a different tack, when judging how well the datasets compare it might be better to concentrate regionally and on the land.
The ocean record is the biggest influence on the global record and three of the datasets seem to rely either heavily (2) or to some degree(1) on Hadley datasets. Also there is a shared data between the satellite record and at least one of the established datasets which makes validation to the satellite record a bit dubious.
Perhaps it would be better to concentrate on why they are different at all, than marvelling at how closely they correspond.
Alex
What do you make of this page of raw and processed data from Real Climate? http://www.realclimate.org/index.php/data-sources
The gift that keeps on giving!
When you strip away all the methodologies of anomaly calculations. conflicting graphs and overblown projections of runaway extrapolation, we are left with the notion of measurable global temp rise (within instrument limitations and type) and baseline CO2 as the normal.
Using 280ppm and 15C, neither of which is certain for the longterm average, the road to the finish line has become quite narrow; we must go off the edge to make a U-turn.
son of mulder – I see they’ve closed that HYS, clearly not getting the answers they wanted : )
son of mulder (07:09:40) :
” Plato Says (06:23:33) : But we’re what 10 days in and I tripped across an article in Pravda that it was more informative than anything I’ve seen on the BBC’s own website.”
But just look at this BBC ‘have your say ‘ comments, things are changing
http://newsforums.bbc.co.uk/nol/thread.jspa?sortBy=2&forumID=7283&edition=1&ttl=20091130150834&#paginator
Now that’s cheered me up!
Last weekend when Climategate broke I checked the domain “climategate.co.uk” and it was available. I thought about registering it for £2.99 – which is what it would have cost, but then changed my mind and didn’t bother!
I notice it has now been registered on 24-nov-09, here is the look up for it http://whois.domaintools.com/climategate.co.uk and it looks like they got it just to put it up for sale under Sedo! http://www.climategate.co.uk/
Always someone out to make a packet!
Which normal to use? the one that shows more warming of course…..
http://www.eastangliaemails.com/emails.php?eid=462&filename=1105019698.txt
At 09:22 05/01/2005, Parker, David (Met Office) wrote:
There is a preference in the atmospheric observations chapter of IPCC
AR4 to stay with the 1961-1990 normals. This is partly because a change
of normals confuses users, e.g. anomalies will seem less positive than
before if we change to newer normals, so the impression of global
warming will be muted.
That doesn’t sound like Jo writing to me. It’s more than likely been written by a Warmist spin-doctor and passed to her for publication. It completely ignores the issue.
I would love to show proof, but I’m pretty sure the emails will have been deleted.
Need some help.
One of the WUWT items included a brief statement which was an very short, but a very excellent list of the punch points of Climate Gate. I believe it was written by an Australian Professor from the University of Western…
Since I don’t recall his name at the moment (sorry) or the University,
I am having trouble finding that very well put quote. I believe it may have
originally been in a comment section, but WUWT singled it out in one of
their reports – perhaps at the bottom as an “oh by the way, you might like this” sort of thing. Well, yes, I did like it and I would like to see it again.
If someone has it, would you please repost it. Perhaps the Moderators could add it into the bottom of another report as well. In my humble opinion, it
was very short, but filled with punch, as it listed major problems that
are being uncovered by ClimateGate.
Thank you
Not sure where you want links now.
This in Fox News today could possibly cause a Beatles-like backlash, when the Beatles created a row by saying they were more popular than JC. Cannot recall.
Glad they revealed this though!
http://www.foxnews.com/story/0,2933,577827,00.html
Document Reveals U.N.’s Goal of Becoming Rule-Maker in Global Environmental Talks
Monday, November 30, 2009
By George Russell
Environmentalism should be regarded on the same level with religion “as the only compelling, value-based narrative available to humanity,” according to a paper written two years ago to influence the future strategy of the United Nations Environmental Program (UNEP), the world’s would-be environmental watchdog.
pwl (08:32:54) :
For the bit I have looked at previously (GHCN v.2) it depends on what you expect raw to imply. If by raw you mean actual thermometer readings then it is not raw. The data is processed into monthly data and I do not know how that is done (e.g. how the mean of the data has been defined).
The readme is here: ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v2/v2.temperature.readme
From reading just now USHCN is version dependent:
v1 seems to be contain a lot of adjusted data.
ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/README.TXT
v2 seems to contain both some “raw” and some adjested datasets. But again I think raw is monthly data not thermometer readings.
ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2/monthly/readme.txt
If by raw data anyone expects thermometer readings then I guess you would have to go to the actual log books.
Alex
SteveS (03:12:58) :
“On a more serious note: I don’t feel optimistic about this at all.”
Give it time. The Team has made many enemies in their field. (For instance, that guy Karlen from Scandanavia who got brushed off by T____.) They have been waiting for an opportune moment to strike back and be heard. Now they have it. As some of them speak up, others will be encouraged to come forward. And the strength of their condemnation of the IPCC and the Team will rise, and lots more dirty linen will be aired. It’ll develop along the lines of Watergate, with the public getting hooked on their weekly scandal, and the defenders in the bunker getting more and more implausible and desperate.
As that happens, the media will be more inclined to pay attention to them. There will be articles exploring, with the aid of graphics, the links (remember that word?) between the Team, the IPCC, and the various gatekeepers in the field. There may even be articles exploring topics like, “What is climate science all about anyway?”
A tectonic shift is underway. The media’s current silence is an indication that they are re-assessing the situation, and that their treatment in the future will be less outrageous. Even if you don’t grant them any sense of fairness at all, which is silly, you should realize that they have to be concerned about not alienating the skeptical portyion of their readership too badly, now that noticeable segments of it are sounding off in their online comments sections. Previously, they were only getting badgered by the enviros, whenever they failed to toe the line. Now, the forces on the contrarians side are coalescing, mainly thanks to internet sites like this, and making their impact felt.
Oops—when I wrote, “As that happens, the media will be more inclined to pay attention to them,” I was referring to the emerging voices of dissent, not to the beleaguered boffins in the bunker.
DISSENTERS!!! DIOXIDE DISSENTERS!! That’s a D-word we can live with! How about it, gang?
A) How can you tell if the data at RC is truly the RAW data and not processed? B) Are we seeing a competition between CRU, NOAA, GISS for grant money? Does that explain the back-stabbing comments?
pwl (02:30:12) : “This decline in the tree ring data suggests that the tree ring temperature proxy data is NOT useful for accessing temperature histories.”
That was exactly what I thought when I read about the discrepancy. No one (with any scientific authority) has yet commented on your valid question.