CRU Emails "may" be open to interpretation, but commented code by the programmer tells the real story

When the CRU emails first made it into news stories, there was immediate reaction from the head of CRU, Dr. Phil Jones over this passage in an email:

From a yahoo.com news story:

In one leaked e-mail, the research center’s director, Phil Jones, writes to colleagues about graphs showing climate statistics over the last millennium. He alludes to a technique used by a fellow scientist to “hide the decline” in recent global temperatures. Some evidence appears to show a halt in a rise of global temperatures from about 1960, but is contradicted by other evidence which appears to show a rise in temperatures is continuing.

Jones wrote that, in compiling new data, he had “just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (i.e., from 1981 onwards) and from 1961 for Keith’s to hide the decline,” according to a leaked e-mail, which the author confirmed was genuine.

Dr. Jones responded.

However, Jones denied manipulating evidence and insisted his comment had been taken out of context. “The word ‘trick’ was used here colloquially, as in a clever thing to do. It is ludicrous to suggest that it refers to anything untoward,” he said in a statement Saturday.

Ok fine, but how Dr. Jones, do you explain this?

There’s a file of code also in the collection of emails and documents from CRU. A commenter named Neal on climate audit writes:

People are talking about the emails being smoking guns but I find the remarks in the code and the code more of a smoking gun. The code is so hacked around to give predetermined results that it shows the bias of the coder. In other words make the code ignore inconvenient data to show what I want it to show. The code after a quick scan is quite a mess. Anyone with any pride would be to ashamed of to let it out public viewing. As examples [of] bias take a look at the following remarks from the MANN code files:

Here’s the code with the comments left by the programmer:

function mkp2correlation,indts,depts,remts,t,filter=filter,refperiod=refperiod,$

datathresh=datathresh

;

; THIS WORKS WITH REMTS BEING A 2D ARRAY (nseries,ntime) OF MULTIPLE TIMESERIES

; WHOSE INFLUENCE IS TO BE REMOVED. UNFORTUNATELY THE IDL5.4 p_correlate

; FAILS WITH >1 SERIES TO HOLD CONSTANT, SO I HAVE TO REMOVE THEIR INFLUENCE

; FROM BOTH INDTS AND DEPTS USING MULTIPLE LINEAR REGRESSION AND THEN USE THE

; USUAL correlate FUNCTION ON THE RESIDUALS.

;

pro maps12,yrstart,doinfill=doinfill

;

; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions

; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually

; plot past 1960 because these will be artificially adjusted to look closer to

; the real temperatures.

;

and later the same programming comment again in another routine:

;

; Plots (1 at a time) yearly maps of calibrated (PCR-infilled or not) MXD

; reconstructions

; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually

; plot past 1960 because these will be artificially adjusted to look closer to

; the real temperatures.

 

You can claim an email you wrote years ago isn’t accurate saying it was “taken out of context”,  but a programmer making notes in the code does so that he/she can document what the code is actually doing at that stage, so that anyone who looks at it later can figure out why this function doesn’t plot past 1960. In this case, it is not allowing all of the temperature data to be plotted. Growing season data (summer months when the new tree rings are formed) past 1960 is thrown out because “these will be artificially adjusted to look closer to the real temperatures”, which implies some post processing routine.

Spin that, spin it to the moon if you want. I’ll believe programmer notes over the word of somebody who stands to gain from suggesting there’s nothing “untowards” about it.

Either the data tells the story of nature or it does not. Data that has been “artificially adjusted to look closer to the real temperatures” is false data, yielding a false result.

For more details, see Mike’s Nature Trick

UPDATE: By way of verification….

The source files with the comments that are the topic of this thread are in this folder of the FOI2009.zip file

/documents/osborn-tree6/mann/oldprog

in the files

maps12.pro

maps15.pro

maps24.pro

These first two files are dated 1/18/2000, and the map24 file on 11/10/1999 so it fits timeline-wise with Dr. Jones email where he mentions “Mike’s Nature trick” which is dated 11/16/1999, six days later.

UPDATE2: Commenter Eric at the Climate Audit Mirror site writes:

================

From documents\harris-tree\recon_esper.pro:

; Computes regressions on full, high and low pass Esper et al. (2002) series,

; anomalies against full NH temperatures and other series.

; CALIBRATES IT AGAINST THE LAND-ONLY TEMPERATURES NORTH OF 20 N

;

; Specify period over which to compute the regressions (stop in 1960 to avoid

; the decline

;

Note the wording here “avoid the decline” versus “hide the decline” in the famous email.

===============

I’ll give Dr. Jones and CRU  the benefit of the doubt, maybe these are not “untowards” issues, but these things scream for rational explanations. Having transparency and being able to replicate all this years ago would have gone a long way towards either correcting problems and/or assuaging concerns.


Sponsored IT training links:

Need help for EX0-101 exam ? We offer self study 642-436 training program for all your 642-974 exam needs.


The climate data they don't want you to find — free, to your inbox.
Join readers who get 5–8 new articles daily — no algorithms, no shadow bans.
0 0 votes
Article Rating
480 Comments
Inline Feedbacks
View all comments
kwik
November 26, 2009 9:10 am

Hello,
I think the politicians needs to see this video here first, before spending any more taxpayers money on anything that has to do with research;
http://www.ted.com/talks/hans_rosling_shows_the_best_stats_you_ve_ever_seen.html

pope john
November 26, 2009 1:11 pm

It is difficult to get a man to understand something when his salary depends upon his not understanding it.
A trial without a defense is a sham
Business without competition is a monopoly
Science without debate is propaganda.
The most telling point is that after spending $30 billion on pure science research no one is able to point to a single piece of empirical evidence that man-made carbon dioxide has a significant effect on the global climate.

tonydej
November 27, 2009 3:58 am

Any suggestions where I should register outrage? Petitions, web-sites etc.
I see AL Gore is on the front page of the FT, saying ‘there is no more place for short-termism’ or such like. Such hucksterism: Rush me my planet-saving kit now… I agree to pay over 24 years etc. etc.

November 27, 2009 12:15 pm

Looking at comments is hardly code analysis.
IIRC, there was a comment in the patches for one of the DEC disk drivers that asked: “What do you get if you multiply 6 X 9?” The answer, of course is 42 (as any Hitchhiker’s Guide aficionado knows). But it matters not what the comments say; you want to show that something untoward is happening, show it IN THE CODE. The comments are meaningless by themselves.
Cheers,

Jack Enright
November 27, 2009 10:13 pm

I don’t know about programming; I can barely cope with using a PC. The only thing I know about climate is that it changes.
Sadly, though, at 62 I’ve met a whole bunch of frauds in my time, and I’ve got pretty good at spotting them. One of the biggest giveaways is when, if somebody starts asking detailed questions, they get evasive, then huffy, then downright hostile – and then start character assassination behind the questioner’s back.
Also, I’ve worked in enough universities to know that ANYONE who submits a piece of work in any discipline, even a first year student, has to reference sources of data or quotations (in an approved format), and methods of working, so that anyone else can check back on their work and verify it. Failure to do so will result in no marks for that work. UEA have not only allowed one of their academics to publish work in their name which failed to meet the above conditions, but have turned a blind eye while he broke the law, by deleting files to prevent them being accessed under the Freedom of Information Act.
Jones, and his supporters, can say what they like about the e-mails, the comments, and the software; in my opinion, those are side issues – though, I grant you, serious ones. But the fact that he has deliberately refused other academics access to his raw data and methodology so that they can independently verify his work damns him professionally. And the fact that UEA has allowed him to do so, whilst publishing work in their name, means they have no right to any academic credence either.
Compare his attitude with that of an archaeologist, when told that some perfectly formed wooden javelins had been found in a peat bog in Germany – and had been carbon dated at 140,000 years old (long before homo sapiens, and long before anyone suspected that manufactured tools were in use). He said;
“This is the most exciting thing that’s ever happened to me since I started studying archaeology! It means that everything I’ve learnt and been taught is ALL thrown out the window – and we have to start all over again!”

tonydej
November 28, 2009 2:10 pm

Here are two links to register dissent, both ‘official’ for the UK.
http://petitions.number10.gov.uk/UEACRU/
http://www.sciencemuseum.org.uk/proveit.aspx

layman
November 30, 2009 8:27 pm

I’m not much of a scientist and and even worse programmer so I try to avoid debates on both subjects as much as possible.
What appears to be one of the most galling aspects of the global warming crowd is their immediate and vicious attacks at the scientific credentials of those who remain skeptical, yet appear to marinate in their adoration of al gore and or anyone else who shares their view. Last I checked, Mr. Gore’s scientific credentials are about as feeble as they get. Simply reading up on subjects and shouting them louder than everyone else in the room typically makes one appear as little more than a drunken didact. Gore evidently has touched on the right subject but then he did major in government and took a shot a law (without much success) so while his science may be lacking his ability to manipulate a heathen throng appears second to none.
IN any case, the Climate change crowd (or GW, or AGW, or ACC if you prefer) seem to rely on the pejorative “denialist” (is that even a word?) and hissily insist that no-one is capable of looking at the data and infering anything of significance. Presumably hoping to imply that while they have the required knowledge to divine the truth in the data, no one else should try lest they hurt their tiny brains, or worse have the temerity to disagree with them. A political tactic if ever there was one so I guess Gore’s students are absorbing the right messages.
When one looks at that quality of the debate it’s not hard to come away with the impression that it is the climate change folks who are doing the most damage. Possibly based on their seemingly unshakeable belief that every single person who disagrees with them is either a loon, and idiot, or funded by Exxon. This belief is evidently acceptable justifcation to charge into any old discussion braying like a mad cow and insulting everyone within earshot.

December 1, 2009 4:55 pm

I reviewed the posted bits of “data” where the comments refer to “negative values”…
I was shocked. Shocked that a supposed scientist and “programmer” wouldn’t recognize this immediately. It’s a floating point overflow. Simple. Basic. Rookie error. I don’t know what the bit field length is, but..it’s simple. Square an number that fits in 16 bits…and if the result is too large to fit in 16 bits…voila !!!… you get a negative number…and consequently…you should have gotten an overflow error as well. This was basic, Programming 101 material. G-d help us all. I’m glad these guys are in the weather industry and not launching men into space.

Andrew Sykes
December 2, 2009 2:40 pm

What isn’t always apparent with computer-based climate modelling is that it’s still a developing science. No one’s claiming all the kinks have been worked out, merely that the overall direction of the research points conclusively towards humans having an impact on climate. The upshot of this is that data may seem to be altered/kept out/changed to fit expected outcomes not as part of some bizarre conspiracy but as part of good, well supported and mostly transparent science. It’s not perfect, but it ain’t wrong either.

kwik
December 2, 2009 8:49 pm

It seems to me that many Scientists working for the Government has a very twisted view of the private sector.
Someone working for the private sector cannot really be trusted. And yet, within Climate Science (some of them) are doing the very thing they themself claim the private sector is doing.
Hiding data. Hiding algorytms.
Well, there are honest Climate Scientists too. Like those at Alabama University doing Satelite measurements. They put their raw data on the Web, for anyone to plot them.
But then again, they are not claiming that we are on a tipping point, and that we are all doomed, unless we all subdue ourselves under a planetary Goverment, controlled by a very,very small elite. On the contrary. John Christy, which to me seems to be the arch-type of an honest scientist, says that this CO2 forcing factor is plain wrong. That data from the real world is indicating the very opposite.
So does Dr. Lindzen say. And he is rooted in the real world too. Not in the xbox 360 science.
The CRU-crew should have put all the raw data, collected using our tax-money, on the web.
And they should have put the specifications for out-sourced state of the art software weighting all these data into understandable plots, on the web. And the source code, for the core-software doing this weighting, so it could be inspected.
If they had had more contact with the private sector, and the technology advances going on within software in a free marked economy, this would have been in place by now.
Unfortunately, the seem to be on another planet than us, with some tipping point just around the corner. With one leader, one people, one Government. Have you seen it before?

KevinUK
December 3, 2009 2:21 am

Andrew Sykes (14:40:45) :
Atthe risk of being snipped for questioning your motivation, you rpost looks like it’s been made by someone who is a climate modellers trying to defend the indefensible. If you are a climate modeller or are part of the process (you use the output from climate models to predict when the ‘tipping point’ will come) ten you will know full well that the parameters built into the (parameterised functions in the) climate models are specifically chosen to show significant (more often than not dangerous/catastrophic warming) in the future.
The assumptions built into these climate models of significant net positive feedback are rarely questioned by those who are funded to produce these ‘the sky will collapse’ in xxxx predictions. And I’ve yet to see an real laboratory based experiments (with one exception) which justify the values derived for some of the key claimed net signiicant positive feedback parameterisations e.g. the effects of clouds. That one exception is Svensmark’s work and the soon to start CERN CLOUD experiments.
How therefore justify your statement that
“merely that the overall direction of the research points conclusively towards humans having an impact on climate. The upshot of this is that data may seem to be altered/kept out/changed to fit expected outcomes not as part of some bizarre conspiracy but as part of good, well supported and mostly transparent science.”
The models aren’t anywhere near at a stage of their development where they can ‘point conclusively towards humans having an impact on climate.’ and that climate modelling is ‘part of good, well supported and mostly transparent science’. I’d agree with your well-supported bitthough as sadly likely many in theUK have little choice in funding this well-funded (by me and other UK taxpayers) pseudo-science.
KevinUK

Ian
December 5, 2009 3:23 am

When they speak of not plotting data beyond 1960 because of the divergence problem you have to understand what is meant by the divergence problem.
The divergence problem is that tree ring data beyond 1960 does not show the expected increase in temperature. In fact the tree ring data would indicate that temperature had declined. This is the decline that gets frequent mention elsewhere.
If tree ring data is a good proxy for temperature, that would mean the temperature hadn’t increased since 1960. This contradicts the carefully (re)constructed temperature record from land based measurements. On the other hand if tree ring data isn’t a good proxy for temperature now then why should we think it was ever a good proxy for temperature in the past.
The honest thing to do in this situation is to plot all the tree ring data. A scientist shouldn’t only publish results which confirm their expectations. The approach at the CRU seems to have been to either throw away the data after 1960 on the grounds that it was contaminated by this mysterious `divergence problem’, or worse still to compensate for the divergence problem using a fudge factor.
There is really nothing wrong with that tree ring data. They don’t plot it or use it because it is ‘climatically incorrect’.

Jandui
December 6, 2009 7:56 am

its remembering me someone: Adolf Hittler? I think so…

Kevin007
December 9, 2009 6:44 am

an excerpt from one November 1999 e-mail authored by Phil Jones reads:
“I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie, from 1981 onwards) and from 1961 for Keith’s to hide the decline.”
Some of you climate denialists cite this sentence as evidence that temperature statistics are being manipulated. Several scientific sources have said that the decline being referred to is a decline in tree ring metrics, not temperature. RealClimate characterizes the e-mail excerpt as follows:
The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the ‘trick’ is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term “trick” to refer to a “a good way to deal with a problem”, rather than something that is “secret”, and so there is nothing problematic in this at all. As for the ‘decline’, it is well known that Keith Briffa’s maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the “divergence problem”–see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommended not using the post-1960 part of their reconstruction, and so while ‘hiding’ is probably a poor choice of words (since it is ‘hidden’ in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.
The “trick” is simply a concise way of showing the two kinds of data together while still clearly indicating which was which and has denied that there was anything “hidden or inappropriate” about it. His method of combining proxy data had been corroborated by numerous statistical tests and matched thermometer readings taken over the past 150 years

Ian W
December 9, 2009 8:08 am

Kevin007 (06:44:37) : yet again repeats the ““trick” is simply a concise way of showing the two kinds of data together while still clearly indicating which was which and has denied that there was anything “hidden or inappropriate” about it. His method of combining proxy data had been corroborated by numerous statistical tests and matched thermometer readings taken over the past 150 years”
The problem is that it did NOT match thermometer readings and that was what was hidden. If they didn’t hide that it didn’t match then the entire collated data set would be questioned. It would be remarkable if they managed to get tree rings to match temperatures – as they don’t – well not without some repeated manipulation of the figures. Biffra actually admits this in one of his papers. And you are aware that botanists do not agree that tree rings indicate temperature alone indeed recent research has shown they appear to correlate to levels of cosmic rays.
As you are obviously a statistician you can also tell us the statistical significance of the tree rings of the single tree on the Yamal peninsula in Siberia that drove Biffra’s results on _global_ temperatures.

Hideshi Nomura
December 12, 2009 6:16 pm

May the Physics be with you.

unbound
December 26, 2009 10:56 am

Lovely…you want a rational answer, but you seem to refuse to actually look for that answer (or possibly refuse to accept it).
The problem is how to deal with tree rings that are no longer consistent in size due to the increased carbon emissions in the atmosphere. The “trick” to account for the messed up tree rings after 1960 so they could see if trending after correction to the tree rings would continue to line up with the recorded temperatures.
Don’t act like you are trying to find the truth if you are not actually going to look for it.

Admin
December 26, 2009 11:12 am

sigh.
The problem is misleading policy makers and the public as to the certainty of the reconstructions and the reliability of these tree-ring based temperature reconstructions.
Unbound:
How do you know ” with tree rings that are no longer consistent in size due to the increased carbon emissions in the atmosphere. “?
Your statement quoted is just a made up arm waving dismissal of data to reach a preconceived conclusion. There is no way to know that long term treemometers ever existed in the first place, except for some apparently cherry picked data correlations.

Ian W
December 26, 2009 1:43 pm

I realize that ‘Climatologists’ wanted to find a proxy for temperature – so they took tree rings. Perhaps if they had briefly spoken to a botanist – someone who actually knows about plant growth – even the university gardener – they would have been told ( and they were told by several unasked) that tree rings are as reliable an indicator of past temperature as reading tea leaves. Far to many variables affect plant growth, all that can be said about a ‘thick year’ is that the tree grew well.
This became very apparent when the proxy temperature guesses from the tree rings went down after 1960 when the actual temperatures were going up. This actually showed that the proxies were invalid. If they were invalid after 1960 then they were invalid before 1960 despite possible short term correlations. Thus all the reasoning based on THE tree at Yamal etc was based on invalid assumptions. The entire argument using tree rings was baseless. Instead of being honest about this – ‘the team’ hid this validation failure.
So the validation failure exposed and compounded a larger ethics failure in University of East Anglia.

David
December 26, 2009 4:25 pm

Local weather is not global climate. The temperatures at Yamal could have declined after 1960 as they have in many places. We don’t use one thermometer to track global weather and neither should a single stand of trees be used. BAD SCIENCE!

December 26, 2009 8:43 pm

So, Ian, once they consulted with the botanists and determined that they could filter out static from other causes and use the rings as an approximate proxy for thermometers, and the botanists told them that tree rings are accurate indicators of past growth much, much more accurate than your own tea leaf reading, what then? What sort of hubris makes you think there were no such consultations?
You can’t be bothered to read the papers and the science?
David, how many different sites measured does it take to equal good science on the issue? How do you know? What sort of hubris was it that caused you to think these scientists didn’t bother to get more than one location?
You can’t be bothered to read the papers, either?

Ian W
December 27, 2009 1:04 am

Ed
If you read the CRU emails you will see that they _were_ told that there were far too many variables affecting tree growth rate for it just to be temperature. For example a hot dry year will result in less growth than a temperate wet year, a pest or fungus infestation will result in less growth. This is not ‘noise’ it is that many variables affect tree growth and use for just temperature is unsafe. Especially when the claims are for accuracy to better than plus or minus a few degrees
If you must use a proxy for a variable then your research and results should also include totally open validation of that proxy to prove that it does act as a reliable proxy.
This was not done. Indeed when the unreliability of the proxy became apparent – everyone involved – agreed to hide it. Now you may have different ethics to the rest of the scientific community – but most scientists report their results openly even if they are unfavorable to their research. Research is intended to increase knowledge, not prove preconceptions.
so Ed, climate research is based on proxies. Do you have a problem with validating these proxies as correct and reporting the validation carried out their level of accuracy or inaccuracy, and any ‘adjustments’ as part of that research?
I ask as it is notable that the IPCC committee (at the meeting in Tanzania) and University of East Anglia and Penn State University appear to be of the opinion that you should hide proxy validation failures and original data.
Perhaps scientific ethics have changed?

December 27, 2009 7:27 pm

If you read the CRU emails you will see that they _were_ told that there were far too many variables affecting tree growth rate for it just to be temperature. For example a hot dry year will result in less growth than a temperate wet year, a pest or fungus infestation will result in less growth. This is not ‘noise’ it is that many variables affect tree growth and use for just temperature is unsafe. Especially when the claims are for accuracy to better than plus or minus a few degrees

And if you read the papers you’ll see that those problems were taken into consideration. No one argues that tree rings substitute for thermometer readings, but as a tool to corroborate other measures, they work just fine. The original allegation here was that the dendrochronologists don’t know botany. That’s demonstrably wrong, and the papers do consider the stuff you claim they don’t consider. Read the papers.

If you must use a proxy for a variable then your research and results should also include totally open validation of that proxy to prove that it does act as a reliable proxy.

The research is totally open. Read the papers.
You’re complaining about a chart based off the research papers. The chart was designed to show how dendrochronological data correlate and corroborate other data dealing with global warming. As with many other measures in many other areas — carbon dating has to deal with increased carbon after about 1850, too, for one example — there are variations in the correlation of the proxy measure over time. These variations are well known, not secret, and in this case have been subject to more than a decade of rather vigorous public debate about how to deal with them.
Anyone who reads the literature knows that the dendro data depart from the general trend in the 1950s or 1960s. We don’t know why — it may be, as with the carbon dating data, that the increased amount of carbon has something to do with it. Or it may be that acid rain predominates the effects shown in dendro data at that time, which is about the same time acid rains started to become a major factor. There is a wide variety of potential causes. More research may tease out which is the ultimate cause, or which are the causes.
But in any case, we know that the tree-ring data stop being a good proxy after about 1960. So, for the purposes of this one chart, real temperature readings were substituted.
In other words, known-to-be-accurate data were substituted for known-to-be-inaccurate data.
I’d love to hear one of the warming skeptics explain why actual data should be considered “fraud” in this case. In the real world, that’s a good move, to put in more accurate data in a chart. In any case, this was the substitute data.
If the proxy data are not accurate for the years prior to 1960, the inaccuracies should be explained in the discussions in the papers prior to 1998. Can you point me to a paper that says the data don’t work at all? I’ve not found it. In general it’s fair to say that the proxy data are well known to be well correlative to temperature readings in places and times we couldn’t have temperature readings, with the qualifications well identified in the papers.

This was not done.

Seriously? What do you call the debate over the previous decade? Your argument that we should have used inaccurate data is specious from the start, and here you claim that the difficulties with the proxy data were not known — when your only sources of information on the difficulties are the research papers already on the record identifying those problems.
This shouldn’t be so convoluted. Are the data inaccurate? Can you show us where and how?

Indeed when the unreliability of the proxy became apparent – everyone involved – agreed to hide it.

That’s a false statement on your part. The unreliability of the proxy applied only after 1960, and it’s well discussed in the papers. None of those papers was pulled. All of them remain in public, in research journals.
Your failure to study the issue is not “hiding the data” on the part of the people who did the work.

Now you may have different ethics to the rest of the scientific community – but most scientists report their results openly even if they are unfavorable to their research. Research is intended to increase knowledge, not prove preconceptions.

You may not subscribe to science ethics which requires an accurate summary of the data one criticizes. But that doesn’t affect he accuracy of the overall data. It only means that trying to figure out what you’re saying will be difficult for laymen — you’ve hidden the facts. Odd, isn’t it? You’re doing what you falsely accuse the scientists of doing, and you’re crowing that your ethics are superior.
Research can increase knowledge only if people read it.

so Ed, climate research is based on proxies.

That’s false. Among the more famous non-proxy research, climate research is based on measurements of CO2 on the mountains of Hawaii over the past 60 years, on temperature readings from various science agencies kept over the last 400 years, on the coming of spring, on the zones in which plants grow, on the first freezes and last freezes of a year, on the extent of glaciers as measured during the past 300 years, and a variety of other non-proxy data.
If you were laboring under the misconception that climate data is proxy data, or mostly proxy data, or significantly proxy data, we’ve identified a major source of error for your other observations.

Do you have a problem with validating these proxies as correct and reporting the validation carried out their level of accuracy or inaccuracy, and any ‘adjustments’ as part of that research?

I have no difficulty in using proxies for the purposes they are intended, so far as they tend towards accuracy we can use. In this case, dendrochronology data are used to substitute for actual temperature measurements in times and places we don’t have actual measurements. The proxies are calibrated against real measurements. Adjustments for accuracy are par for the course, to be expected, and not to be fought tooth and nail by people who wish for accuracy.
What is your real question, I cannot tell. If you’re asking whether the proxy data corroborate other measures that show super-warming, the answer is yes. If you’re asking whether proxy data drive climate research, the answer is no.

I ask as it is notable that the IPCC committee (at the meeting in Tanzania) and University of East Anglia and Penn State University appear to be of the opinion that you should hide proxy validation failures and original data.

That’s a false claim.

Perhaps scientific ethics have changed?

Nope, still much stiffer than blog ethics.
REPLY: Ethics or not, they still felt the compelling need to “hide the decline” by substituting one set of data for another, and that’s wrong. It’s like telling the US population that the dollar is strong by splicing the recent values of the Yen onto the value of the dollar data after it started declining and showing it all in one graph, saying “see, the dollar is robust!”. That doesn’t fly, and neither should mixing proxy and real data. It’s just plain wrong and was only done to strengthen their point when the data nature provided went the way they didn’t want it to.
All explained in detail with supporting evidence here: http://climateaudit.org/2009/12/10/ipcc-and-the-trick/
-Anthony

Ian W
December 28, 2009 1:25 am

Climate research IS based on proxies.
There are relatively accurate actual measures for temperature and CO2 going back to the 1800s before that there are historical records and such things as ships logs that can give relatively reliable wind records but once you are back to say the 1600’s then you are left with proxies.
The ENTIRE AGW hypothesis is that temperatures are rising at an exceptional rate and CO2 is at an exceptional level. To do this the it is necessary to know the normal so one can claim the current climate behavior is exceptional. The ‘normal’ climate behavior is only available through proxies and some historic records. For the AGW hypothesis to be accepted the proxies needed to show that the known warm periods such as the Medieval Warm Period and the Roman Optimum were not actually as warm as today. This I believe is what drove the proxy research. Even historic ‘global’ CO2 is based on ice core data the problem is (as the satellite imagery shows) that the CO2 concentration at the poles is significantly lower than elsewhere on the Earth and CO2 also diffuses through ice over time this results in an unnaturally low ‘historic global CO2 from ice cores’ it is unsurprising that historic ice core records show low CO2. What IS surprising is the disregard of actual measures of CO2 taken in the 1800s that show far higher CO2 concentrations.
Climate research is not weather research it is based on multi-century timescales. We have the ability to measure things from satellite now that is only 40 years old at most. But climatology studies this entire inter-glacial and other glacial and inter-glacials and have to rely n proxies. It is time that an independent validation exercise/audit was carried out on all these proxies as peer review is insufficient. A proxy that departs from actual measures in current times may well have similarly departed in previous times and is therefore unsafe.
Anthony has answered the issue of concealing proxies that fail validation.

kwik
December 28, 2009 7:51 am

I think this finnish documentary gives a pretty good overview;
Seems station-data is important now;
Climategate på finsk TV gir en bra oversikt
kwik skrev for 0 minutter siden:
Vi ser ikke stort om Climategate på norsk TV.
De prøver nok å holde lokket på så lenge som mulig.
Men her er 3 deler på Youtube fra en finsk film om Climategate;

http://www.youtube.com/watch?v=Clpmt5_8MBg&feature=related