McIntyre: The deleted data from the "Hide the Decline" trick

By Steve McIntyre from his camirror.wordpress.com site.

For the very first time, the Climategate Letters “archived” the deleted portion of the Briffa MXD reconstruction of “Hide the Decline” fame – see here. Gavin Schmidt claimed that the decline had been “hidden in plain sight” (see here. ). This isn’t true.

The post-1960 data was deleted from the archived version of this reconstruction at NOAA here and not shown in the corresponding figure in Briffa et al 2001. Nor was the decline shown in the IPCC 2001 graph, one that Mann, Jones, Briffa, Folland and Karl were working in the two weeks prior to the “trick” email (or for that matter in the IPCC 2007 graph, an issue that I’ll return to.)

A retrieval script follows.

For now, here is a graphic showing the deleted data in red.

Figure 1. Two versions of Briffa MXD reconstruction, showing archived and climategate versions.shown below, clearly does not show the decline in the Briffa MXD reconstruction.

Contrary to Gavin Schmidt’s claim that the decline is “hidden in plain sight”, the inconvenient data has simply been deleted.

The reason, as explained on Sep 22, 1999 by Michael Mann to coauthors in 938018124.txt, was to avoid giving “fodder to the skeptics”. Reasonable people might well disagree with Gavin Schmidt as to whether this is a “a good way to deal with a problem” or simply a trick.

Figure 2. IPCC 2001 Fig 2.21 showing Briffa, Jones and Mann reconstructions together with HadCRU temperature.

Retrieval script:

##COMPARE ARCHIVED BRIFFA VERSION TO CLIMATEGATE VERSION
#1. LOAD ARcHIVED DATA
url<-"ftp://ftp.ncdc.noaa.gov/pub/data/paleo/treering/reconstructions/n_hem_temp/briffa2001jgr3.txt"

 #readLines(url)[1:50]

 Briffa<-read.table(url,skip=24,fill=TRUE)

 Briffa[Briffa< -900]=NA

 dimnames(Briffa)[[2]]<-c("year","Jones98","MBH99","Briffa01","Briffa00","Overpeck97","Crowley00","CRU99")

 sapply(Briffa, function(x) range( Briffa$year[!is.na(x)]) )

 #      year Jones98 MBH99 Briffa01 Briffa00 Overpeck97 Crowley00 CRU99

 #[1,] 1000    1000  1000     1402     1000       1600      1000  1871

 #[2,] 1999    1991  1980     1960     1993       1990      1987  1997

 Briffa= ts(Briffa,start=1000)
#2. LOAD CLIMATEGATE VERSION

 loc="http://www.eastangliaemails.com/emails.php?eid=146&filename=939154709.txt"

 working=readLines(loc,n=1994-1401+104)

 working=working[105:length(working)]

 x=substr(working,1,14)

 writeLines(x,"temp.dat")

 gate=read.table("temp.dat")

 gate=ts(gate[,2],start=gate[1,1])
#Comparison

 briffa=ts.union(archive= Briffa[,"Briffa01"],gate )

 briffa=window(briffa,start=1402,end=1994) #

 plot.ts(briffa)
X=briffa
par(mar=c(2.5,3,2,1))

 plot( c(time(X)),X[,1],col=col.ipcc,lwd=2,ylim=c(-1.2,.5),yaxs="i",type="n",axes=FALSE,xlab="",ylab="")

 for( i in 2:1) lines( c(time(X)),X[,i],col=i,lwd=1)

 axis(side=1,tck=.025)

 labels0=seq(-1,1,.1);labels0[is.na(match(seq(-1,1,.1),seq(-1,1,.5)))]=""

 axis(side=2,at=seq(-1,1,.1),labels=labels0,tck=.025,las=1)

 axis(side=4,at=seq(-1,1,.1),labels=labels0,tck=.025)

 box()

 abline(h=0)

 title("Hide the Decline")

 legend("topleft",fill=2:1,legend=c("Deleted","Archived"))

Sponsored IT training links:

Using 70-646 virtual exams, you’ll pass your 350-030 exam on first try plus get free demos for next 640-822 exam.


Advertisements

  Subscribe  
newest oldest most voted
Notify of

Note to US readers, here in UK the warmists are still running interference in the MSM. Check out (ex-commie) Aaranovitch in The Times.
sample:
“They somehow believe that the whole global warming schtick is an amazing confidence trick performed upon the peoples of the world by a group of scientists and socialists, and pursued by politicans keen to get their hands on green taxes (though for what nefarious purpose we do not know), and which has taken in almost all the governments of the world, from the US to China.”
Note the use of the word ‘trick’.
Keep up the pressure guys n gals. Don’t let them get away with it.
(PS – great work so far – props to you all)

Henry chance

It is all clearing up. We have several cheats working together and a handfull are in agreement with the cheating.
No wonder Hansen, Jones and Mann attack and mess with McIntyre when he wanted data.

SandyInDerby

Just watching “Question Time” on BBC, and the second question is “Is global warming a scam?”
The first person answering was a sceptic,
Mind you Marcus Brigstock than well known scientist is a zealot of warmist, UEA – CRU is insignificant.
At least it is the open.

So either 70ties were as cold as deep LIA – or trees are bad thermometers, probably underestimating real temperatures.
Hide the declineeeeeee – http://www.youtube.com/watch?v=nEiLgbBGKVk

Joe A

Time for criminal charges to be placed on the appropriate scientists. The sooner this happens the sooner we can get to the bottom of all this.

SABR Matt

There is another problem with their graph…they preferentially adjusted all of the data DOWN in the 1500s and 1600s…look at the reconstruction…I see a lot of years with very similar anomalies to the 20th century.

ET

The wiki “Climategate” page needs attention. It is being redirected to a locked page called “Climatic Research Unit e-mail hacking incident”.

Tom Bolger

The scientists posting the emails are the IPPC experts.
(They get plenty of money and fame from it.)
These scientists plot (see emails) put their theory
to the IPPC and public and goverments follow.

What is unbelievable to me is the miss direction used by the offenders. They say things like “These e-mails will ultimatly show the integrity of the scientists”, hoping that people will not read the e-mails, but will just think “ya, he is right, some may look bad, but in the end it is all good”.
Then there is the notion that this is only 1 or 2 pieces of evidence out of hundreds. What they don’t realize is that most of this data is re-used in many other studys, and that realy this is the only quantitative evidence. The rest (arctic ice etc) is all qualitative, and could be interpreted many different ways.

Every time I look at such graphs, it strikes me that manipulators thrive where anecdotal and academic history go missing.
Though born in ’49, I learned from parents and grandparents about weather conditions between the wars. Apart from a brief but serious spell of drought around 1960, it was clear that the earlier period had been hotter and drier. The dustbowls in the US seemed to co-incide with Aussie conditions. Anecdotal, yes, but deeply ingrained.
As to accepting the absence of Medieval Warming in Mann’s original graph…you’d just have to be dumber than dog’s-do and more ignorant than a highly paid news-anchor.

Sorry guys, being a mere mortal…what does this all mean (well, not the deleted data bit, but the data itself)?
Regards
Mailman

Tim S.

From the band “Toto”, 1979, I give you the satirized version of the song “Hold the Line”:
Hold the decline
Warming isn’t always on time
Oh, oh, oh
Hold the decline
Warming isn’t always on time
Oh, oh, oh
Somebody please make a YouTube music video of this! 😀

Hoi Polloi

I wait for the first of the Climatatii to step out of the closet….

Splice

Mailman it means tree rings are not robust proxy thermometers. The hockey stick shaft is a nonsense

Vg

I presume this is the beginning of analysis of the DATA files (not emails)?. This is where AGW will be buried and legal action will ensue. Expect to see massive resignation within weeks.

chillybean

Maybe when the ‘world climate alarmist leaders’ arrive back from Copenhagen we should just take them straight to a specially constructed climate crimes unit and that can be the new reality TV. Milliband comes to mind, daddy was a Marxist after all.
Adolphe (Ralph) Miliband (7 January 1924 – 21 May 1994) was a notable Marxist political theorist. He was the father of two British MPs.
David Wright Miliband Member of Parliament, who is/was the current Secretary of State for Foreign and Commonwealth Affairs and Member of Parliament for the constituency of South Shields ….
and
Edward Samuel Miliband is a United Kingdom Labour Party politician. He is/was the Member of Parliament for Doncaster North and the Secretary of State for Energy and Climate Change….
Both are part of the Brown Ministry of deception on climate change.

hotrod

I think the wiki article is actually quite balanced given how coverage of climate science in wiki has been subject to manipulation itself by immediate “corrections” of updates to critical articles.
http://en.wikipedia.org/wiki/Climategate
Larry

Ron de Haan

I am in total agreement with Finbar (14:55:39) :
“Don’t let them get away with it”.
But that’s not all.
From John Holdren to Al Gore who will be in Copenhagen together with Obama.
Don’t let them get the away with it and don’t allow them to make further legal commitments based of crooked and corrupt data. Stop Copenhagen.

SandyInDerby

Re Question Time
Should be on BBC iPlayer soon. Second question in, worth watching for those who can access the iPlayer. The person who raised the question gave a good account of the facts – Roman warm period etc. Also Melanie Philips of the Daily Mail nailed her colours to the skeptic mast.
Has to be said most of the audience were converts to the church of warming.

John Cooke

SandyInDerby (15:06:44)
I also watched this item in BBC’s Question Time.
Melanie Philips was given a reasonable chance, but a promise she could come back with a response to a point which basically required “weather is not climate” didn’t materialise.
David Davis was also reasonable, stating that the science isn’t final and never will be, though admitting his opinion to be 80% confident in the AGW idea.
The argument that “most scientists” agree with the AGW view was again pushed. I think this is one area where there needs to be more clarity out there, along the following lines:
(a) Just how big is the climate science community? What percentage of this is involved in the IPCC papers (and Climategate)?
(b) Most scientists are not climate scientists. Before I retired I was a physicist. However I have been pretty dismayed by some of the data handling that appears to have been going on in the climate science community from what has been revealed recently (from Yamal to Climategate). I used to think that climate science was being done properly – now I am far from convinced, to say the least. I hope that most of it is – but most scientists don’t have the time to get into another field sufficiently well to be able to be sure; they expect other scientists to be working honestly. At least that’s how it used to be!
(c) The “peer review” idea needs to be explained – especially if the relevant community is as small as it appears to be. If it is really the case that a very small group has been peer reviewing all this stuff then of course we can expect problems.
Still, at least there’s a discussion with views on both sides on the “Question Time” follow up discussion on the web:
http://newsforums.bbc.co.uk/nol/thread.jspa?forumID=7278&edition=1&ttl=20091126232705

Spartan79

Tim S.: is the what you’re looking for?

By Minnesotans for global warning, posted at Air Vent.
REPLY: on our main page too -A

Heidi Deklein

Tim S – after all the talk about “true believers”, maybe something more like…
I thought science was nothing more than fairy tales
Meant for someone else but not for me
Facts were out to get me
That’s the way it seemed
Disappointment haunted all my dreams.
Then I made this trace
Now I’m a believer
Not a trace of doubt in my mind
Hockey stick – ooh!
I’m a believer, IPCC-er and I lie.
Who thought data had to be a static thing?
Seems the more I changed the less I plot .
What’s the use in tryin’?
Just fix the maths again.
Medieval sunshine is now rain.
And I saw my trace
Now I’m a believer
Not a trace of doubt in my mind
Hockey stick – ooh!
I’m a believer, IPCC-er and I lie. 🙂

Vg

I wonder if in fact RC has basically closed down? re Thanksgiving etc. I mean it would be hard for G Schidmt to continue in public given this and CA re hide the decline posting? I suspect there getting the feeling the game is up with the data analysis JUST starting…. I would tremble knowing CA is going to have a look at my data and analysis LOL….

Keith Minto

I just fear that Post Modernism and Relativism plays a large part in this saga. I blame the Universities for disseminating this tripe, it really means that they feel immune from criticism, producing the “my reality is as as good as your reality” attitude.
This is far, far, removed from the spirit of scientific enquiry, it is as if we live on two different planets.

James Allison

Joe A (15:12:32) : Time for criminal charges to be placed on the appropriate scientists. The sooner this happens the sooner we can get to the bottom of all this.
As my old man always said Joe “there’s no dignity in haste” and another “revenge is best served cold”.
Is there no wonder Gav and his Team say they refuse to visit this blog as doing so would certainly give them very very bad nightmares however I doubt anybody this side will be losing any sleep over that thought.
Anthony and Steve and all your associates – Unbelievably amazing information coming through each day resulting from your collaboration, combined expertise and diligence.
Thank you

Spartan79

Anthony: Oops! I looked for it on your main page, didn’t see it, but didn’t scroll far enough down.

It appears the period from 1910-1950 appears to be way warmer than the last 20 years based on that grapth. But with a slight upturn from 1980-1990…The bottom appears to be around 1970’s…So 1930’s and 1950’s were .2-.5 warmer than today.
Is that right.
Also to note is around 1650 the height of the little ice age it’s only around .4-.5c cooler than today. I thought the little ice age was 2c colder than today.

chillybean

Spartan79 (15:47:12) :
Tim S.: is the what you’re looking for?
Must admit that I have watched it a few times as it just makes me laugh sooo much.

Ron de Haan

“Climategate” has everybody rethinking global warming. Many are wondering — if leading scientists were tempted to finagle their data, is the evidence for catastrophic climate change weaker than previously thought?
Actually, the evidence was never even evidence.
http://pajamasmedia.com/blog/what-is-%E2%80%94-and-what-isnt-%E2%80%94-evidence-of-global-warming/

yonason

Finbar (14:55:39) :
That’s one of their favorite propaganda techniques, mocking our legitimate concerns as if they are just some nut-ball conspiracy. They tell the truth about themselves, and laugh at it, as if it couldn’t possibly be true. It’s surprising how effective that has been, not just in perverting science, but in politics and business, as well. It’s harder to get away with in engineering, because when the design fails, it’s usually pretty clear who’s fault it is, though that won’t stop a good con artist from blaming the victim anyway. After all, what does he have to lose?

chillybean

Heidi Deklein (15:50:39) :
Now I’m a believer
Not a trace of doubt in my mind
Come on Heidi, get a few friends together and make a Utube vid for all us people that know the song and where you are coming from.
p.s I’ve got a Heidi of my own now.

b_C

Pre-crime …
A staple of the “Human Rights” Industry in, among others, Canada, a *pre-crime* can be committed by utterance of any statements that is “*likely* to expose a person to hatred or contempt”. (Check out the litany of woes Ezra Levant and Mark Steyn have had to endure over the past three or four years.)
How is this any different from the *pre-crimes* committed by the Haldley CRUw and IPCC.
Effectively, by their collective malfeasance, anyone associated with pushing for climate change action resulting in increased taxes, is guilty of a pre-crime … in that the ensuing tax burden is guaranteed to have “likely resulted in the exposure of governments to the hatred and contempt of their taxpayers.”
In Ezra’s immortal words:
Fire.Them.All.

Syl

That graph is actually very frightening. If those proxies really were behaving like thermometers, we’ve been cheated out of 0.3-0.4C of cooling due to dodgy surface records. One could eyeball a continuation of the slow downward overall trend in temps since the Eemian and hello glaciation.
yeah, I know, one set of proxies. But still…

yonason

Finbar (14:55:39) :
P.S.
In fact, Monbiot is using a variant of that technique. Here illustrated by Claude Raines

Very effective, given all those he’s taken in with it.

enough

Climategate reaches White House.
Holdren conspires with Mann and Jones to bash Soon/Baliunas when they write about Hockey Stick validation
From FOIA file
1066337021.txt

There seems to be a consistent missunderstanding by media interviewers about the meaning of Jone’s boast of using Mann’s ‘trick’ ‘to hide the decline.’ Should we worrry about it, I dunno. Too difficult for sound-bite interviews? Perhaps not.
The interviewers take the quote, out of context, to suggest that these guys were trying to hide the recent decline found in global temperature indicators (i.e., no warming since this century).
Whereas, as I understand it:
This is all about the dubious use of proxy data to make recent warming look historically extraordinary. The need to hide the decline since 1960 suggests that the proxy data set used to hide the MWP etc is not a good proxy. If the data was assessed as divergent from measured temperature after 1960 then maybe it is also wrong about 1260, or 1360 when other indicators suggest it was warmer than now. What has been found is clear evidence that these IPCC authors knowingly manipulated the data used as proof of recent extraordinary global warming….and then actively obstructed other scientists requests to check the data and what they had done with it.
A bit more than a sound bite, but, if this is right then, we might want to encourage this correction where possible. Otherwise, we might stand accused of allowing the quote to be missinterpreted out of context.

Glenn

Mailman (15:20:19) :
“Sorry guys, being a mere mortal…what does this all mean (well, not the deleted data bit, but the data itself)?”
Read the legend in figure 2, look it up if you’re interested. You shouldn’t expect an education of a complex topic to be provided everytime someone new asks.
The data itself shows there was temperature data (inferred from reading tree rings – “tree data”) to around 1990, and that the “tree data” plotted on the graph left off around the most recent 30 years worth of that “tree data”. Look at the temperature anomaly in figure 1 at around 1990, about -.3c. That doesn’t show on figure 2.
HTH

steve

‘ HACK ‘ ?? If this is a hack, I’m a climate scientist!

Syl, I agree, but I feel that the 1930s-1950’s where very close to the late 1980-1990’s. Also, I believe that the little ice age was at least 2c cooler than the 1998 peak.
I agree the 1930’s and 1950’s where very very warm across the globe. But looking at snow fall trends has me thinking that the 1980s and 1990 were much alike that time period. I feel its a pattern,,,but we are a middle of a respectable down turn not much unlike the 1960’s and 1970s when they where crying ice age.

tom

I couldn’t help but giggle at the Google ad for solar power associated with this article 🙂

Glenn

Why are people claiming RC is down? Last post by “Dan” on
26 November 2009 at 5:43 PM

DeNihilist

Looks like this is finally breaking in the Canadian mainstream news media
http://www.torontosun.com/comment/columnists/lorrie_goldstein/2009/11/26/11929676-sun.html
Toronto Sun – Lorrie Goldstein – SCATHING!

Curiousgeorge

Mailman (15:20:19) :
Sorry guys, being a mere mortal…what does this all mean (well, not the deleted data bit, but the data itself)?
Regards
Mailman
———————————————————–
Among other things it means that the jury is still out on whether the planet is experiencing a long term warming trend( leaving aside the question of it actually being a bad thing or not ). If it is (getting hotter ), then are people really the cause? If no (to either of the above ), then future behavior of the global society becomes somewhat less clear. Translation: Global political and financial power will change hands yet again. How, and to whom, is undecided at present, but there are many contenders. It may turn out to be ” the devil you know” outcome, or a continuation of the well known ” King of the Hill” game we’ve been playing since we became bi-pedal. My bet is on the latter.
But of the ~ 6.7 billion people who occupy the planet, no more than a few million have any idea what any of it means and even fewer have any real say so in the outcome. The rest of us just live our lives the best we can.

jef

This is maybe a little OT, but yesterday on CSPAN (in the US) they had a big trans-Atlantic conference on health issues surrounding AGW (but originating in the UK, I think). The rhetoric was pretty much that billions are going to die for lack of water, etc.
But I watched a little on the off chance that someone said something about the CRU-gate. Indeed one person did: Richard Horton, the editor of The Lancet.
He was rather full of venom toward those who were trying to make something of this (all timed to disrupt Copenhagen, you see). So I queried the internet on Mr. Horton.
He was the editor when the controversial article on Iraqi war casualties was printed…which I anticipated.
But I didn’t know that he published a rather infamous article of MMR and autism which was influential in reducing the vaccination rate in the UK.
This website is from a reporter from the Sunday Times which (if you click thru) will tell an interesting story.
http://briandeer.com/mmr/richard-horton.htm

royfomr

There still is life in the old girl yet!
Straight after a dismal ‘question time’ on the BBC, that briefly touched on the subject of Global Warming in front of a mostly sullen audience, the programme ‘This Week’ came on the box.
Hosted by the excellent Andrew Neil, the first five minutes really rips into the politics and bad science behind AGW. Thanks messrs Neil and Portillo for that. Thanks also to Ms Abbott who after tentatively mentioning the recent floods in North England was clever enough not to pursue the matter when Mr Neil, puckishly, asked her about the 1954 floods in the same area.
Why do I think the above is on topic?
Firstly, it shows how effective the combination of humour and undoubted mastery of the topic (Neil and Michael) are in countering dissent ( Diane)
SMc, Anthony and many others have this gift in abundance but their talents are known of by too few at the moment. For those of us with less talent but the same aspirations to put Science back on its rightful pillar our role must be to make more aware of the issues permeating climate change.
I just tell acquaintances to Google Climategate- it seems to help.
Secondly, for those who can get ‘This Week’ on the BBC iPlayer the first five minutes will render your keyboard a soggy mess unless you remove all potable beverages first.
You have been warned!

slow to follow

Keith Minto (15:54:42)
Hear, hear!

>>>Sorry guys, being a mere mortal…what does this all mean?
And it also means that if the tree-ring data is not a good proxy for climate temperature, then they should not be using it to delete the Medieval Warming Period from history.
I fact, I wonder how they have derived this tree-ring data in the first place. I have always thought that you should only use tree rings for the last half of a tree’s life. For the first half, until it gets its head above the canopy, the tree rings represent the tree’s struggle for light with its neighbours – totally unconnected with temperature and climate. Only when the tree is mature, should its rings be counted.
Do they do this? I bet they don’t.
.

Geoff Sherrington

Melbourne 11 am Friday 27 Nov. A quick poll has indicated the Australian public are 81% against Emission Trading Tax and 7% are in favour. Yet we have both main political parties pushing for it. What is the modern definition of democracy?

Calvin Ball

They somehow believe that the whole global warming shtick is an amazing confidence trick performed upon the peoples of the world by a group of scientists and socialists, and pursued by politicians keen to get their hands on green taxes (though for what nefarious purpose we do not know), and which has taken in almost all the governments of the world, from the US to China.

Sounds familiar. Aren’t these the same bunch who were hollering from the rooftops that the WMD intelligence that was accepted by all of these countries was a con? Sorry guys, you already played that card.