Wegman responds to USA Today

The plagiarism charge, reported by USA Today, is not the worst part of the indictment of the Wegman report. It is the easiest to understand and report on, and I think that is why USA Today focuses on it.
The Wegman report is purported to be an independent analysis of MBH ’98, when in fact, a lot of it was simply copied from the McKintyre and McKittrick 2005 paper. In addition, its criticism of the non centered proxies used by MBH, which was simply an echo of M&M was incorrect.
Since then a lot of papers on climate reconstruction have been published in the peer reviewed literature and all are shaped like a hockey stick.

Wow, the skeptical community entirely dissected the bs plagiarism claim over a month ago and now the alarmists are bringing it up? Is their attention span as deficient as their critical thinking?

I love the answer to proof that a vital piece of Cliamte science is bunk is to point out the bits that you copied and not the actual sicence, brilliant, next they’ll be just making things up as they go along, oh wait that’s what the models are for.
http://wattsupwiththat.com/2010/11/23/old-climate-models-do-a-bad-job-with-clouds-so-a-new-model-says-warming-must-be-worse/
Here come the trolls. By the way trolls, might be worth reading the book on the right called ‘The hockey Stick illusion’, although I will not hold my breath as it goes against the doctorine. Great book by the way.

[snip – policy, invalid email address]

Wegman doesn’t appear to be favouring the “It’s a report what else do you expect???” defence I see repeated often here. It’s a shame there wasn’t room in the screengrab to include his actual defence
“Wegman said he and his report co-authors felt “some pressure” from a House committee to complete the report “faster than we might like.” But he denied that there was any attempt to tilt the influential climate report politically.”
This suggests that he accepts the plagiarism is real and is not acceptable and is saying it was done to complete the report on the required deadline.
It is useful though that the “Wegman whiners: this post’s for you” handily lists all the previous posts about the issue so we can compare the defence put forth here with Wegman’s.

“He certainly understands John Mashey”. Well, that’s more than I do. This is confusing. Who is accusing whom of plagiarism? Plagiarism of what? Which one is the skeptic, Mashey or Wegman? If you post articles by illiterate journalists, they need an explanation to go with them. Better still, just post the explanation.

Just read that USA Today article:
just few points from this para alone….
-Bradley only objected via Masheys analysis? – so he never read the report himself or if he did he did not ‘instantly’ recognised plagiarism?
– Masheys analysis took a whole freekin year????
– copying without credit? – has anyone read the bibliography?
and finally, what about the USA Today ‘example’ in the link given – it’s pathetic! The WR is a summary report and by definition must use common language and terminology of previous publications. Moreover, in presenting the summary type view – it seems (to me) that Wegman et al are demonstrating that they have (necessarily) had to read and understand the basic premises in order to undertake their mathematical method analysis. I would take the view that the written description (which may well be re-written in a highly similar vein, but with more personal understanding ‘tweaks’) is correctly undertaken and in order to give the background information, cannot be done any other way! I mean, how else do you describe a tree ring, for example?
Honestly, I have never seen so much smoke being generated from such a tiny spark! And that can really be the only explanation – its a smokescreen!

sharper00 says:
November 23, 2010 at 11:26 am
In other words, not even you understand what you are talking about…..

sharper00 says:
November 23, 2010 at 11:21 am

This suggests that he accepts the plagiarism is real and is not acceptable and is saying it was done to complete the report on the required deadline.

No, it suggests he was under pressure to hurry, exactly as it states. YOU suggest otherwise, but nobody in here really cares what YOU suggest, other than perhaps eadler, because you obviously have an axe to grind. You can’t argue the facts (Wegman is correct) so you’d rather argue something immaterial as a distraction.
Mark

We should review for our trollish friends.
Bradley’s name is mentioned in the report 35 times. Here are some of the examples of Wegman’s failure to “cite”, copied from the report….
Featured prominently in the IPCC report was the work of Dr. Michael Mann, Dr. Raymond Bradley,……
Table 1 based on Bradley (1999) illustrates……
Table 2 found in Bradley (1999), which was reproduced from Bradley and Eddy
(1991) summarizes……
After Bradley and Eddy (1991)….
See Bradley (1999) for a discussion……
together with his colleagues Dr. Bradley and Dr. Hughes, continued…..
in previous studies by R.S. Bradley…….
Then in the section titled “BIBLIOGRAPHY Academic Papers and Books”, Bradley is referenced 13 times.
Yeh, he sure tried to pull a fast one there. If this is the best the alarmists can do in form of a rebuttal, the game’s over.

eadler says:
November 23, 2010 at 11:09 am
I have briefly read Wegman, (I confess to skipping the detailed statistical bumpf as far too mind blowing for me!) but I have not noted anywhere in the report where it says that they specifically analysed Manns methods and code, etc. It would be appreciated if you could indicate where such analysis is present.
I did note in the Exec Summary, it specifically states that Mann never provided the full code or data so presumably such an independent analysis wasn’t possible and this seems to be generally mentioned elsewhere, in that they have deconstructed Manns work, along the lines of M&M and confirm M&M’s findings. Or have I got that wrong?

…as in most “Team” efforts, they shoot the messanger
if they can’t kill the message.
At least the folks in Cancun will have something other to
talk about instead of Republicans in the House and drug
lords in the countryside.

Notice how “Maple Leaf” posting is quite discreet now after being outed at Judith Curry’s site…

James Sexton says:
November 23, 2010 at 11:14 am
Wow, the skeptical community entirely dissected the bs plagiarism claim over a month ago and now the alarmists are bringing it up? Is their attention span as deficient as their critical thinking?
===
LOL,
That was my take but they decided on new “Alarmist” Spin.
Let’s see if Congress has “enough” Bug Spray and Sense.

eadler
November 23, 2010 at 11:09 am
Bull!

From what I understand from the various pieces done on this site, it wasn’t a peer reviewed paper anyway, just a simple report, where cut-and-paste is about expected. If it’s just to inform a congressional committee of some information, it doesn’t matter. There wasn’t any science, or independent research or any proofs or anything in it.

I wonder whether Wegman will every agree to release his code? I don’t buy the idea that sworn, supposedly scientific testomony before congress should be held to a lower level than normal academic publications. In the 1990s extensive presentation of raw data and materials was not the standard, but it has been over the last five years or so. At one point, Wegman said that he was planning to an academic publication for his congressional analysis. Why did he seem to give up on that?

sharperoo wrote: “I can accept Mann’s method is flawed but the conclusion is correct.”
Really? What “method” proves his flawed method led to a correct conclusion?

If mother nature isn’t cooperating, obfuscate, obfuscate, obfuscate. They are doing it here on this topic right now. Try not wasting too much breath, my fellow WUWT?er’s. Could sure use some of that GW where I live. It is cold as a witches mammalian protuberance (Anthony runs a clean blog). My post on the last Wegman thread still stands.

Continuing discussion of the plagiarism claim against Wegman seems a very foolish move for the AGW crowd. The true agenda behind this weak complaint has already been revealed in correspondence between Ray Bradley and a third party which reads –
“I filed a complaint with George Mason University (where Wegman is a Professor) & they have set up a committee to investigate my complaint. I[A] recent letter from their Vice-Chancellor indicates that they expect the committee to report their findings by the end of September.
That’s the long & short of it. I have told the University that I am prepared to drop this matter if Wegman makes a request to have his report withdrawn from the Congressional Record. No response on that.
Thanks
Ray [Bradley]”
What is most revealing is that this was a request for the withdrawal of the whole report rather than corrections or additions to references and attribution for material in the report. CA has a thorough discussion of the implications of Bradley’s correspondence here-
http://climateaudit.org/2010/10/21/bradley-tries-to-deal/
However in attacking the Wegman report, the sound and fury that AGW believers are creating could prove very useful. I would love to see the report reworked with special attention to the social networking section. Especially after the Climategate leak showed that the situation was worse than Wegman indicated. I am sure that Mann’s email 31/10/03 which included the line –
“Let’s let our supporters in higher places use our scientific response to push the broader case against M&M.”
-requires further investigation. Just who were these supporters in higher places?

Wegman can easily position this inquiry in a wider context, which will get excellent press for the ill wind toward problematic climate science that is blowing around the country of recent years.
This is a windfall opportunity for increased public awareness of the weakness of IPCC supported consensus climate science.
It is a gift. We keep getting these gifts from the IPCC supported consensus climate science followers. Just a few gift examples are: Climategate itself; No Pressure; Patchy, IAC report on IPCC, and everytime Gore & Mann opens their mouths. : )
Precious gifts.
John

@ eadler and louise
Do you people even read the stuff you guys are throwing around?
From the linked article,“Thus, McIntyre and McKitrick’s original Fig 1-1, mechanically reproduced by Wegman et al, shows a carefully selected “sample” from the top 1% of simulated “hockey sticks”. And Wegman’s Fig 4-4, which falsely claimed to show “hockey sticks” mined from low-order, low-autocorrelation “red noise”, contains another 12 from that same 1%!”
Wow, pretty damning stuff, it appears the writer of this article is stating Wegman copied M&M and then misrepresented the significance of the graphs. Let’s go to the report itself and see what it says about figure 4.4
“Figure 4.4: One of the most compelling illustrations that McIntyre and McKitrick have produced is created by feeding red noise [AR(1) with parameter = 0.2] into the MBH algorithm. The AR(1) process is a stationary process meaning that it should not exhibit any long-term trend. The MBH98 algorithm found ‘hockey stick’ trend in each of the independent replications. then, immediately under the hockeystick graphs,
“Discussion: Because the red noise time series have a correlation of 0.2, some of these time series will turn upwards [or downwards] during the ‘calibration’ period6 and the MBH98 methodology will selectively emphasize these upturning [or downturning] time series.
Yeh, he copied, he said he did! Further, when reading after the word “discussion” we can see that the graphs in 4.4 weren’t implied to be random, but rather illustrative. It states clearly some will turn upward while others turn downwards. All of the graphs shown in 4.4 turn upwards. So, clearly Wegman’s graph weren’t placed to reflect a random selection of the runs. More from section 4
“We have been able to reproduce the results of McIntyre and McKitrick (2005b). While at first the McIntyre code was specific to the file structure of his computer, with his assistance we were able to run the code on our own machines and reproduce and extend some of his results. In Figure 4.1, the top panel displays PC1 simulated using the MBH98 methodology from stationary trendless red noise.”
You guys are something else.

You guys are something else.

They blindly read DC (and the like) and assume he *must* know what he is talking about (he agrees with the consensus and all that) and then reword and repost what he is claiming (ironic, eh?) without understanding the details.
When ideology is involved, a person can make himself believe all sorts of things simply by ignoring anything contrary.
Mark

sharper00 says:
November 23, 2010 at 12:33 pm
‘I can accept Mann’s method is flawed but the conclusion is correct.’
Enough said. Guess the brain washing is working then or as you say do you acept the conclusion even though you know the method is wrong. Sorry Antony but you cannot do anything about people that just cannot see what is wrong with comments like this.
Or WUWT has finally gotten true trolls that only write comments without substance to get a rise.

James Sexton says:
November 23, 2010 at 2:20 pm
@ eadler and louise
Do you people even read the stuff you guys are throwing around?
From the linked article,“Thus, McIntyre and McKitrick’s original Fig 1-1, mechanically reproduced by Wegman et al, shows a carefully selected “sample” from the top 1% of simulated “hockey sticks”. And Wegman’s Fig 4-4, which falsely claimed to show “hockey sticks” mined from low-order, low-autocorrelation “red noise”, contains another 12 from that same 1%!”
Wow, pretty damning stuff, it appears the writer of this article is stating Wegman copied M&M and then misrepresented the significance of the graphs. Let’s go to the report itself and see what it says about figure 4.4
“Figure 4.4: One of the most compelling illustrations that McIntyre and McKitrick have produced is created by feeding red noise [AR(1) with parameter = 0.2] into the MBH algorithm. The AR(1) process is a stationary process meaning that it should not exhibit any long-term trend. The MBH98 algorithm found ‘hockey stick’ trend in each of the independent replications. then, immediately under the hockeystick graphs,
“Discussion: Because the red noise time series have a correlation of 0.2, some of these time series will turn upwards [or downwards] during the ‘calibration’ period6 and the MBH98 methodology will selectively emphasize these upturning [or downturning] time series.
Yeh, he copied, he said he did! Further, when reading after the word “discussion” we can see that the graphs in 4.4 weren’t implied to be random, but rather illustrative. It states clearly some will turn upward while others turn downwards. All of the graphs shown in 4.4 turn upwards. So, clearly Wegman’s graph weren’t placed to reflect a random selection of the runs. More from section 4
“We have been able to reproduce the results of McIntyre and McKitrick (2005b). While at first the McIntyre code was specific to the file structure of his computer, with his assistance we were able to run the code on our own machines and reproduce and extend some of his results. In Figure 4.1, the top panel displays PC1 simulated using the MBH98 methodology from stationary trendless red noise.”
You guys are something else.
The criticism of M&M is that the MBH98 produced an upward pointing hockey stick because of an inherent bias in their methodology. They fooled a lot of people by presenting a number of graphs of upward pointing hockey sticks produced from red noise. The point is that this is a fallacious argument, when 1% of the red noise results in upward pointing hockey sticks and 1% results in downward pointing hockey sticks. Showing only the upward pointing graphs, and claiming this proves that there is a bias in the analysis of MBH 98, is an incorrect argument. The question is was this a result of a conscious desire to deceive, or did they really miss the point. Whichever it is, it is pretty clear the both M&M and Wegman’s Report made the identical errors. It is clear that Wegman did not produce an “independent verification” because he used M&M’s methodology and software, and made the same fallacious argument.
Since hockey sticks like MBH 98 appeared in only 1% of the cases when random noise was fed into it, it seems to me there is 99% confidence that MBH98 ‘s results are not the result of analysis of random noise.
In addition other forms of analysis of proxy data consistently produce hockey sticks. It is pretty clear that any unbiased investigation of what Wegman has done, would condemn it as wrong and misleading. We will see what George Mason University’s investigation says.

James Sexton says:

Do you people even read the stuff you guys are throwing around?

From the linked article, Wegman says:

Figure 4.4: One of the most compelling illustrations that McIntyre and McKitrick have produced is created by feeding red noise [AR(1) with parameter = 0.2] into the MBH algorithm. The AR(1) process is a stationary process meaning that it should not exhibit any long-term trend. The MBH98 algorithm found ‘hockey stick’ trend in each of the independent replications.

But James, if *you* bothered to read that whole article you would discover that (somehow) Wegman assumed that McIntyre used AR1(.2) to generate his red noise. In fact, as Deep Climate verified by reading McIntyre’s archived code, he used ARFIMA instead. Huge difference. It is more like AR1 with a parameter of .9 (way more *persistent*), and is much more likely to produce a hockey stick shape when you derive a PC1 from the resulting red noise.
And, of course, you probably don’t have a problem with someone drawing the conclusions of their report from a hand-picked sample representing only 1% of the total 10,000 simulation runs. Does that seem like proper use of statistics to you, or does it in fact seem like someone with an agenda?
Between these revelations and the plagiarism, Wegman is in deep academic doo-doo. He was tasked to *independently verify* the statistics used in M&M05 and he obviously did nothing of the sort. The Wegman Report turns out to be nothing but a stitch-up, and is likely to be retracted as a result.

slightly O/T but
Is it me, or does there seem to be a concerted flaming effort by the warmists hereabouts and elsewhere?
This whole Wegman thing is way old and it just seems as if deliberate flaming is taking place. Thing is, it’s a terrible faux pas by the warmistas if there is such a campaign as it is simply reminding everyone of how weak the AGW arguments are ahead of Cancun!
Perhaps we should treat the warmistas with the same contempt that they show those outside their little campus?

Martin Brumby says:
November 23, 2010 at 11:55 am
Just be happy about the damage your silly beliefs have already done to the world’s poorest & most vulnerable, …
—-
…. and they surely are happy.
They were desperate for the Arctic ice to follow the prophecies of the Goreacle. They yearned for the loss of Arctic ice. Why ?? …… so that they could save the Arctic ice, of course ??
Now there are lots more poor and vulnerable to save ……. with taxpayer’s money, of course.
I’ve read on here on several occasions that the global warming hoax is the biggest lie ever told. I don’t think so. I think it’s just the baby brother of “Poor people are better off under socialism”. Watch out poor people – fake-socialism is even worse.

Robinson says:
November 23, 2010 at 3:55 pm
I see The Mail has now has an article on Wegman. It seems to me that someone is promoting this story. I don’t think the MSM would have picked it up through research and investigation; something they do very little of these days.

——————-
Robinson,
Looks like a pre-Cancun mini-offensive coordinated to offset some pre-Cancun bad news coming to the AGW folks.
A fore-shadow?
John

Of course someone is promoting this garbage: deepclimate, Mashey and the activists scientists, some of them working in a building paid for a retired oilman… I guess the irony must escape them.

So now the attention is on the Wegman Report. With friends like these plagarism hysterics AWG does not need enemies. Wegman Report simple, incestuous group of climatology members engage in group think, cannot comprehend statistics and will not call in proffessional statistitions to check their maths.

@Steve Metzler
Points 1 -5…my answer: So? Doesn’t bother me as a scientist.
Why do you expect a fully independent review? It is as if you are looking for clean room reverse engineering. Sorry, but that is a waste of time and not needed. All Wegman needed to do was review MM code and statistics. Why reinvent the wheel if the wheel was already made?
Sorry, but your points are weak.

So many people will be able to go to their grave without any trace of worry about the kind of world they’re leaving to their children and grandchildren, and we have wonderful sites like WUWT to thank for it. It’s so reassuring to know it’s not really getting warmer, or if it is it’s not our fault and it’ll cool off again real soon, and if it doesn’t then a bit warmer is good for us. It’s nice to have that reaffirmed day after day in the face of otherwise slightly troubling data from the real world. And if Mr Wegman and Mr MacIntyre’s next work is a study of the mean height of American males, taken from a random sample of delegates to a basketball convention, I’m sure you’ll be in here defending that with the same vigour and blind enthusiasm and yes, whininess. If that’s a word.

Steve Metzler says:
November 23, 2010 at 3:42 pm
James Sexton says:
Do you people even read the stuff you guys are throwing around?
…..
You said, “But James, if *you* bothered to read that whole article you would discover that (somehow) Wegman assumed that McIntyre used AR1(.2) to generate his red noise. In fact, as Deep Climate verified by reading McIntyre’s archived code, he used ARFIMA instead. Huge difference. It is more like AR1 with a parameter of .9 (way more *persistent*), and is much more likely to produce a hockey stick shape when you derive a PC1 from the resulting red noise.
And, of course, you probably don’t have a problem with someone drawing the conclusions of their report from a hand-picked sample representing only 1% of the total 10,000 simulation runs. Does that seem like proper use of statistics to you, or does it in fact seem like someone with an agenda?”
========================================================
Sis, I’m not a statistician. I’ve taken some statistics classes. That said, I know what I read, I quoted it back to you. I know what was implied, the implications were clear. I, also, know the statistical significance of 1% and whether it is significant or not. Do you?
=======================================================
Again you said, “Between these revelations and the plagiarism, Wegman is in deep academic doo-doo. blather, blather, blah, blah….”
=======================================================
You guys are hilarious. You are attacking M&M by proxy? You mention plagiarism? I’ve posted here that Bradley was mentioned 35 times in the report. Some 13 times in the bibliography. He stated in his report to congress that he used M&M’s code, but only because Mann wouldn’t submit his! IT’S RIGHT THERE IN CONGRESSIONAL TESTIMONY!!! You want to blather on about academic acrimony? Why don’t you discuss M&M if you want to attack the statistics? I can supply the link if you or DC have the eggs to do so. It has been my experience that CA allows open debate. Give it a whirl! But, that’s not what I find laughable.
What I find laughable is the admittance of the logarithmic nature of heat caused by CO2. This concept is fairly well accepted by both alarmist and skeptical communities. In any of the graphs offered, be it Mann et. al. or M&M or Wegman, do you see a logarithmic graph? You’re chasing an academic fantasy. Its a discussion of statistics. Not reality. (BTW,I bet on the statisticians, but that’s just me.) The entire prospect is preposterous. No, I’m not talking about the critique of Wegman, that’s expected, albeit several years late. Nor am I talking about the back and forth of M&M and Mann et. al. That, too, is expected. BTW, we forgot to include other statisticians such as McShane to the list, why don’t you guys go for the triflecta? Oh, right, there are more that disagree with the way the hockey stick is created. But that brings us back to my point of hilarity.
By this tree ring, or a collective of tree rings, hundreds of years old, I can tell you with certainty that in the year of our Lord 1535, the global average temperature was 57.56 degrees Fahrenheit and through the rest of the century, the global anomaly was -0.22 degrees Fahrenheit.
Yeh, I’d hang my hat on that, too. And you’re going to worry that Wegman didn’t mention Bradley a 36th time? No sis, if this is your position, you’ve much more to worry about than that.
Good luck,
James Sexton

I vote, “slightly troubling data from the real world” as the quote of the week.

And Pamela, I was going to vote for “So in this case, do the errors invalidate the conclusions? Not one tiny bit.” as quote of the week. Since it’s equally applicable to Wegman, McIntyre and Mann et al. Wegman’s report is clearly so flawed we should probably all just pretend it never existed from here on in. McIntyre made useful points about Mann’s inadequate use of PCA. And despite the errors, the result is much the same as Mann’s later study where he modified the technique. And of course the hockey stick results have been independently verified. Over and over again.
And don’t give me the ‘fact free zone’ nonsense Doug. Find me any facts in this current thread, outside of Steve Metzler’s summary of the problems with Wegman. And ‘dogmatic beliefs’? Puhleeze. I lurk here, and at The Air Vent and occasionally Jo Nova’s blog, and I have a number of AGW sites bookmarked as well. I’ve been trying to get my head around the science and the uncertainties involved for about three years. I’m not a scientist or a statistician, but I can see where the weight of evidence leads. And I’ve seen enough to consider an accusation of ‘dogmatism’ from a denizen of this blog to be pretty damned funny.

Ben says:
November 23, 2010 at 6:10 pm
Just a couple questions:….. very nice!
Kforestcat says:
November 23, 2010 at 7:11 pm
Gentlemen
Frankly, I am getting pretty fed-up with the AGW academic communities on this issue. I draw a line when any group (AGW or not) believes they may establish such standards as to prevent United States Congress from executing its duties….Perfect!
To both, great arguments, wish I’d have thought of them myself! It would have saved me considerable time and effort. (Well, a few minutes, anyway.) These guys are fun to play with!

Actually what M&M did was nonsense. They used an noise algorithm that generated some persistant noise, used the MBH data and counted how many hockey sticks were generated. It was about 1% of the cases for upward pointing hockey sticks and 1% downward.

I suggest you go back and actually read their analysis of MBH98/99, as well as the subsequent work they’ve done. It is clear you’ve only picked up your talking points from listening to others complain and, as is often the case with a grapevine tale, you’ve muddled much of what was really said (and done.)
There are numerous problems with MBH98/99, not all of which directly involve the “method” itself, some of which are the simple failures of the authors to demonstrate (or prove) their own assumptions.
Mark

Crispy and Proud of it. says:
November 23, 2010 at 8:46 pm
“….Wegman’s report is clearly so flawed…..”
=======================================================
Yeh, sure, please point, specifically, to the flaws. Start with the statement, be it textual or mathematical, show where in the report it is stated, and then the reasoning behind your assertion that it is flawed. If you can do that, I will, with most certainty, consider your arguments. Please expect follow-up questions.

Loved Crispy’s “do it for the children and grandchildren” crap. How many have come to this site and tried to pull that bullshit line?!?! What a crock of hooey. Next comes the oil company shill diatribe. Another brainwashed useful idiot.

Crispy and Proud of it. says:
November 23, 2010 at 8:46 pm
And I’ve seen enough to consider an accusation of ‘dogmatism’ from a denizen of this blog to be pretty damned funny.
The difference of the denizens of this blog with the denizens of Real Climate for example, is that they are dogmatic on demanding,
1)Original data available to all researchers,
2)clear methods, including programming code, available to all researchers
3) correct use of statistics
4) error calculations on all plots
5) global evaluation of models: example anomalies might fit but temperatures are off by more than the projected heating in 100 years.
6) etc. wherever the scientific method is not followed.
I am a scientist, and am dogmatic about following the scientific method, and not astrology type hand wavings.on which the AGW religion relies.

this sorry episode is a good example of the power of personal destruction and distraction
the only reason that we’re even talking about this is ClimateGate. An anonymous blogs claims to have found plagiarism in a 4 year old document to 1. attack the criticizer of Mann and 2. distract from Climategate itself.
It was highly useful when the VA AG filed his suit against Mann…then Mann’s advocates could whip this out to make Mann look falsely accused by a plagiarist.
The tactic has been so successful that blogger (including CA) have spent considerably effort looking into it. The humor being that Wegman’s main point was the statistical point that McIntyre was right, and whatever was “lifted” from Bradley was background stuff…Wegman is not a dendro.
Since the congress has been taken over by the Rs, this will certainly get widespread play in the press…when another hearing is scheduled. And certainly no one in the press will give a nuanced look a this.

Doug Badgero, as you say, this thread is not about the science of AGW, and my comments do not touch on it. I do not see how you can draw any conclusions about my knowledge on the subject. I will do as you suggest and read Lindzen. I searched Google Scholar a few weeks back looking for a recent Lindzen paper setting out his case for a low climate sensitivity to CO2 – since that is his primary departure from the mainstream – with no success. Hopefully he makes his case in the submission to congress. No doubt we’ll get a chance to talk about the ‘real debate’ on another thread. Happy to engage. Blogs on both sides have far too few contrarians.
James Sexton, I’m not a stats man, and neither, as you’ve said, are you. I’ve gone through the DC post, and DC, unlike McIntyre, does go to the trouble of carefully explaining the statistical reasoning. Eadler and Steve Metzler clearly are stats men, and have spelt out the main problem with Wegman duplicating and rubber stamping M&Ms approach, rather than reporting on their varacity. Do you think Congress was well served by this? Answer Eadler and Steve rather than bait me.
anna V, where does plagiarism fit into your list? Is it at No.7? Is No1 a go at Phil Jones? Do we have to go through that again? No2 You want programming code? Surely if you’ve got the data one writes ones own code? Isn’t the point to replicate results, and not just repeat? No3 Statistics. Indeed. Cuts both ways… ‘it’s been cooling since 1998’ and so on. And what did you think of Lubos Motl framing the question for Jones starting at 1995 to get just the right answer about statistical significance to create a cheap meaningless PR coup? No4 Error calculations on all plots. Like in the IPCC report, sure. No.5Models. Couldn’t agree more. Lots of work to be done on models.
[Snip] If you want to believe the few over the many, and conjecture over data, because it suits your view of the world or whatever, that’s your right. Don’t call me an idiot for following the data to a conclusion that gives at least a reasonable amount of concern for the future. You’re hanging around on a site that gave the world ‘Carbonic snow in the arctic’, for Chrissakes. Hang on, so am I 🙂

Please guys stop feeding the Trolls, you can tell the trolls as they have nothing to add to the discussion but unsupported drivel eg:-
eadler says: – Instead of shwoing M&M’s work is incorrect because he knows he can’t ha attacks a report (and thats all it was) that proves that M&M’s work is correct, It’s simply puppy kicking because he’s scared to bedate M&M’s work.
Crispy and Proud of it. says:
November 23, 2010 at 8:46 pm
‘And don’t give me the ‘fact free zone’ nonsense Doug. Find me any facts in this current thread’
When in the story at the very top are all the facts he needs its under ‘Wegman whiners: this post’s for you’, but instead of learning something he did not know he simply came here and tried to Troll. It is something I have seen time and time again by expert trolls, not miss guide fools.
These trolls have taken far to much time up and unless they can show why M&M’s work and not a report that simply chekced the numbers is correct then ignore them for the forum trolls they are.

It is pitiful to watch all of the trolls squirm as they are being batted about by those who do understand the Wegman report. Nothing they have said really was very persuasive when the arguments are compared.

Sorry can’t help myself, some of these RC trolls are hilarious.

Sorry if I come across as a troll. Perhaps this was a bad post to choose to weigh in on, since stats is not my strong point. Eadler and Steve Metzler are certainly not trolls – they offer a perspective on the topic. Geronimo tries to take them down. We all let our confirmation bias kick in and cheer for whom best suits our perspective. I look forward to trying to get to the bottom of it all.
My original intention was satirical. I think blog science is pretty silly. Most people come to sites like this to be reassured. Wegman under attack? Check out WUWT, they’ll set it straight. And yes, the same confirmation bias happens at the AGW sites. I think everyone needs to get out more. The Judith Curry approach, perhaps. But in the end, if we want to learn about the science, I think we should study the science. Not so much what people with agendas SAY about the science.
Christopher Hanley, thanks for the link to the animation. I think it’s a bit of train wreck. I wish I could pause it and study it properly. It seems to splice the Mann global temp reconstruction onto the Vostok core data (Antarctica, one location) and then throws in the NOAA Greenland cores – just one location, mind you. Do you think that’s kosher? The temp scale on the left jumps around, but it is seriously sub zero, but as the longitudinal scale lengthens the Mann Hockey stick is allowed to shrink, so that the modern uptick in temperatures seems to take us to… er, about 30 below zero according to the scale. I guess the point to be illustrated is that ice age temperature anomalies are on a par with what’s happening now. Is that the point? And sure, they are, on a millennial scale. But that has nothing to do with the present rapid warming as CO2 forcing accumulates heat, over and above natural variability, does it? nevertheless, the animation is reassuring if you just want to be… reassured.

Larry in Texas says:
November 24, 2010 at 1:05 am
It is pitiful to watch all of the trolls squirm . . . [edit] . . .
——————
Larry in Texas,
What is even more pitiful is they want us to do their research for them. Maybe they are under aged trolls who don’t get out much on their own without parental supervision of their troll masters at RC?
For instance, there is this pitiful plea.

Crispy and Proud of it. says:
November 24, 2010 at 1:12 am
Sorry, kinda OT. Does anyone have a link to the Lindzen congressional report pdf? Anthony’s link on the Nov 17 post seems to be cactus. Doug? Many thanks

Yuck, I just had a mental flash of a pre-pubescent troll-like being chained to his computer by RC overlords. : )
John

Gail Combs says:
November 24, 2010 at 4:01 am

No special exemption for Congress is necessary. The copyright law/s have always included the exception for “fair use” and “fair dealing” as the means of implementing the original purpose of copyright law, securing publications for comment, review, and otherwise advancing the public benefit. The U.S. Code is explicit below where it says, “reproduction of a work in legislative or judicial proceedings or reports….The Committee has considered the question of publication, in Congressional hearings and documents, of copyrighted material…the Committee believes that the publication would constitute fair use.”
-CITE-
17 USC Sec. 107 02/01/2010
-EXPCITE-
TITLE 17 – COPYRIGHTS
CHAPTER 1 – SUBJECT MATTER AND SCOPE OF COPYRIGHT
-HEAD-
Sec. 107. Limitations on exclusive rights: Fair use
-STATUTE-
Notwithstanding the provisions of sections 106 and 106A, the fair
use of a copyrighted work, including such use by reproduction in
copies or phonorecords or by any other means specified by that
section, for purposes such as criticism, comment, news reporting,
teaching (including multiple copies for classroom use),
scholarship, or research, is not an infringement of copyright. In
determining whether the use made of a work in any particular case
is a fair use the factors to be considered shall include –
(1) the purpose and character of the use, including whether
such use is of a commercial nature or is for nonprofit
educational purposes;
(2) the nature of the copyrighted work;
(3) the amount and substantiality of the portion used in
relation to the copyrighted work as a whole; and
(4) the effect of the use upon the potential market for or
value of the copyrighted work.
The fact that a work is unpublished shall not itself bar a finding
of fair use if such finding is made upon consideration of all the
above factors.
-SOURCE-
(Pub. L. 94-553, title I, Sec. 101, Oct. 19, 1976, 90 Stat. 2546;
Pub. L. 101-650, title VI, Sec. 607, Dec. 1, 1990, 104 Stat. 5132;
Pub. L. 102-492, Oct. 24, 1992, 106 Stat. 3145.)
-MISC1-
HISTORICAL AND REVISION NOTES
HOUSE REPORT NO. 94-1476
General Background of the Problem. The judicial doctrine of fair
use, one of the most important and well-established limitations on
the exclusive right of copyright owners, would be given express
statutory recognition for the first time in section 107. The claim
that a defendant’s acts constituted a fair use rather than an
infringement has been raised as a defense in innumerable copyright
actions over the years, and there is ample case law recognizing the
existence of the doctrine and applying it. The examples enumerated
at page 24 of the Register’s 1961 Report, while by no means
exhaustive, give some idea of the sort of activities the courts
might regard as fair use under the circumstances: “quotation of
excerpts in a review or criticism for purposes of illustration or
comment; quotation of short passages in a scholarly or technical
work, for illustration or clarification of the author’s
observations; use in a parody of some of the content of the work
parodied; summary of an address or article, with brief quotations,
in a news report; reproduction by a library of a portion of a work
to replace part of a damaged copy; reproduction by a teacher or
student of a small part of a work to illustrate a lesson;
reproduction of a work in legislative or judicial proceedings or
reports; incidental and fortuitous reproduction, in a newsreel or
broadcast, of a work located in the scene of an event being
reported.”
Although the courts have considered and ruled upon the fair use
doctrine over and over again, no real definition of the concept has
ever emerged. Indeed, since the doctrine is an equitable rule of
reason, no generally applicable definition is possible, and each
case raising the question must be decided on its own facts. On the
other hand, the courts have evolved a set of criteria which, though
in no case definitive or determinative, provide some gauge for
balancing the equities. These criteria have been stated in various
ways, but essentially they can all be reduced to the four standards
which have been adopted in section 107: “(1) the purpose and
character of the use, including whether such use is of a commercial
nature or is for nonprofit educational purposes; (2) the nature of
the copyrighted work; (3) the amount and substantiality of the
portion used in relation to the copyrighted work as a whole; and
(4) the effect of the use upon the potential market for or value of
the copyrighted work.”
These criteria are relevant in determining whether the basic
doctrine of fair use, as stated in the first sentence of section
107, applies in a particular case: “Notwithstanding the provisions
of section 106, the fair use of a copyrighted work, including such
use by reproduction in copies or phonorecords or by any other means
specified by that section, for purposes such as criticism, comment,
news reporting, teaching (including multiple copies for classroom
use), scholarship, or research, is not an infringement of
copyright.”
[….]
When a copyrighted work contains unfair, inaccurate, or
derogatory information concerning an individual or institution, the
individual or institution may copy and reproduce such parts of the
work as are necessary to permit understandable comment on the
statements made in the work.
The Committee has considered the question of publication, in
Congressional hearings and documents, of copyrighted material.
Where the length of the work or excerpt published and the number of
copies authorized are reasonable under the circumstances, and the
work itself is directly relevant to a matter of legitimate
legislative concern, the Committee believes that the publication
would constitute fair use.
During the consideration of the revision bill in the 94th
Congress it was proposed that independent newsletters, as
distinguished from house organs and publicity or advertising
publications, be given separate treatment. It is argued that
newsletters are particularly vulnerable to mass photocopying, and
that most newsletters have fairly modest circulations. Whether the
copying of portions of a newsletter is an act of infringement or a
fair use will necessarily turn on the facts of the individual case.
However, as a general principle, it seems clear that the scope of
the fair use doctrine should be considerably narrower in the case
of newsletters than in that of either mass-circulation periodicals
or scientific journals. The commercial nature of the user is a
significant factor in such cases: Copying by a profit-making user
of even a small portion of a newsletter may have a significant
impact on the commercial market for the work.
[….]
The criteria of fair use are necessarily set forth in general
terms. In the application of the criteria of fair use to specific
photocopying practices of libraries, it is the intent of this
legislation to provide an appropriate balancing of the rights of
creators, and the needs of users.
AMENDMENTS
1992 – Pub. L. 102-492 inserted at end “The fact that a work is
unpublished shall not itself bar a finding of fair use if such
finding is made upon consideration of all the above factors.”
1990 – Pub. L. 101-650 substituted “sections 106 and 106A” for
“section 106” in introductory provisions.
EFFECTIVE DATE OF 1990 AMENDMENT
Amendment by Pub. L. 101-650 effective 6 months after Dec. 1,
1990, see section 610 of Pub. L. 101-650, set out as an Effective
Date note under section 106A of this title.

Sorry folks. I had to break off at 2:00AM my time last night. I’m 5 hrs ahead of EST…
_Jim says:

ARFIMA , as in “Autoregressive Fractionally Integrated Moving Average”?
And we are to understand that Deep Climate, by “reading McIntyre’s archived code” verified this technique was used?
I wasn’t aware Deep Climat was in any way qualified, schooled or educated (either formally, via self-education or night school) in statistics to make a claim such as this?
What are the chances he (DC) is in error?

Virtually non-existent. I just happen to be a professional programmer (and electrical engineer by education so I can handle the math), and was looking at McIntyre’s R code myself. He definitely used ARFIMA rather than the AR1(.2) algorithm that Wegman somehow assumed he used.
And if you actually took the pains to read the article over at Deep Climate, you could see for yourself that doing this adds more persistence to the red noise, thus making it more likely to recover a hockey stick shaped PC1 from the noise. Please refer to the last diagramme in DC’s article, and of course, the accompanying text above and below it.

Steve Melzer 6:35 am –
“I just happen to be a professional programmer (and electrical engineer by education so I can handle the math), and was looking at McIntyre’s R code myself. He definitely used ARFIMA rather than the AR1(.2) algorithm that Wegman somehow assumed he used.”
Steve McIntyre covered this in 2006 –
http://climateaudit.org/2006/09/03/more-tangled-webs/#comments
Specifically, “In our red noise discussions, we did two calculations – one with ARFIMA noise and one with AR1 noise. The ARFIMA noise produced pretty hockey sticks but introduced a secondary complication and replications have focused on AR1 examples. To set parameters for the simulation, we calculated AR1 coefficients on the North American AD1400 tree ring network using a simple application of the arima function in R:
arima.coef = arima(x,order=c(1,0,0))
This is what Ritson is criticizing, arguing that application of a standard arima function to a tree ring network without previously removing trends is incorrect. Now it seems to me that Ritson has recently argued that VZ’s implementation of MBH made some sort of ghastly error by removing a trend prior to regression – so it’s hard to say what Team policy is on when trends should be removed and when trends shouldn’t be removed – but that’s a story for another day.
However here my point is different. Whatever the right method may be, the method that I used simply followed Mann’s own methodology.”
Now I’m not clever enough to be able to comment on that, but does it answer your point?

smokey says:

…with skeptical scientists [the only honest kind of scientist].

ROTFLMAO! See my last post (November 24, 2010 at 7:06 am).

Sorry, I mis-spelt Steve Metzler’s name in my 7.06 am comment above.
BTW only 23 comments have been published on the Daily Mail article. Mine was never published and I suspect they’ve closed off the comments as too many were critical of the article.

Should Peter Bloomfield be “hung out to dry” too?

Steve Metzler says:
November 24, 2010 at 6:35 am
“And if you actually took the pains to read the article over at Deep Climate,…..”
========================================================
Steve, why do you think people here haven’t read the article? I’m willing to bet many regulars here are re-reading the damned thing.
It seems the point of contention is the AR1 vs ARFIMA. Looking at the code snippet, it seems M&M did use ARFIMA, as opposed to AR1 as Wegman claimed, and it is somehow very significant towards the critique of Mann? But, even before the writer gets to the point, we see this “It’s true that NRC did provide a demonstration of the bias effect using AR1 noise instead of ARFIMA. “
Yeh, ok, got that. Then there’s the hand waving about copying M&M’s code. Wegman stated that he did. Did you bother to read the report? The rest is simply advocacy and cheer-leading.
Steve, is there any point to that article or reason why anyone should care about this?
What part of we were able to run the code on our own machines and reproduce and extend some of his results… do you guys not understand? ITS IN THE REPORT!!! AND HAS BEEN FOR YEARS!
Another point totally missed by DC was the fact that in the report, Wegman shows where he couldn’t use Mann’s code because Mann refused to provide it.
One other thing. Doing a graph comparison, DC shows us a graph (right) red noise null proxies. that carries striking similarities to some graphs produced by McShane and Wyner.

Jockdownsouth says:
November 24, 2010 at 7:06 am
Steve Melzer 6:35 am –
“I just happen to be a professional programmer (and electrical engineer by education so I can handle the math), and was looking at McIntyre’s R code myself. He definitely used ARFIMA rather than the AR1(.2) algorithm that Wegman somehow assumed he used.”
Steve McIntyre covered this in 2006 –
http://climateaudit.org/2006/09/03/more-tangled-webs/#comments
Specifically, “In our red noise discussions, we did two calculations – one with ARFIMA noise and one with AR1 noise. The ARFIMA noise produced pretty hockey sticks but introduced a secondary complication and replications have focused on AR1 examples. To set parameters for the simulation, we calculated AR1 coefficients on the North American AD1400 tree ring network using a simple application of the arima function in R:
=======================================================
Ouch! That’s gotta hurt. Oh well, that was fun while it lasted. Does any DC fan have anything to add?

“”Crispy says:
November 24, 2010 at 4:04 am””
“…We all let our confirmation bias kick in and cheer for whom best suits our perspective. I look forward to trying to get to the bottom of it all.
My original intention was satirical. I think blog science is pretty silly. Most people come to sites like this to be reassured. Wegman under attack? Check out WUWT, they’ll set it straight. And yes, the same confirmation bias happens at the AGW sites. I think everyone needs to get out more. The Judith Curry approach, perhaps. But in the end, if we want to learn about the science, I think we should study the science. Not so much what people with agendas SAY about the science…”
As a fellow ‘Crispy’ I agree with most of the above, I would amend thus: “We all let our confirmation bias kick in and cheer for whom follows the scientific method that most closely reflects our observational data.”
The question, debate, and discussion circles around who/how, and to what degree this has been achieved. In regards to the current Wegman see: James Sexton says:
November 23, 2010 at 2:20 pm
and: James Sexton says:
November 23, 2010 at 7:42 pm
These really are enough of a refutation.
Perspective as such may help give us direction in pursuing a given direction when in search-mode but offers nothing in regards to results: the results are what they are, regardless of perception/perspective. MBH was thoroughly deconstucted by M&M, or as far as it could be without having the specific code and methodology used by MBH. Congress wanted verification from Wegman as to the veracity of the critiques and methods leveled by M&M. Wegman achieved this and verified that the M&M critique was valid and that some of the statistical methods used by MBH were inappropriate. The North report corraborated Wegman’s results while admitting that Wegman went further in their efforts; North agreed (under oath) that their results, essentially, were in agreement.
Here’s where my own “confirmation bias” kicks in: IMO (I’ve read others that might agree with me on this) There has been an underlying political agenda being foisted upon us since the early 1970’s (’60’s maybe) starting whith the “Coming Ice Age” scare ‘tatctic’, evolving into the current warming scare. That is, that global banking cartels wanted a centralized global government to better control the volatility of global currencies. Utilizing energy consumption control, again globally, to achieve this end. My point: that the MBH study was politically motivated, hence the code and methodology COULD NOT be shared as the underlying motive would be exposed. It was fraud at the start, in the middle and presently. The MC&W paper corroborated what M&M, Wegman and North found. Perhaps you have been following this as I have been: from the sidelines and may recall that many times on this site and others that AGW”s have MANY times cited the Wegman and North reports as VALIDATING/EXONERATING the MBH products. Isn’t it funny the stance they are now taking? That irony really should make this fun reading except for the fact that billions of our tax dollars have been wasted (your tax $$) and they want MORE!! I’ll start laughing as soon as I stop crying.
Hope this helps.

I should add as further evidence for politcal collusion in regardes to the MBH products that the congressional outcome was to uphold that the criticisms of M&M were valid, yet the committee STILL upheld the results of MBH as being correct. How is this possible? There was a pre-determined political outcome that had to be upheld.
Recall that the McS&W paper showed (as was shown at the original hearings by Wegman et al) that the results were skewed due to improper use of statistical methods and showing that there were/are no statistically significant results to be found in the MBH et al datasets concerning global temperature trends.

Steve Metzler,
One step at a time. Is the argument regarding ARIFMA and AR1 now moot?

Here is a link to McIntyre’s response to the nonsense being purveyed by the trolls:
http://climateaudit.org/2010/11/08/the-hockey-league/#comment-245884

Down goes Wegman, down goes Barton ( no chair for dorks), down goes Watts.
[Reply: Another comment insulting our host, and down goes “John McManus.” ~dbs, mod.]

And just for proper context, here are relevant passages from MM05.

Computer scripts used
to generate simulations, figures and statistics, together with
a sample of 100 simulated ‘‘hockey sticks’’ and other
supplementary information, are provided in the auxiliary
material1. We carried out 10,000 simulations, in each case
obtaining 70 stationary series of length 581 (corresponding
to the 1400–1980 period).

Also

The simulations nearly always yielded PC1s with a
hockey stick shape…

If “back up with facts” ever makes it back, maybe he can also comment on Figure 2 in MM05, which caption reads:

Histogram of ‘Hockey Stick Index’ for PC1s. For
the 10,000 simulated PC1s described in text, the histogram
shows the distribution of the difference between the 1902–
1980 mean and the 1400–1980 mean, divided by the 1400–
1980 standard deviation. Top: Conventional (centered)
calculation; Bottom: with MBH98 data transformation.

http://climateaudit.files.wordpress.com/2009/12/mcintyre-grl-2005.pdf
Is it possible poor old DC found the part of the code that picked out the “sample of 100 simulated ‘hockey sticks’…” and convinced himself that’s all MM05 looked at?

John M says:
November 24, 2010 at 11:47 am
Is it possible poor old DC found the part of the code that picked out the “sample of 100 simulated ‘hockey sticks’…” and convinced himself that’s all MM05 looked at?
========================================================
It is starting to appear that this is the case. He took code snippets but failed to look at the code in its entirety. This was either real lazy, or intentional. Which, I find humorously ironic.
What is even more interesting, both Macs have recent comments on CA, yet, no one from DC is directly address either one…?????? In fact, Steve’s comment is sitting at the bottom of the page with that “reply” button just sitting there, waiting to be pressed.
Oh well, maybe in the next month or so they’ll have invented another story about Wegman to where once again, I’ll have to find all of the studies and re-read them just to find the assertions from the warmistas were once again, invented.
I wonder where the obsession over the Wegman report comes from? It’s not like there haven’t been other papers written that totally destroys the hockey sticks, jeez. They invent plagiarism as an attempt to discredit Wegman, where it is quite obvious that he didn’t. Now their inventing some bizarre copying/misinterpretation of Wegman to M&M.

Steve Metzler,
“There is a *link* to McIntyre’s M&M05 code and saved-off cherry-picked ‘top 100 hockey sticks’ in the article. Deep Climate found *all twelve* hockey sticks that were shown in the Wegman Report in that top 100.”
Ok, I’ve found the relevant code and the DeepClimate response.
#SAVE A SELECTION OF HOCKEY STICK SERIES IN ASCII FORMAT
order.stat<-order(stat2,decreasing=TRUE)[1:100]
order.stat<-sort(order.stat)
hockeysticks<-NULL
for (nn in 1:NN) {
load(file.path(temp.directory,paste("arfima.sim",nn,"tab",sep=".")))
index<-order.stat[!is.na(match(order.stat,(1:1000)+(nn-1)*1000))]
index<-index-(nn-1)*1000
hockeysticks<-cbind(hockeysticks,Eigen0[[3]][,index])
} #nn-iteration
dimnames(hockeysticks)[[2]]<-paste("X",order.stat,sep="")
write.table(hockeysticks,file=file.path(url.source,
"hockeysticks.txt"),sep="\t",quote=FALSE,row.names=FALSE)
The first line sorts the set of PC1s by descending HSI and then copies the first 100 to another array.
=========================
Well, that may be a case of confirmation bias. Firstly, there is no mention of HSI (hockey stick index) in the code. Secondly, I cannot see how you could ascribe an index number for something as vague as 'hockeystickness'. You'd have to do that by eye. And thirdly, I'm not sure how you could sort something with a single iteration – you would need 2 levels of nesting.
I'm not saying DeepClimate is wrong, just that I'm not convinced that that code snippet supports the conclusion that it was designed to mine hockey shapes.

Vince Causey,
MM05 do indeed define a “hockey stick index”

For convenience, we define the ‘‘hockey
stick index’’ of a series as the difference between the mean of
the closing sub-segment (here 1902–1980) and the mean of
the entire series (typically 1400–1980 in this discussion) in
units of the long-term standard deviation (s), and a ‘‘hockey
stick shaped’’ series is defined as one having a hockey stick
index of at least 1 s.

However, as I said before, Figure 2 in their paper is for all 10,000 calculations.

James Sexton says:
November 24, 2010 at 12:11 pm

John M says:
November 24, 2010 at 11:47 am

Is it possible poor old DC found the part of the code that picked out the “sample of 100 simulated ‘hockey sticks’…” and convinced himself that’s all MM05 looked at?

It is starting to appear that this is the case. He took code snippets but failed to look at the code in its entirety. This was either real lazy, or intentional. Which, I find humorously ironic.

Neither. You should go back and read some of DC’s “early” posts on this subject when he was first attempting to learn what Steve had done. DC did not get it, not in the least. His background on the subject matter appeared weak at best, if not entirely non-existent. DC began with the assumption that Steve was wrong and has literally spent the past several years looking for a way in which he can prove his own foregone conclusion. Hardly the scientific approach, but give a dog his bone, he stuck with it… until he realized the best he could do was impugn the integrity of the hall of famer that backed-up Steve’s initial analysis.
Really, think about it, DC’s argument is that Steve is wrong because a) Wegman plagiarized the background section regarding Bradley’s work and b) Wegmans analysis was not original. Seriously. It beggars belief that anybody in their right mind would believe this idiot, yet here we are arguing with utter morons who are doing exactly that. None of them understand the underlying statistics or methods, yet somehow they just know DC is correct. Amazing.
For the record, I needed neither Steve’s nor Wegman’s analysis to see the flaws in MBH98/99. The entire argument should have ended in peer review when assessing the validity of using PCA (or any variance based detection method) on this data. We never would have gotten to this stage had a single, independent statistician been one of the reviewers.
Mark

Mark T says:
November 24, 2010 at 1:00 pm
Thanks for the DC background. I remember before he had his own blog he made several appearances at CA. Seemed to be learning statistics as he went. Sometimes showed some insight, but other times got his head handed to him statistically.

None of them understand the underlying statistics or methods

I can’t claim to understand them in detail myself, but even just a little bit of reading through MM05 and reading the thread Jockdownsouth linked to earlier (http://climateaudit.org/2006/09/03/more-tangled-webs/#comments) makes me think of a mechanic, who after you’ve told him your engine blew up 30 seconds after he did an oil change, and you point to the still fresh path of oil from his bay to your car, tries to argue that the problem is that you used the wrong kind of vacuum cleaner the last time you swept out your car.

Mark T, John M,
Thanks for the clarification. There is definately something wiffy about DC’s conclusions that don’t stack up. The logic seems to be: M&M has stated that Manns methods produced mostly hockey sticks, but I’ve found some code that saves the 100 most hockeyfied shapes. Therefore, M&M have deliberately presented a cherry picked sample to ‘prove’ that Manns methods were biased towards hockey sticks.
This is the prosecutors fallacy – because a hockey stick sort is used to select specific data, and a biased sample selection would use a hockey stick sort to mine the data, then M&M falsely presented biased data.
And he’s spent several years of his life on this quest? Wow!

Steve Metzler says:
November 24, 2010 at 2:10 pm
Steve, now you are intentionally being obtuse and obfuscating. Stop it. Read the code in its entirety. I believe you will find that you were mislead. I tried to give you a hint for discovery of this fact, but you either didn’t read my statements or ignored them. More over, you were “straight up” told that this assertion about AR1 not being ran by M&M was blatantly false. Does this look familiar?
“Specifically, “In our red noise discussions, we did two calculations – one with ARFIMA noise and one with AR1 noise. The ARFIMA noise produced pretty hockey sticks but introduced a secondary complication and replications have focused on AR1 examples. To set parameters for the simulation, we calculated AR1 coefficients on the North American AD1400 tree ring network using a simple application of the arima function in R:
arima.coef = arima(x,order=c(1,0,0))“
More to the point, the NAS even states you’ll get hockey-sticks with AR1. So, even if the lies were true, there is no point in furthering the argument, with the exception of yet another patently blatant attempt at character assassination.
Further, you’re changing your argument.
You earlier stated, “He definitely used ARFIMA rather than the AR1(.2) algorithm that Wegman somehow assumed he used.” Well, which is it that he “definitely” used? Was it ARFIMA or AR1(.9)? (please try to tell me they are synonymous.)
Steve, believe it or not, there are some of us here who welcome an intelligent discussion of relevant issues. Character assassination of a guy that was asked by congress for an assessment of a few studies, several years ago, isn’t one in which someone should engage. You don’t like his assessment? Fine, but people don’t have to make stuff up. Engage in the math and science. Really, what’s next from the team? Are they going to ask him when he quit beating his wife?

James Sexton says:

You earlier stated, “He definitely used ARFIMA rather than the AR1(.2) algorithm that Wegman somehow assumed he used.” Well, which is it that he “definitely” used? Was it ARFIMA or AR1(.9)? (please try to tell me they are synonymous.)

Completely missing the point. Wegman says that M&M produced their red noise using AR1(.2). But they *didn’t*. This *fact* is conclusively demonstrated by Deep Climate. For the *umpteenth time*, see the last diagramme in Deep Climate’s article and the accompanying text above and below. I reproduce only a small portion of it below, as pasting large parts of it in here just makes for a tl;dr post:

So Wegman et al’s “compelling” demonstration is shown to be completely false; the biasing effect of “short-centered” PCA is much less evident when applied to AR1(.2), even when viewing the simulated PC1s in isolation. To show the extreme effect claimed by McIntyre, one must use an unrealistically high AR1 parameter. This is yet one more reason that the NRC’s ultimate finding on the matter, namely that “short-centered” PCA did not “unduly influence” the resulting Mann et al reconstruction, is entirely unsurprising.

(my emphasis added)
Further discussion on this point is futile, as you are obviously trying to draw attention away from the elephant in the room: Wegman didn’t do what he was tasked to do. He took obvious shortcuts. Forget about the plagiarism aspect of the text (for the moment). The statistics were also plagiarised directly from M&M with no apparent attempt to verify them independently. If he did, he would have noticed that the red noise could not possibly have been generated using AR1(.2)!

_Jim says:
November 23, 2010 at 8:57 pm
” eadler, November 23, 2010 at 7:47 pm says:
… They used an noise algorithm that generated some persistant noise, used the MBH data and counted how many hockey sticks were generated.
I perused the graphs plotted over at Deep Climat, and I didn’t see any flat lines or even downward turning hockey sticks … ergo the Mann-O-Matic tree-ring data processor still creates upward-bladed hockey sticks when fed random data*?”
You may have perused the graphs, but it is clear that you didn’t read the text. It is pointed out that 10,000 PC1 graphs were created using proxies generated by red noise as intputs. Only 100 looked like hockey sticks based on the hockey stick index values. These were selected and saved as proof that MBH’s procedure would create a hockey stick out of red noise. The other 9,900 graphs were discarded.
Recall M&M’s description of the “sample” PC1 in figure 1 (Wegman et al 4.1):
The simulations nearly always yielded PC1s with a hockey stick shape, some of which bore a quite remarkable similarity to the actual MBH98 temperature reconstruction – as shown by the example in Figure 1.
That’s “some” PC1, all right. It was carefully selected from the top 100 upward bending PC1s, a mere 1% of all the PC1s.
I also note that replications using LabVIEW result in, um, well, the author at that blog states: “Voila! A Hockey Stick from noise…
http://climatesanity.wordpress.com/2009/12/13/michael-mann-averaging-error-demo/
He even includes the source code (for those of you presenting severe allergic reaction to “R”). Just for info, a 30-day trial of the full LabVIEW dev platform can be downloaded from the ni.com website (850 some MB) …
(* subject to some stipulations)”
DC shows using McIntyre’s code that they selected 100 PC1’s that look most like the hockey stick from 10,ooo cases saved those to show everyone and discarded the rest. This shows that ony 1% of the time will the red noise create a PC1 that looks like a hockey stick using the non centered procedure used by MBH. Based on that we are 99% confident that random noise will not create a hockey stick. This is pretty good confidence that the PC1 stick created by MBH using their procedure did not create a hockey stick out of red noise.
The web page you have linked to, focuses only on the first principal component i.e PC1. In fact there is a procedure for selection of the number of principal components that should be used to create a simpler expression that is a fair representation of the data. Using only PC1 is not correct. Actually the noncentered procedure used by MBH in the original paper makes use of PC1 and PC2. Using the centered procedure on the same data, the first 4 principal components are needed, and they will produce a hockey stick graph, even though PC1 in this case does not look like a hockey stick.
The author shows sample graph of PC1 derived from centered and non-centered procedure at the end of the post, with the title “Voila a Hockey Stick from Noise” but doesn’t specifically state how each case was selected. Until he does that, I am not convinced that his graphs prove anything.

Steve Metzler says:
November 24, 2010 at 3:57 pm
James Sexton says:
You earlier stated, “He definitely used ARFIMA rather than the AR1(.2) algorithm that Wegman somehow assumed he used.” Well, which is it that he “definitely” used? Was it ARFIMA or AR1(.9)? (please try to tell me they are synonymous.)
Completely missing the point. Wegman says that M&M produced their red noise using AR1(.2). But they *didn’t*. This *fact* is conclusively demonstrated by Deep Climate. For the *umpteenth time*, see the last diagramme in Deep Climate’s article and the accompanying text above and below. I reproduce only a small portion of it below, as pasting large parts of it in here just makes for a tl;dr post:
So Wegman et al’s “compelling” demonstration is shown to be completely false; the biasing effect of “short-centered” PCA is much less evident when applied to AR1(.2), even when viewing the simulated PC1s in isolation. To show the extreme effect claimed by McIntyre, one must use an unrealistically high AR1 parameter. This is yet one more reason that the NRC’s ultimate finding on the matter, namely that “short-centered” PCA did not “unduly influence” the resulting Mann et al reconstruction, is entirely unsurprising.
Further discussion on this point is futile, as you are obviously trying to draw attention away from the elephant in the room: Wegman didn’t do what he was tasked to do. He took obvious shortcuts. Forget about the plagiarism aspect of the text (for the moment). The statistics were also plagiarised directly from M&M with no apparent attempt to verify them independently. If he did, he would have noticed that the red noise could not possibly have been generated using AR1(.2)!
====================================================
Isn’t that nice, we can agree on a point! I agree, that we should forget about the plagiarism. It isn’t pertinent to the discussion, nor was it ever. I believe I’ve substantially shown where it was never a case of plagiarism. But then, so have thousands of other people.
But, you said,Completely missing the point……., …..No, no I’m not. For the umpteenth time, (while I admit to being vague at some times) this isn’t the case. I’ve asked you to read the code in its entirety. Either you haven’t, or you’ve misinterpreted, or you are of a character of something less than who I wish to engage. I’ve shown you where two of the people that have direct knowledge are, yet, you persist here, engaged with me.
You said,(in your most recent statement) The statistics were also plagiarised directly from M&M … Friend, that’s very hard to reconcile when from the report itself says, “While at first the McIntyre code was specific to the file structure of his computer, with his assistance we were able to run the code on our own machines…” And still, you persist in using the word “plagiarised”[sic], all the while, I’ve repeated this same statement over and over again from Wegman’s report.
You also said, “But they *didn’t*. This *fact* is conclusively demonstrated by Deep Climate.”
No, no it’s not. There are some wild assumptions made by DC. Obviously, they are not correct. See the statements by Steve Mac, Ross Mac, the NAS and anyone else who actually cared to follow the statistics and statements. Or, it could be, that DC found something that no one else has found, in spite of rigorous attempts of refutation. Really, it can’t be both ways. Either the warmistas are really very smart in which most people are wrong, or over the past several years, they couldn’t find what DC did. Even though they were directly refuted. Think about that for a second. Are you telling me it didn’t occur to Mann, Whal, and all the other cast of idiots that Wegman did a statistical no-no? Yeh, ok, tree rings are validated by whom? All the while they couldn’t rebut this contention? Really? Is this your position? Tell DC to keep trying, he’s not quite there.
Again, I’ll point you to your arguments earlier in the conversation. But lately, you said,“If he did, he would have noticed that the red noise could not possibly have been generated using AR1(.2)!”
Again, no, that hasn’t been shown. It was asserted. Nothing more, nothing less.
I could and probably should go on about your changing arguments. And I will if you persist in the unseemly persecution of a person that has no axe to grind in this discussion. Wegman was asked by this nation to give his opinion. He gave it and went back to whatever the hell he was doing. He was never engaged in the climate debate, and after his report he still wasn’t. Is there no low you people won’t stoop to?
The fact is, what is obvious, by your own words, is that you don’t know AR from ARFIMA from ARIMA. And you people worry about what a non-descript statistician thinks about the climate debate several years ago? Why? When reality is kicking your ass throughout history, present, (and by all meaningful predictions), tomorrow.
Pathetic.

Mods, nvm, time-space continuum kicked my butt. Sry!

“”eadler says:
November 24, 2010 at 8:53 pm””
Sorry about this but I’m missing something concerning the issue of de-centering and the 1% of upturned hockey sticks out of a run of 10,000. And this may be a stupid question but:
Assuming (I know, I know – don’t assume anything, still…) that M&M was similiar to what MBH did, in order to deduce their method and code, and they were successful at reproducing the results, hence the critique, why would MBH report a result out of 1% with an uptick when clearly (or not?) the majority of their (MBH) test runs must have produced flat-line or down-tick hockey sticks…many of which had been stored in the now infamous “Censored” htp file?
Actually anyone can answer, I just want to understand.
Thanks,
Gaylon