He certainly understands John Mashey. Here are some examples of the debunking for this whole affair:
Wegman whiners: this post’s for you
Here’s the USA Today article
He certainly understands John Mashey. Here are some examples of the debunking for this whole affair:
Wegman whiners: this post’s for you
Here’s the USA Today article
Mark T says:
November 24, 2010 at 1:00 pm
Thanks for the DC background. I remember before he had his own blog he made several appearances at CA. Seemed to be learning statistics as he went. Sometimes showed some insight, but other times got his head handed to him statistically.
I can’t claim to understand them in detail myself, but even just a little bit of reading through MM05 and reading the thread Jockdownsouth linked to earlier (http://climateaudit.org/2006/09/03/more-tangled-webs/#comments) makes me think of a mechanic, who after you’ve told him your engine blew up 30 seconds after he did an oil change, and you point to the still fresh path of oil from his bay to your car, tries to argue that the problem is that you used the wrong kind of vacuum cleaner the last time you swept out your car.
Apologies again for going quiet. I’m at least 5 hrs ahead of most of you here, and ‘work’ is a 4-letter word that gets in the way sometimes…
Let’s get back to where this discussion all started so that we have a baseline to work from, rather than disparate commenters coming at me from all angles. First, I quote the caption under Figure 4.4 of the Wegman Report (p. 33) which shows 12 ‘hockey sticks’:
This is *very* important. He goes on record saying that the MBH98 algorithm derives hockey stick shaped principle components (PCs) from red noise generated using “[AR(1) with parameter = 0.2]”.
But we can see from McIntyre’s archived code that the ARFIMA algorithm was used instead:
#NOW DO SIMULATION method2<-"arfima" . #SIMULATE RED/WHITE NOISE #arima version (not used here) if (method2=="arima") {b<-array (rep(NA,nyear*n), dim=c(nyear,n) ) for (k in 1:n) {b[,k]<-arima.sim(list(order = c(1,0,0), ar = Data1[k]), n = nyear)} } #arima bracket #arfima version (used here) if (method2=="arfima") {N<-nrow(tree); b<-array (rep(NA,N*n), dim=c(N,n) ) for (k in 1:n) { b[,k]<-hosking.sim(N,Data[,k]) }#k }#arfimaYou can see from the last diagramme in Deep Climate’s article that the NCR was completely unable to replicate McIntyres PC ‘hockey sticks’ using red noise generated from AR(1) with parameter = 0.2. Instead, they had to crank the parameter all the way up to 0.9! This gives a much higher persistence to the noise, and isn’t realistic at all (it gives the noise about a 19-year persistence, whereas AR1(.2) gives only a 1.5-year persistence). So, let’s recap:
1. Wegman says the red noise was generated using AR1(.2), when in fact it was the equivalent of AR1(.9).
2. This proves Wegman didn’t perform proper due diligence with respect to validating McIntyre’s methodology. The PC plot used in Wegman’s Figure 4.1 (p. 30) and all 12 PC plots in Figure 4.4 were taken from McIntyre’s cherry-picked 100 out of 10,000 simulation runs, which could not possibly have used AR(1) with parameter = 0.2. Deep Climate has conclusively shown that to be the case.
Discuss this one point, please. And please don’t keep asking me questions that are answered by the Deep Climate article. It shows that you didn’t bother to read it.
Mark T, John M,
Thanks for the clarification. There is definately something wiffy about DC’s conclusions that don’t stack up. The logic seems to be: M&M has stated that Manns methods produced mostly hockey sticks, but I’ve found some code that saves the 100 most hockeyfied shapes. Therefore, M&M have deliberately presented a cherry picked sample to ‘prove’ that Manns methods were biased towards hockey sticks.
This is the prosecutors fallacy – because a hockey stick sort is used to select specific data, and a biased sample selection would use a hockey stick sort to mine the data, then M&M falsely presented biased data.
And he’s spent several years of his life on this quest? Wow!
Mark T says:
November 24, 2010 at 1:00 pm
James Sexton says:
November 24, 2010 at 12:11 pm
John M says:
November 24, 2010 at 11:47 am
Is it possible poor old DC found the part of the code that picked out the “sample of 100 simulated ‘hockey sticks’…” and convinced himself that’s all MM05 looked at?
It is starting to appear that this is the case. He took code snippets but failed to look at the code in its entirety. This was either real lazy, or intentional. Which, I find humorously ironic.
“……..Really, think about it, DC’s argument is that Steve is wrong because a) Wegman plagiarized the background section regarding Bradley’s work and b) Wegmans analysis was not original…..”
Yeh, I think I mentioned earlier that they were attacking M&M by proxy, which, clearly, they are.
Steve Metzler says:
November 24, 2010 at 2:10 pm
Steve, now you are intentionally being obtuse and obfuscating. Stop it. Read the code in its entirety. I believe you will find that you were mislead. I tried to give you a hint for discovery of this fact, but you either didn’t read my statements or ignored them. More over, you were “straight up” told that this assertion about AR1 not being ran by M&M was blatantly false. Does this look familiar?
“Specifically, “In our red noise discussions, we did two calculations – one with ARFIMA noise and one with AR1 noise. The ARFIMA noise produced pretty hockey sticks but introduced a secondary complication and replications have focused on AR1 examples. To set parameters for the simulation, we calculated AR1 coefficients on the North American AD1400 tree ring network using a simple application of the arima function in R:
arima.coef = arima(x,order=c(1,0,0))“
More to the point, the NAS even states you’ll get hockey-sticks with AR1. So, even if the lies were true, there is no point in furthering the argument, with the exception of yet another patently blatant attempt at character assassination.
Further, you’re changing your argument.
You earlier stated, “He definitely used ARFIMA rather than the AR1(.2) algorithm that Wegman somehow assumed he used.” Well, which is it that he “definitely” used? Was it ARFIMA or AR1(.9)? (please try to tell me they are synonymous.)
Steve, believe it or not, there are some of us here who welcome an intelligent discussion of relevant issues. Character assassination of a guy that was asked by congress for an assessment of a few studies, several years ago, isn’t one in which someone should engage. You don’t like his assessment? Fine, but people don’t have to make stuff up. Engage in the math and science. Really, what’s next from the team? Are they going to ask him when he quit beating his wife?
As to more of the BS that DC floated,
http://climateaudit.org/2010/11/22/escape-from-jonestown/#comment-245719
“This is a bizarre summary. We used 10,000 red noise series to benchmark the RE score, but there was no cherry-picking a 100-series subset; and certainly no use of a white noise benchmark. Nor do W&A offer such an argument. I take it the crackpot who wrote that didn’t include any page references or source citations; more to the point he would be unable to provide any if pressed.”——Ross McKitrick
Steve, both Ross and Steve Mc are right there. Why don’t you just go and ask them for clarification. I’ve found them both to be quit gracious and patient with proper inquisitors. Ask them something about their work, notate it, then check for any inconsistencies between their work and statements. Then you don’t have to attack by proxy, nor do you have to put your stock in the interpretations of their work in 3rd parties such as myself or DC. It’s really not that difficult. Backward engineering is rife with pratfalls. Its best not to attempt it if one doesn’t have to.
James Sexton says:
Completely missing the point. Wegman says that M&M produced their red noise using AR1(.2). But they *didn’t*. This *fact* is conclusively demonstrated by Deep Climate. For the *umpteenth time*, see the last diagramme in Deep Climate’s article and the accompanying text above and below. I reproduce only a small portion of it below, as pasting large parts of it in here just makes for a tl;dr post:
(my emphasis added)
Further discussion on this point is futile, as you are obviously trying to draw attention away from the elephant in the room: Wegman didn’t do what he was tasked to do. He took obvious shortcuts. Forget about the plagiarism aspect of the text (for the moment). The statistics were also plagiarised directly from M&M with no apparent attempt to verify them independently. If he did, he would have noticed that the red noise could not possibly have been generated using AR1(.2)!
Off for some poker. BBL if someone has some more fun!
Steve Metzler says:
“…you are obviously trying to draw attention away from the elephant in the room: Wegman didn’t do what he was tasked to do.”
That there, folks, is a prime example of psychological projection. The ‘elephant in the room’ is the fact that MBH was thoroughly debunked by Wegman, McIntyre and McKittrick – in the Congressional Record.
So of course the alarmist crowd says, “Oh, look over there! A bird!”☺
So the one specific argument relates to Wegman’s use of AR(1)/ARFIMA. It’s a pity you guys had to do so much spewing about “cherry-picking” and the ridiculous claims of “using only 1% of the simulations” (see Figure 2 of MM05) to try to draw McIntyre and McKitrick into this, which threw me a little, but I now see the tactic of attacking by proxy.
Don’t know if McIntyre and McKitrick will go out of their way to defend Wegman, since they’ve already commented on direct attacks on themselves, and Wegman doesn’t do blog defending, so he may be vulnerable here, but for the record:
DR NORTH. We also question some of their statistical methodology, in fact, some of the same claims that were put forward by Dr. Wegman and you will hear some later as well.
I don’t think there is anything dishonest about it or anything like that, but I think that the analyses that the Wegman group did really were–some of those were examined by the statisticians on our committee and I don’t think that we are in any great disagreement about it.
*************************************************
MR. WHITFIELD. And I think that is the whole basis of this hearing because this hockey stick–all of us are concerned about global warming but I do think we have an obligation and responsibility–everyone has latched onto this hockey stick and almost created a panic in a way, and maybe we should be panicked, but I think it is important that we understand how the hockey stick came about, and that is what we are talking about today. Now, Dr. North, do you agree with Dr. Wegman’s centering analysis or not?
DR. NORTH. I do. I think that he is right about that. However, you know, we have to be careful here and not throw the baby out with the water.
**************************************************
CHAIRMAN BARTON. I understand that. It looks like my time is expired, so I want to ask one more question. Dr. North, do you dispute the conclusions or the methodology of Dr. Wegman’s report?
DR. NORTH. No, we don’t. We don’t disagree with their criticism. In fact, pretty much the same thing is said in our report. But again, just because the claims are made, doesn’t mean they are false.
****************************************************
MR. BLOOMFIELD. Thank you. Yes, Peter Bloomfield. Our committee reviewed the methodology used by Dr. Mann and his coworkers and we felt that some of the choices they made were inappropriate. We had much the same misgivings about his work that was documented at much greater length by Dr. Wegman.
_Jim says:
November 23, 2010 at 8:57 pm
” eadler, November 23, 2010 at 7:47 pm says:
… They used an noise algorithm that generated some persistant noise, used the MBH data and counted how many hockey sticks were generated.
I perused the graphs plotted over at Deep Climat, and I didn’t see any flat lines or even downward turning hockey sticks … ergo the Mann-O-Matic tree-ring data processor still creates upward-bladed hockey sticks when fed random data*?”
You may have perused the graphs, but it is clear that you didn’t read the text. It is pointed out that 10,000 PC1 graphs were created using proxies generated by red noise as intputs. Only 100 looked like hockey sticks based on the hockey stick index values. These were selected and saved as proof that MBH’s procedure would create a hockey stick out of red noise. The other 9,900 graphs were discarded.
Recall M&M’s description of the “sample” PC1 in figure 1 (Wegman et al 4.1):
The simulations nearly always yielded PC1s with a hockey stick shape, some of which bore a quite remarkable similarity to the actual MBH98 temperature reconstruction – as shown by the example in Figure 1.
That’s “some” PC1, all right. It was carefully selected from the top 100 upward bending PC1s, a mere 1% of all the PC1s.
I also note that replications using LabVIEW result in, um, well, the author at that blog states: “Voila! A Hockey Stick from noise…
http://climatesanity.wordpress.com/2009/12/13/michael-mann-averaging-error-demo/
He even includes the source code (for those of you presenting severe allergic reaction to “R”). Just for info, a 30-day trial of the full LabVIEW dev platform can be downloaded from the ni.com website (850 some MB) …
(* subject to some stipulations)”
DC shows using McIntyre’s code that they selected 100 PC1’s that look most like the hockey stick from 10,ooo cases saved those to show everyone and discarded the rest. This shows that ony 1% of the time will the red noise create a PC1 that looks like a hockey stick using the non centered procedure used by MBH. Based on that we are 99% confident that random noise will not create a hockey stick. This is pretty good confidence that the PC1 stick created by MBH using their procedure did not create a hockey stick out of red noise.
The web page you have linked to, focuses only on the first principal component i.e PC1. In fact there is a procedure for selection of the number of principal components that should be used to create a simpler expression that is a fair representation of the data. Using only PC1 is not correct. Actually the noncentered procedure used by MBH in the original paper makes use of PC1 and PC2. Using the centered procedure on the same data, the first 4 principal components are needed, and they will produce a hockey stick graph, even though PC1 in this case does not look like a hockey stick.
The author shows sample graph of PC1 derived from centered and non-centered procedure at the end of the post, with the title “Voila a Hockey Stick from Noise” but doesn’t specifically state how each case was selected. Until he does that, I am not convinced that his graphs prove anything.
Steve Metzler wrote:
““And, of course, you probably don’t have a problem with someone drawing the conclusions of their report from a hand-picked sample representing only 1% of the total 10,000 simulation runs. Does that seem like proper use of statistics to you, or does it in fact seem like someone with an agenda?”
Well if M&M did in fact, hand pick their sample then I would agree with your conclusion, but you are merely speculating. How do I know this? Because M&M doesn’t say anywhere that they “hand picked a sample.” So you go to DeepClimate and read some opinion that because 1% was used, then this must have been hand picked – an opinion that you now regurgitate as fact.
For anybody who has any background in statistics, random sampling is a standard and accepted technique to gain information about a population. A sample of 1% from 10,000 is 100. As long as this was selected randomly – ie by using a random number generator – then it is likely to represent the whole population to a high accuracy. There is nothing sinister about this, nor does it point to an ‘agenda’ as you claim.
Where is your evidence that they hand-picked the sample?”
In his post DC goes through McINtyre’s code and demonstrates that the samples were pulled if they showed a values in the calibration period that were more than one standard deviation above the data.
Now, was a random sample of these PC1s saved? Or perhaps just the first 100 (which would also be reasonably random)? Not quite.
############################################
#SAVE A SELECTION OF HOCKEY STICK SERIES IN ASCII FORMAT
order.stat<-order(stat2,decreasing=TRUE)[1:100]
order.stat<-sort(order.stat)
hockeysticks<-NULL
for (nn in 1:NN) {
load(file.path(temp.directory,paste("arfima.sim",nn,"tab",sep=".")))
index<-order.stat[!is.na(match(order.stat,(1:1000)+(nn-1)*1000))]
index<-index-(nn-1)*1000
hockeysticks<-cbind(hockeysticks,Eigen0[[3]][,index])
} #nn-iteration
dimnames(hockeysticks)[[2]]<-paste("X",order.stat,sep="")
write.table(hockeysticks,file=file.path(url.source,
"hockeysticks.txt"),sep="\t",quote=FALSE,row.names=FALSE)
The first line sorts the set of PC1s by descending HSI and then copies the first 100 to another array. That array is then sorted by PC1 index so that each PC1 selected for the archive can be retrieved from the appropriate temporary file and saved in ASCII format.
Steve Metzler says:
November 24, 2010 at 3:57 pm
James Sexton says:
You earlier stated, “He definitely used ARFIMA rather than the AR1(.2) algorithm that Wegman somehow assumed he used.” Well, which is it that he “definitely” used? Was it ARFIMA or AR1(.9)? (please try to tell me they are synonymous.)
Completely missing the point. Wegman says that M&M produced their red noise using AR1(.2). But they *didn’t*. This *fact* is conclusively demonstrated by Deep Climate. For the *umpteenth time*, see the last diagramme in Deep Climate’s article and the accompanying text above and below. I reproduce only a small portion of it below, as pasting large parts of it in here just makes for a tl;dr post:
So Wegman et al’s “compelling” demonstration is shown to be completely false; the biasing effect of “short-centered” PCA is much less evident when applied to AR1(.2), even when viewing the simulated PC1s in isolation. To show the extreme effect claimed by McIntyre, one must use an unrealistically high AR1 parameter. This is yet one more reason that the NRC’s ultimate finding on the matter, namely that “short-centered” PCA did not “unduly influence” the resulting Mann et al reconstruction, is entirely unsurprising.
Further discussion on this point is futile, as you are obviously trying to draw attention away from the elephant in the room: Wegman didn’t do what he was tasked to do. He took obvious shortcuts. Forget about the plagiarism aspect of the text (for the moment). The statistics were also plagiarised directly from M&M with no apparent attempt to verify them independently. If he did, he would have noticed that the red noise could not possibly have been generated using AR1(.2)!
====================================================
Isn’t that nice, we can agree on a point! I agree, that we should forget about the plagiarism. It isn’t pertinent to the discussion, nor was it ever. I believe I’ve substantially shown where it was never a case of plagiarism. But then, so have thousands of other people.
But, you said,Completely missing the point……., …..No, no I’m not. For the umpteenth time, (while I admit to being vague at some times) this isn’t the case. I’ve asked you to read the code in its entirety. Either you haven’t, or you’ve misinterpreted, or you are of a character of something less than who I wish to engage. I’ve shown you where two of the people that have direct knowledge are, yet, you persist here, engaged with me.
You said,(in your most recent statement) The statistics were also plagiarised directly from M&M … Friend, that’s very hard to reconcile when from the report itself says, “While at first the McIntyre code was specific to the file structure of his computer, with his assistance we were able to run the code on our own machines…” And still, you persist in using the word “plagiarised”[sic], all the while, I’ve repeated this same statement over and over again from Wegman’s report.
You also said, “But they *didn’t*. This *fact* is conclusively demonstrated by Deep Climate.”
No, no it’s not. There are some wild assumptions made by DC. Obviously, they are not correct. See the statements by Steve Mac, Ross Mac, the NAS and anyone else who actually cared to follow the statistics and statements. Or, it could be, that DC found something that no one else has found, in spite of rigorous attempts of refutation. Really, it can’t be both ways. Either the warmistas are really very smart in which most people are wrong, or over the past several years, they couldn’t find what DC did. Even though they were directly refuted. Think about that for a second. Are you telling me it didn’t occur to Mann, Whal, and all the other cast of idiots that Wegman did a statistical no-no? Yeh, ok, tree rings are validated by whom? All the while they couldn’t rebut this contention? Really? Is this your position? Tell DC to keep trying, he’s not quite there.
Again, I’ll point you to your arguments earlier in the conversation. But lately, you said,“If he did, he would have noticed that the red noise could not possibly have been generated using AR1(.2)!”
Again, no, that hasn’t been shown. It was asserted. Nothing more, nothing less.
I could and probably should go on about your changing arguments. And I will if you persist in the unseemly persecution of a person that has no axe to grind in this discussion. Wegman was asked by this nation to give his opinion. He gave it and went back to whatever the hell he was doing. He was never engaged in the climate debate, and after his report he still wasn’t. Is there no low you people won’t stoop to?
The fact is, what is obvious, by your own words, is that you don’t know AR from ARFIMA from ARIMA. And you people worry about what a non-descript statistician thinks about the climate debate several years ago? Why? When reality is kicking your ass throughout history, present, (and by all meaningful predictions), tomorrow.
Pathetic.
Mods, there is a post out there from me, would you? Its a bit aged by now.
Mods, nvm, time-space continuum kicked my butt. Sry!
Calling it a night, Happy Thanksgiving to all! It is a good time to consider all of the bountiful blessings that are bestowed upon us.!
“”eadler says:
November 24, 2010 at 8:53 pm””
Sorry about this but I’m missing something concerning the issue of de-centering and the 1% of upturned hockey sticks out of a run of 10,000. And this may be a stupid question but:
Assuming (I know, I know – don’t assume anything, still…) that M&M was similiar to what MBH did, in order to deduce their method and code, and they were successful at reproducing the results, hence the critique, why would MBH report a result out of 1% with an uptick when clearly (or not?) the majority of their (MBH) test runs must have produced flat-line or down-tick hockey sticks…many of which had been stored in the now infamous “Censored” htp file?
Actually anyone can answer, I just want to understand.
Thanks,
Gaylon
eadler,
“In his post DC goes through McINtyre’s code and demonstrates that the samples were pulled if they showed a values in the calibration period that were more than one standard deviation above the data.”
Yes, thank you for that, I got it one wrong. As John M explained
“MM05 do indeed define a “hockey stick index”.
My bad.
However, he added “as I said before, Figure 2 in their paper is for all 10,000 calculations.”
The assertion being made is that M&M fraudently mined hockey stick shapes and presented this as conclusion that Manns methods always produce hockey sticks. As Ross has explained on this “We used 10,000 red noise series to benchmark the RE score, but there was no cherry-picking a 100-series subset; and certainly no use of a white noise benchmark.”
Someone is drawing the conclusion that because hockey stick shapes were saved to a dataset this proves they were presented as evidence that Mann’s methods produce only hockey sticks.
Hi Gaylon,
You asked:
“…why would MBH report a result out of 1% with an uptick when clearly (or not?) the majority of their (MBH) test runs must have produced flat-line or down-tick hockey sticks…many of which had been stored in the now infamous “Censored” htp file?”
The answer to your question appears to be that nothing matters in mainstream climate science, except keeping the AGW scare going in high gear in order to keep the grant money flowing.
For example, Mann’s censored file, which was discovered by chance by Steve McIntyre, debunked the MBH hockey stick. It is crystal clear that specific data was hidden [censored] by MBH, so the remaining proxies would generate an alarming hockey stick shape.
In normal times being caught in that kind of dishonesty would have put an end to the matter, and resulted in the destruction of Mann’s reputation. The “censored” file shows conclusively that MBH deliberately used bad proxies in order to generate false results: the infamous Hockey Stick chart used extensively by the UN/IPCC. [For more background I recommend A.W. Montford’s The Hockey Stick Illusion, available along the right hand sidebar at the top of this page.]
But these are not normal times, and Mann’s clique understands that. Thus, his chart using the Tiljander upside-down proxy was recently published, despite Mann being previously informed that the sediments used were not pristine, but were the result of grading, which put older soil on top of newer sediment; by using an honest proxy the resulting chart would have shown a downward pointing hockey stick, rather than what Mann wanted: another scary looking rising hockey stick.
Mann was informed of this beforehand, but he published the false results anyway – knowing that a compliant media would not report it, nor would the corrupt climate peer review system of tame journal boards, nor would fellow travelers on the same grant gravy train, including referees and universities. Only the blogosphere has exposed Mann et al. as scientific charlatans who lie outright for grant money, fame, and endless trips to vacation spots around the world at the public’s expense.
Thanks Smokey, I appreciate your response and in closing…that’s what I suspected.
Have a great Turkey Day!!
Thank you, Gaylon. Happy Thanksgiving to you, too…
…and to everyone who reads this great science site.
eadler made this assertion indicating his lack of understanding:
You and DC alike don’t understand much about anything when it comes to this subject. There’s no such thing as a hockey stick index nor is there some way to subjectively (or objectively) sort, as the code snippet indicates, on “PC1’s that look most like the hockey stick.” The sort was on stat2, as I’ve already pointed out, which is likely variance. Variance, btw, is also the parameter on which any vectors that result from an eigenvalue-like decomposition are typically sorted.
Before you comment further on this, or any related statistical subject, I’d suggest you actually make an attempt at understanding the underlying concepts before passing on another’s information as if it is correct. You don’t know enough to critically assess what DC is doing, yet you believe him anyway, probably because you want what he says to be true. Bad new: DC doesn’t know what he’s talking about.
Mark