Steve McIntyre comments on Wegman’s work and Deep Climate’s recent dissing of the Said et al paper. It seems there’s a clique of too close for comfort relationships that may have tainted peer review:
Lost in the recent controversy over Said et al 2008 is that the Climategate documents provided conclusive evidence of the hypothesis originally advanced in the Wegman Report about paleoclimate peer review – that reviewers in the Mann “clique” had been “reviewing other members of the same clique”.
I won’t attempt to analyze the plagiarism issues today (I will return to this on another occasion), other than to say that some recent literature on the topic attempts to distinguish between degrees of plagiarism e.g. Bouville, Clarke and Loui.
In addition, contrary to recent false claims by USA Today, Said et al 2008 was not “a federally funded study that condemned scientific support for global warming”. It does not mention global warming nor even climate. Nor is Said et al 2008 a “cornerstone” of criticisms of either Mann or IPCC as Joe Romm falsely claimed. For example, it has never been referred to or discussed at Climate Audit even in comments. (Nor at any other climate blog, to my knowledge.)
…”The Wegman Report was vindicated on its hypothesis about peer-review within the Mann ‘clique’.
‘The Wegman Report hypothesized, but were unable to prove, that reviewers in the Mann ‘clique’ had been ‘reviewing other members of the same clique’. Climategate provided the missing evidence, Climategate documents showed that clique member Phil Jones had reviewed papers by other members of the clique, including some of the articles most in controversy – confirming what the Wegman Report had only hypothesized”
Read it at: Climategate Documents Confirm Wegman’s Hypothesis
An important distinction to keep in mind is that while the Said et al paper has bee retracted (temporarily we hope) this in no way affects the Wegman report to Congress or its conclusions (i.e. it would not automatically cause a retraction of the Congressional report, which operates under a different scenario of rules and public record).
nice work
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
Seems to me that novel statistical techniques should be reviewed by statisticians, rather than statistical wannabes from within the clique.
Said et al in fact conceded they had no evidence of actual “clique” reviewers. They simply listed people who had previously coauthored papers. Well, we knew papers were coauthored.
Steve M had a few cases where people in this small field had reviewed papers written by people in their “clique” (which simply means people with whom they have at other times been coauthors). Experts are few – that is not surprising.
But it also fits with the way reviewers are selected. A journal editor will seek a range of opinions. The reviewers will usually include someone who shares the outlook of the author, and a likely critic. To save time, many journals invite authors to nominate people that they would like to see as referees.
For example, CSDA, where Said et al appeared, says:
“Please submit, with the manuscript, the names, addresses and e-mail addresses of 3 potential referees. Note that the editor retains the sole right to decide whether or not the suggested reviewers are used.”
That’s a standard phrase. If you google it, you get 18000 hits from journals which make that invitation.
Note the caveat. Only a few, if any, of the reviewers will be from that list. And of course authors will nominate people they have worked with.
Wheels within wheels. Twas ever thus when rules are relaxed.
Anthony:
For those who want to refresh their memories, the Climategate
e-mails can be reviewed at:
http://www.eastangliaemails.com/index.php
The original 2006 Wegman report to Congress in PDF format is at:
http://www.uoguelph.ca/~rmckitri/research/WegmanReport.pdf
But pal reviews are so warm and fuzzy …
Keith Minto says: May 24, 2011 at 12:39 am
“In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?”
Very sensible point. But perhaps that is a reason to see this as a potentially general problem for scientific peer review. It’s almost inevitable (given the evidence of climategate) that similar “groupthink” exists in many other areas of science.
In the end global warming will be just another hysteria consigned to the dustbin of history, but the problems within science are unlikely to be addressed unless society forces science to really look at the abysmal mess that it seems to be in, in areas like climate “science” and does something about it.
Perhaps 30% of all reviewers should be from outside the discipline and not connected with the other reviewers or author. Their specific scientific knowledge may not be as good, but they are far more likely to asked the simple questions that challenge the groupthink at a stage where it can be tackled and corrected (and not at the paranoia stage when all criticism is viewed as some kind of conspiracy to destroy the discipline)
“Old Boys Club”
Kieth,
but doesn’t the IPCC claim membership in the thousands ? also we are talking about less than 2 dozen of the top names who where participating in this nonsense …
I have always maintained that the *only* surprising thing in Climategate to followers of ClimateAudit was the fact of the release. The emails mesh so perfectly with Steve’s blog up to then that, had CRU not acknowledged them, one could be forgiven for suspecting that they had been faked up using ClimateAudit as an outline.
The Alarmists talk about climate science as if it requires a different logic and mathematics than other sciences. It doesn’t. They can’t explain a lot of their stuff because it is hand-waving to get them to their predetermined conclusion.
BTW,
Who is Deep Climate?
“Keith Minto says:
May 24, 2011 at 12:39 am
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?”
That would be true if this was actually a small field of study – but it isn’t. If you want to self define “climate-change science” then you can use this argument, but there is no such thing as “climate change” as a scientific discipline. Michael Mann, for example, is a paleontologist – there are plenty of those to choose from to review his work, but no, we have to get someone who is already part of the “climate-change” team. Why? Because we (the journal editors who are also part of the team) have decreed that he is a “climate-change scientist”.
This is at the heart of the alarmist strategy – create a field in which you (the team) are the only experts and therefore only you can properly pontificate. Then get some fellow-travellers to keep referring to you as the experts (social scientists like Oreskes and “journalists” like Romm) and dismissing everyone else as not qualified. Hey presto, not only do you become the only authoritative voice, but you also make sure all of the grant money comes to you and people you accept as part of the team.
As an academic scientist, I often reviewed papers that were outside my direct experience, but to which I was asked to bring my expertise. In some cases, I could only comment on parts of the paper, because that was all I had experience of, but that was what I was asked to do by the editor. This is peer review done properly. I have seen this most clearly when identifying reviewers for PhD theses: Supervisors I have worked with have gone out of their way to find someone with specialized knowledge of a critical topic, but often with a different background, in order to bring something extra to the review process. The students might not necessarily like it, but it makes for a better thesis.
A few people here have commented that this a “small field” with “few experts”.
I dont know about your planet, but here on Earth, climate research is a muti-billion dollar industry. It is the most overfunded and overstaffed field in all of science.
Keith Minto says, May 24, 2011 at 12:39 am:
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
No, in a small field, it is a bit incestuous. However, going back some 30 years, it looked to me (as an outsider) that this also encouraged proper discussions between reviewers and authors, which made papers better.
Additionally – scientists in small fields of study did not and do not expect the developed world to dismantle their economy, tax their citizens into oblivion and generally expect supranational organisations to administer life on earth to an extent that everybody can look forward to a prehistoric lifestyle …
I would ask a more learned person to weigh in, but…
Keith Minto says:
May 24, 2011 at 12:39 am
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
On many of the papers, is there not enough that a Statistician or Physicist could, and maybe should, participate in the peer review process? If so, the field is certainly not small.
Get serious, it always comes back to Mann, Jones, Briffa and a couple of others. It’s not even a clique, they could all fit in a closet.
Nick Stoke said:
Only a few, if any, of the reviewers will be from that list. And of course authors will nominate people they have worked with.
Weelll – that form for authors where potential peer reviewers should be suggested is always difficult for me. because I’m trying to find someone with whom I did not work, or at least do not work for several years. More fool me…
“Jeff says:
May 24, 2011 at 4:29 am
Kieth,
but doesn’t the IPCC claim membership in the thousands ? also we are talking about less than 2 dozen of the top names who where participating in this nonsense …”
Good observation Jeff. The alarmists seldom see the knife that they cut their own throats with.
“An important distiction to keep in mind is that while the Said et al paper has bee retracted (temporarily we hope) this in no way affects the Wegman report to Congress or its conclusions.”
Looks like a spelling bee stung two words in that sentence, “distinction” rather than “distiction”, and “been” rather than “bee”.
Interesting article.
The Mann Clique… sounds like some deviationalist Marxist-Leninist group. Must notify the NKVD to start making arrests.
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
Isn’t there a code of ethics. I can’t do an unbiased review because Mike and I wrote these papers together…
Dave says:
“It’s not even a clique, they could all fit in a closet.”
True dat. However, Wegman’s statistics definition of a clique is the designated group being studied.
And the definition of a claque is: A group of people hired to applaud (or heckle) a performer or public speaker.
• a group of sycophantic followers
Claque: Romm, Schmidt, Mann, Cook, and other hired alarmist bloggers in particular, and their sycophantic true believer followers in general.
[Sycophant: A person who acts obsequiously toward someone in order to gain advantage; a servile flatterer.]
Norm814 says:
May 24, 2011 at 6:38 am
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
Isn’t there a code of ethics. I can’t do an unbiased review because Mike and I wrote these papers together…
It isn’t a small field; it is actually far broader than the “team” pretends it is. They us data and -more or less- theory from numerous disciplines, but fail badly to actually consult within those disciplines on a regular basis to insure their thinking is in a trap they may be unaware of, but which practitioners of the borrow discipline would be well aware of.
A dead horse is being beaten here. The Said et al paper is dead. Note to Wegman: don’t hire a former grad student to write a paper on “reviewing other members of the same clique”, especially one who seems to have sub-contracted part of that paper to another grad-student who uses Wikipedia as a cut-and-paste reference source.
I believe McIntyre used to work in the area of mineral exploration. He should be equally shocked to see the “clique” that exists in that industry – one that controls far more money than does climate science. Without putting the level of “review clique” into context, it is impossible to single out climate “science” as a singular problem. The entire scientific industry is likely rowing in the same chummy rotten boat.
A WUWT reader posted a link to the Said et al paper, which I clicked on. You have the option of clicking to Said’s other papers or to her university. I clicked the latter. Figuring prominently on that page was the number of papers published by that university over the years as a rapidly growing bar graph. This is the core of the problem – quantity of publication, best accomplished in a clique.
Not even McIntrye can rescue this one. The horse is dead. Move on.
Steve from Rockwood says:
May 24, 2011 at 7:50 am
“A dead horse is being beaten here. The Said et al paper is dead…..”
==============================================
Steve, from my perspective, you’ve entirely missed the point. Not to mention, your analogy to mineral exploration shows that your view is a bit twisted.
As Steve Mc points out, it isn’t about the Said paper. It was never a point of contention or even casual conversation. Ok, so it gets retracted, so what? Does it mean the information is invalidated? Does it mean the Wegman report was wrong because some undergrad did a “copy and paste”? Nothing could be further from the truth. The fact is, the Wegman report showed the ‘incestuous’ relationship of the cliararmists, and regardless of the critiques and howls of protest, the climate-gate e-mails bear the posit out. This proves that by-in-large, climate studies aren’t peer review, but rather pal review. I’ve probably typed this over one hundred times, and I’ll type it again. Having like minded ideologues agreeing with each other isn’t science and the e-mails show this is exactly what has occurred.
As to your mineral exploration industry being as cliquish as climate science, I know I may be a bit naive, but I don’t typically regard science/academia in the same light as industrial mineral exploration.
McIntyre: Climategate Documents Confirm Wegman’s Hypothesis
Posted on May 24, 2011 by Anthony Watts
Steve McIntyre comments on Wegman’s work and Deep Climate’s recent dissing of the Said et al paper. It seems there’s a clique of too close for comfort relationships that may have tainted peer review:
A classic case of that being Said et al. 2008 where two of the authors are editors of the journal and friends of the the editor who ‘reviewed’ the paper, which was outside his area of expertise, in 5 days! It also refuted Wegman’s supposition that coauthorship with graduate students was healthier because the student would be unlikely to be an editor. Wegman should have followed his own advice and involved other authors familiar with the fields he was commenting on; instead of using the notes of a student who had attended a course with an expert on social network analysis and plagiarizing from Wikipedia perhaps he should have invited Carley to help him? Similarly instead of plagiarizing material on dendroclimatology it would have been better to involve someone familiar with the material.
Sounds like a talking point straight out of moveon.org. I am sure they have moved on (not) as perhaps you should since you have declared it dead. However, for the rest of rational intelligent thinking adults, the horse is feeling his oats.
You scratch my back, I’ll scratch yours. You pass my study through review, I’ll pass yours.
A big part of the problem with peer review is that it is secret, and secrecy breeds mistrust. There is no way to know when a study is published if the reviewer was truly independent or not.
Secrecy is appropriate during the review, to prevent the author from influencing the reviewer, but once the study is published the reviewer and their comments should become part of the formal record.
An important part of restoring confidence in Climate Science is to restore the credibility of the peer review process. This requires that the process itself be sunmitted to the same scientific review as any other scientific study.
The first step in this process is formal, reviewed studies of the peer review process. Rather than building confidence, the withdrawl of studies on peer review leaves a vacuum.
In this vacuum, the Climategate Papers become the most authorative reference. It should be no surprise that public confidence in Climate Science has been lost. Climate Science has not corrected the flaws revealed by Climategate, and continues to fight attempts to correct the flaws revealed.
You would think that if Climate Science truly was confident in their findings, scientists like Phil Jones would welcome an independent review of their work. Instead, what Climategate revealed is that Climate Scientist have fought very hard to prevent independent review.
The average person on the street has no problem understanding what this means. When someone doesn’t want to have their work reviewed for errors by anyone but their close friends and associates, when the won’t publish their data and methods, when they seek to silence opposing points of view, this is something the public understands.
This has nothing to do with the theory of climate change, it has to do with the underlying principles of the scientific method. Inherent in every theory there is prediction. If the theory is correct, then this is what we should observe.
All successful theories follow this formula. Relativity gained acceptance not because it sounded good, but rather it made a number of predictions for events that had not yet been observed or explained. Thus it provided a test for its correctness. As we have looked for these events they have been found to occur within the limits predicted by relativity. Thus, with each observation we conclude that relativity provides a good description of reality and our confidence in the theory builds.
Climate science has not followed this formula. Climate science AGW greenhouse theory predicts a tropic hot spot and ever increasing temperature with increasing CO2, largely based on computer models.
When temperatures moderated and departed from the model predictions post 2000, this should have signalled that the theory was in error. When the tropic hot spot was found not to exist, this should have signalled that the theory was in error.
In every other branch of science, when your theory does not match observation, this is a signal that your theory is wrong and needs formal review. However, this is not what occurred in climate science. Instead, climate science argues that since the theory matches the models, the theory is correct, regardless of observation. This suggests that the review process in climate science has failed and that climate science continues to resist calls for reform of the process.
“An important distinction to keep in mind is that while the Said et al paper has bee retracted (temporarily we hope) this in no way affects the Wegman report to Congress or its conclusions (i.e. it would not automatically cause a retraction of the Congressional report, which operates under a different scenario of rules and public record).”
This is true to a large degree. However, I expect Rep. Henry Waxman, the ranking member of the Energy and Commerce Committee, to make some political hay over this.
The closing paragraph in Deep Climate’s May 15th article:
“There will now be even more pressure on George Mason University to finally resolve the myriad misconduct complaints launched against Wegman and his team. Not to mention increased calls for the retraction of all the other examples of the pair’s dubious scholarship – including the Wegman et al report to Congress itself.”
http://deepclimate.org/2011/05/15/retraction-of-said-wegman-et-al-2008-part-1/#more-3299
Since May 15th., Waxman has sent two letters to the Energy and Commerce Chair requesting hearings. One wrt Koch Industries financial interest in Canadian tar sands and the Keystone pipeline. Another regarding the studies released by the NAS and the Vatican.
http://democrats.energycommerce.house.gov/index.php?q=news/reps-waxman-and-rush-urge-committee-to-request-documents-from-koch-industries-regarding-keyston
http://democrats.energycommerce.house.gov/index.php?q=news/reps-waxman-and-rush-request-hearing-on-climate-change-science
Clearly, the warmist community has his ear, and he is aware of the Wegman/Said issue. Though it is unlikely that he will be successful in having the Wegman Report stricken from the public record, it will make for some interesting political theater when and if the hearings occur. I’m looking forward to it.
ferd berple,
Excellent comments. It is clear that climate pal review will not be changed, because if reviewers and referees names were published along with the paper they approved through peer review, the incestuous gaming of the system would end.
And I note for the record that Phil. is not disputing Wegman et al’s conclusions, but rather he is merely arm-waving over the absence of a footnote. Pontificating about the mote in someone else’s eye, while there is a beam in your own eye is hypocritical. Wegman made no money on his Report to Congress, while many $billions in grant money are distributed every year based on the secretive climate pal reviews. Which is worse?
Waxman – a democrat-socialist firmly in the grip of Pelosi’s and Obama’s CAGW tax-and-control community – finds himself with much less power now that the realists (well, mostly realists) in the republican party have control of the House of Representatives.
Waxman’s “democrats” ruled like tyrannical despots under Pelosi from the 2006 elections through through December 2010, propagandizing and even editing (and denying republican representatives the right to mail their OWN political letters to their OWN districts!) all correspondence from all House offices. But now? He will find it harder to manipulate things to his own advantage.
“less than 2 dozen of the top names who where participating in this nonsense”
Unfortunately, the “nonsense” is the destruction of the scientific method that hundreds if not thousands before us fought to achieve, often at the cost of their freedom and their lives.
Subversion of peer review, inversion of the null hypethesis, supplanting observation with models as the ultimate validation of theory. These pose a significant risk to the scientific method and rational thought.
As night follows day, individual freedoms are based on rational thought. All that is required to subvert individual freedom and personal liberty is to subvert rational thought. At the point at which the irrational is accepted to be true, all things become possible. The elimination of the “inferior” from the earth, with confiscation of their lands and property for the benefit of all. This is the lesson of history when we accept that illogic is logical.
The IPCC’s stated position is that CO2 must be responsible for the warming post 1950 because they have studied the question and no other explanation can be found. This is the exact same explantion for witchcraft. When something happens that cannot be explained, it must be due to witches.
The IPCC’s conclusion is fundamentally flawed science that should have been rejected by Climate Science. The fact that Climate Science has not rejected the IPCC hypothesis points to a fundamental problem at the heart of climate science.
Why do witches burn? Because they are made of wood. And does wood sink in water? No it floats. And what else floats in water? A duck. Why? Because it weighs the same as wood. Therefore? A witch weighs the same as a duck.
Isn’t the broader point that traditional peer-review may not be up to the task of reviewing papers which will form the basis of major policy decisions made by governments worldwide? On a similar point, Roger Pielke, Sr. has often made the point that no one would permit decisions regarding the safety and efficacy of medications to be made by the same people who developed the medication. Why do we allow the lead authors of the IPCC to write chapters that review their own publications?
Duke C.: Rep Waxman ranking member , Energy and Commerce Committee. This should read ranking democrat party member of said committee. Thankfully, since the 2010 elections the democrats have been relagated to the irrelevancy they deserve and Waxmans comments carry very little authority.
And your statement exemplifies the mindset of climatologists in the “clique.” When I list prospective reviewers for a paper, I’m careful to not choose people I have worked with as it is seen as a rather unprofessional attempt to stack the deck.
The argument that climatology is such a small discipline that authors must choose people they work with doesn’t wash. [The famous] Dr. Brent Yamal, in a recent interview stated “There are literally thousands of climatologists in North America”. With Yamal confirming that the field isn’t so small, surely a reputable climate scientist can think of at least a few names to choose from without resorting to selecting people they have worked with.
Smokey says: May 24, 2011 at 9:18 am
“And I note for the record that Phil. is not disputing Wegman et al’s conclusions”
Well, the conclusion in the abstract was:
“We conjecture that certain styles of co-authorship lead to the possibility of group-think, reduced creativity, and the possibility of less rigorous reviewing processes.”
I suppose the paper does demonstrate the “possibility of less rigorous reviewing processes”.
But otherwise I can’t see how they otherwise demonstrated the conclusions. Can you explain?
Nick Stokes,
Are you seriously claiming that there is no group-think among Mann’s clique? Or that pointing out the liklihood of less rigorous reviewing among Mann’s pals is not a conclusion? Really?
I suggest you read ferd berple’s comments at 8:59 above, and give them some serious thought.
Not even McIntrye can rescue this one. The horse is dead. Move on.
its called an autopsy. For that you need a forensic specialist like steve.
The retracted Said paper has two issues.
A. the problematic scholarship
B. the claims.
The claims are vindicated by the climategate mails and ironically by the paper itself. Pal review is neither Necessary nor sufficient. The scholarship? If you carefully look at the audit trail you will see the problematic section of Said (section 1) is word for word
lifted from the disertation of a co author. Sharabati. This disertation borrows from the wegman report. The problematic section of the wegman report was written by a grad student. None of the problematic scholarship impinges on the science.
The coverage of the Said paper has numerous problems.
1. the characterization of the paper itself
2. its importance in climate science
The retraction has a severe problem. That has not even been discussed yet.
James Sexton says:
May 24, 2011 at 8:41 am
Steve from Rockwood says:
May 24, 2011 at 7:50 am
“A dead horse is being beaten here. The Said et al paper is dead…..”
==============================================
Steve, from my perspective, you’ve entirely missed the point. Not to mention, your analogy to mineral exploration shows that your view is a bit twisted.
==============================================
James,
I agree entirely with your point that cut-and-paste does not invalidate a paper. But I think you’ve missed my point: Wegman lost his credibility when he coauthored a paper with a former student who was one of the editors for the journal they submitted to where the paper dealt with “pal-review” over “peer-review”. Isn’t the pot calling the kettle black here?
Also with regards to mineral exploration, I have published a few papers in the academic realm (which I was referring to) and I can point to several areas where academic papers have been hijacked by specific people who use publications as a base of power and not so much to inform others. IMO more people are publishing now for personal gain than to enlighten. And yes in this regard I believe you are being naive.
One of the most respected geophysicists around is Dr. Gordon West. He has many publications including a book known simply as Grant & West. If you subjected West to Wegman’s methods he would appear to be very incestuous. But it is because he mentored so many (excellent) students, many of whom went on to great careers and in return coauthored papers with him. My point here is that Wegman’s review shows climate science to be incestuous (which is certainly is) but it tells us nothing of the motives of the authors (I think we know what their motives are). If this method were applied to any other group you would see the same results. So being critical of pal review on its own is not so meaningful.
The Said et al paper is dead.
I can see where Steve McIntyre gets frustrated with you. Again, you are dodging the question and trying to change the subject. The Wegman paper (or even the Said paper) did not prove group think – the climategate emails DID. And that was the point of Steve McIntyre’s original article and this followup. So your statement above is pure non-sequitur.
PhilJourdan says: May 24, 2011 at 12:48 pm
“Again, you are dodging the question and trying to change the subject.”
I actually think I’m good at staying on topic and answering questions. The issue of Smokey that I was explicitly addressing was:
“And I note for the record that Phil. is not disputing Wegman et al’s conclusions”
and I’m saying they haven’t demonstrated any conclusions. And inviting anyone who disagrees to explain how they have.
Having read the responses of Nike Stokes here and at CA on this subject it is clear that he is simply employing obfuscation, one of the primary means of deception utilized by climate fear advocates, in trying to hide the truths uncovered by Mr. McIntyre in his most recent post. Game, set and match to Mr. McIntyre.
Nick Stokes is like a canary in a coal mine – when he starts commenting it means some AGW lie is being demolished.
Good work Steve.
Nick Stokes,
Your ‘conclusion’ question was asked & answered. You just didn’t like the conclusion or the answer.
You also say: “I actually think I’m good at staying on topic and answering questions.”
Not really. I asked the following question up-thread:
“Phil. is not disputing Wegman et al’s conclusions, but rather he is merely arm-waving over the absence of a footnote. Pontificating about the mote in someone else’s eye, while there is a beam in your own eye is hypocritical. Wegman made no money on his Report to Congress, while many $billions in grant money are distributed every year based on the secretive climate pal reviews. Which is worse?”
No answer to that one.
It has been a while since I was in academia but the general understanding was that “if you lift from one source it is plagiarism, if you lift from many sources it is research.”
Academic humour is somewhat dry…
Steve from Rockwood says:
May 24, 2011 at 12:31 pm
======================================
Ok, I see where you’re coming from, sorry it wasn’t obvious to me earlier.
Yes, it is the pot calling the kettle…. I guess. And yes, the Said paper is dead, but as Steve Mc points out, it was never alive, at least in the climate discussion. Prior to this little dust-up, I doubt many were aware of its existence. It was neither pertinent to the skeptic camp nor the alarmist. The information contained within, OTOH is, was, and will continue to be part of the discussion.
Steve, I’ve commented on this before, but I doubt that people hang on to my words so I’ll restate. It seems to me, there are two separate discussions which are being conflated. One, is the academic perspective. There are likely about half of the commentators here with little experience or regard for that part of the discussion. Many are here to discern the truth about our current climate and whether or not people making alarming claims are to be believed. Many of us don’t care if Wegman dotted his ‘i’s in writing background material for his posits or if he wrote them or not in his testimony to congress. It simply isn’t pertinent to our involvement in the discussion.
One of the problems many of us have is coming to the realization that the term “peer-reviewed” isn’t synonymous with valid. The perception is that a scientist submits a paper to an unbiased and independent journal and then it is judged or refereed by unbiased and independent peers (other scientists in the same field.) And if it passes such rigorous scrutiny, then it would be deemed “valid” science. It is painfully obvious to me, this isn’t the case. Are other sciences similar? I wouldn’t be surprised if some were. But then, they’re not advocating the scrapping of our entire socio-economic system of the globe either, so we can address those problems after we save the planet from eco-suicide.
The question is, were the climate scientists in such an incestuous position that their objectivity would have been severely compromised. Is it possible for Jones to adequately referee Wahl and Ammann or is it simply a like minded ideologue agreeing with others of the same mind? I believe it would be quite reasonable to make that conclusion. What of Mann and Trenberth? Or anyone else on this diagram for that matter. http://climateaudit.files.wordpress.com/2011/05/social_network.gif <——————- This is the important question, all others are ancillary. Throw those papers out of the current body of climate science and what’s left?????? Answer…….. An historical data altering jailbird and 3 failed predictions on a 23 y/o paper.
There are really two issues at play in Wegmans report.
The first issue is the mere existence of Pal review. Nick Stokes has not denied that Pal review goes on, rather he has argued that it is par for the course in some places. he readily admits that Pal review takes place. he thinks it is perfectly fine.
The other “conclusion” that Wegman draws is that Pal review may contribute to “bad” science being allowed into the system.
From 2005 RC has argued that Peer review is necessary, but not sufficient.
That is, peer review does not guarantee Truth. It does not prevent error.
What then allows error in thru peer review? gremlins? leprauchans? evil doers,
stupidity?
Wegman offers a hypothesis. That Pal review is more likely to occur in a small field where everybody works as each other co author, and second that this pal review may contribute to error creeping into peer review. Friends reviewing friends may lead to more error. Co Authors reviewer each other may lead to more error. Co workers reviewing each other may lead to more error.
So, we have an explanation of how some error may creep in. We don’t have proof positive that this is the cause ( much as we dont have proof positive that C02 causes warming) In the debate on climate skeptics are often asked to offer their own explanation of the warming that occurs. Fair enough. If nick wants to step up and explain how error creeps into peer review, he is welcome to try. Until then we have Wegman’s theory. I see no evidence to contradict or falsify his theory. I see confirmation of it in the fact that Said’s paper got through. I see evidence of it in the fact that Santer got through, and that Wahl and Amman got through.
If we want an explanation of how Mann 98 got publsihed with decenter PCA, I’d bet on Pal review. or leprauchans.
johnmcguire says:
May 24, 2011 at 11:31 am
~~~~~~~~~~~~~
Ranking member is informally used for the senior member of the minority party. Yes, Waxman has little power now, but judging from his past, he doesn’t miss a chance to grandstand during congressional hearings. I suspect he will bring up the Wegman issue. Even minority members get the microphone for 10 or 15 minutes a shot.
Just a final nail in the coffin of the importance of Said et al. to the overall debate,
I did not include it on my list and I specifically remember looking up Wegman’s work in relation to climate change. I had found Said et al. but was unable to obtain a full copy at the time and the abstract did not seem relevant,
“Social network analysis has proven to be a useful tool in analysis of many situations. We begin by giving an overview of social network analysis. We then illustrate the concepts by examining the social networks of co-authors of scholarly publications. Scholarly publication is in many ways the lifeblood of academic institutions and there are strong incentives, both in terms of prestige and financial compensation, for faculty members to publish. Different disciplines and individuals have evolved distinguishable mechanisms for coping with the publication pressures. We examine the co-authorship networks of a number of prominent scholars. Based on the clustering within the co-author social network, we distinguish several styles of co-authorship including solo models (no co-authors), mentor models, entrepreneurial models, and team models. We conjecture that certain styles of co-authorship lead to the possibility of group-think, reduced creativity, and the possibility of less rigorous reviewing processes.”
Game Over.
James Sexton says:
May 24, 2011 at 2:34 pm
The question is, were the climate scientists in such an incestuous position that their objectivity would have been severely compromised.
=============================================
Nice post James and yes that is the real question. It is not the lack of peer review that troubles me, but the apparent lack of integrity on the approach to the subject. There is little “seek the truth” in climate science and it seems as though those who question the science are pushed aside (at least in the past).
A well published friend of mine once said that some scientists approach publication with an agenda. This to me is the hallmark of climate science.
Some good points raised, but this is no ordinary scientific debate and no ordinary ‘clique’.
These ‘climate scientists’ seem to have a Teflon barrier around them that elevates them in the public eye and a manufactured mainstream level of support that places them to a level of public confidence that they seem to do no wrong. . Writers like Oreskes have been cruising in Australian waters for a while now, locking onto consenting MSM journalists, spreading the ‘correct’ message and elevating this ‘clique’ and smearing critics, their learned opinions and their qualifications.
I don’t think I have come across this phenomenon in my decades on this planet.
Look at the gentle treatment P Jones had in the Climate gate inquiry. These guys are treated as esteemed custodians and interpreters of the data and the establishment bows down to their judgements, in spite of the blatantly obvious abuse of the scientific process laid out for all to see in the Climategate emails.
This is going to be a tough nut to crack, Said et al can throw mud, but it merely slides off, and this clique shines through.
No, you ave very good at changing the subject. Staying on topic you get an F. Notice how I am just contesting your contention, not arguing the point you tried to divert the discussion to. That is staying on topic. Something you apparently have not learned yet.
Funny how your reading other blogs and come across a quotation that seems apt to other discussions.
‘Of course Jane wouldn’t have seen it like that.
In the words of the American Politician and Novelist Upton Sinclair: “It is difficult to get a man to understand something when his job depends on not understanding it.”‘
http://winstonsmith33.blogspot.com/2011/05/climbing-and-absailing-with-burglars.html
Actually, the journals make no secret of the fact that they want buddy review. This way, only insiders get in. For example, if you submit a paper to Science you are invited to designate as many as five reviewers for your paper. Climatic Change – the one started by Stephen Schneider – requires you to give names and academic qualifications of four reviewers for your paper. Should you want give less than four names because you don’t have connections you will not even be allowed to submit your manuscript. This is what goes for “peer review” today.
For the record, the 2006 Wegman report is footnoted in the “Verified
Petition for Mandamus and Injunctive Relief” filed May 15, 2011,
by the American Tradition Institute with the Prince William County
Circuit Court.
See:
http://www.atinstitute.org/wp-content/uploads/2011/05/ATI_v_UVA_FOIA_First_Petition_final_5-15-11.pdf
This led to the current UVa/ATI consent decree for Mike Mann’s
research, e-mails, and other related public documents.
See:
http://www.atinstitute.org/court-orders-university-of-virginia-to/
For those who warmly embrace conspiracy theories (and those
warmists who see “anti-science” conspiracies all over the place):
Please note that the current Virginia Attorney General is listed
among George Mason University’s “Notable Alumni” see:
http://en.wikipedia.org/wiki/George_Mason_University