Steve McIntyre comments on Wegman’s work and Deep Climate’s recent dissing of the Said et al paper. It seems there’s a clique of too close for comfort relationships that may have tainted peer review:
Lost in the recent controversy over Said et al 2008 is that the Climategate documents provided conclusive evidence of the hypothesis originally advanced in the Wegman Report about paleoclimate peer review – that reviewers in the Mann “clique” had been “reviewing other members of the same clique”.
I won’t attempt to analyze the plagiarism issues today (I will return to this on another occasion), other than to say that some recent literature on the topic attempts to distinguish between degrees of plagiarism e.g. Bouville, Clarke and Loui.
In addition, contrary to recent false claims by USA Today, Said et al 2008 was not “a federally funded study that condemned scientific support for global warming”. It does not mention global warming nor even climate. Nor is Said et al 2008 a “cornerstone” of criticisms of either Mann or IPCC as Joe Romm falsely claimed. For example, it has never been referred to or discussed at Climate Audit even in comments. (Nor at any other climate blog, to my knowledge.)
…”The Wegman Report was vindicated on its hypothesis about peer-review within the Mann ‘clique’.
‘The Wegman Report hypothesized, but were unable to prove, that reviewers in the Mann ‘clique’ had been ‘reviewing other members of the same clique’. Climategate provided the missing evidence, Climategate documents showed that clique member Phil Jones had reviewed papers by other members of the clique, including some of the articles most in controversy – confirming what the Wegman Report had only hypothesized”
Read it at: Climategate Documents Confirm Wegman’s Hypothesis
An important distinction to keep in mind is that while the Said et al paper has bee retracted (temporarily we hope) this in no way affects the Wegman report to Congress or its conclusions (i.e. it would not automatically cause a retraction of the Congressional report, which operates under a different scenario of rules and public record).
nice work
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
Seems to me that novel statistical techniques should be reviewed by statisticians, rather than statistical wannabes from within the clique.
Said et al in fact conceded they had no evidence of actual “clique” reviewers. They simply listed people who had previously coauthored papers. Well, we knew papers were coauthored.
Steve M had a few cases where people in this small field had reviewed papers written by people in their “clique” (which simply means people with whom they have at other times been coauthors). Experts are few – that is not surprising.
But it also fits with the way reviewers are selected. A journal editor will seek a range of opinions. The reviewers will usually include someone who shares the outlook of the author, and a likely critic. To save time, many journals invite authors to nominate people that they would like to see as referees.
For example, CSDA, where Said et al appeared, says:
“Please submit, with the manuscript, the names, addresses and e-mail addresses of 3 potential referees. Note that the editor retains the sole right to decide whether or not the suggested reviewers are used.”
That’s a standard phrase. If you google it, you get 18000 hits from journals which make that invitation.
Note the caveat. Only a few, if any, of the reviewers will be from that list. And of course authors will nominate people they have worked with.
Wheels within wheels. Twas ever thus when rules are relaxed.
Anthony:
For those who want to refresh their memories, the Climategate
e-mails can be reviewed at:
http://www.eastangliaemails.com/index.php
The original 2006 Wegman report to Congress in PDF format is at:
http://www.uoguelph.ca/~rmckitri/research/WegmanReport.pdf
But pal reviews are so warm and fuzzy …
Keith Minto says: May 24, 2011 at 12:39 am
“In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?”
Very sensible point. But perhaps that is a reason to see this as a potentially general problem for scientific peer review. It’s almost inevitable (given the evidence of climategate) that similar “groupthink” exists in many other areas of science.
In the end global warming will be just another hysteria consigned to the dustbin of history, but the problems within science are unlikely to be addressed unless society forces science to really look at the abysmal mess that it seems to be in, in areas like climate “science” and does something about it.
Perhaps 30% of all reviewers should be from outside the discipline and not connected with the other reviewers or author. Their specific scientific knowledge may not be as good, but they are far more likely to asked the simple questions that challenge the groupthink at a stage where it can be tackled and corrected (and not at the paranoia stage when all criticism is viewed as some kind of conspiracy to destroy the discipline)
“Old Boys Club”
Kieth,
but doesn’t the IPCC claim membership in the thousands ? also we are talking about less than 2 dozen of the top names who where participating in this nonsense …
I have always maintained that the *only* surprising thing in Climategate to followers of ClimateAudit was the fact of the release. The emails mesh so perfectly with Steve’s blog up to then that, had CRU not acknowledged them, one could be forgiven for suspecting that they had been faked up using ClimateAudit as an outline.
The Alarmists talk about climate science as if it requires a different logic and mathematics than other sciences. It doesn’t. They can’t explain a lot of their stuff because it is hand-waving to get them to their predetermined conclusion.
BTW,
Who is Deep Climate?
“Keith Minto says:
May 24, 2011 at 12:39 am
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?”
That would be true if this was actually a small field of study – but it isn’t. If you want to self define “climate-change science” then you can use this argument, but there is no such thing as “climate change” as a scientific discipline. Michael Mann, for example, is a paleontologist – there are plenty of those to choose from to review his work, but no, we have to get someone who is already part of the “climate-change” team. Why? Because we (the journal editors who are also part of the team) have decreed that he is a “climate-change scientist”.
This is at the heart of the alarmist strategy – create a field in which you (the team) are the only experts and therefore only you can properly pontificate. Then get some fellow-travellers to keep referring to you as the experts (social scientists like Oreskes and “journalists” like Romm) and dismissing everyone else as not qualified. Hey presto, not only do you become the only authoritative voice, but you also make sure all of the grant money comes to you and people you accept as part of the team.
As an academic scientist, I often reviewed papers that were outside my direct experience, but to which I was asked to bring my expertise. In some cases, I could only comment on parts of the paper, because that was all I had experience of, but that was what I was asked to do by the editor. This is peer review done properly. I have seen this most clearly when identifying reviewers for PhD theses: Supervisors I have worked with have gone out of their way to find someone with specialized knowledge of a critical topic, but often with a different background, in order to bring something extra to the review process. The students might not necessarily like it, but it makes for a better thesis.
A few people here have commented that this a “small field” with “few experts”.
I dont know about your planet, but here on Earth, climate research is a muti-billion dollar industry. It is the most overfunded and overstaffed field in all of science.
Keith Minto says, May 24, 2011 at 12:39 am:
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
No, in a small field, it is a bit incestuous. However, going back some 30 years, it looked to me (as an outsider) that this also encouraged proper discussions between reviewers and authors, which made papers better.
Additionally – scientists in small fields of study did not and do not expect the developed world to dismantle their economy, tax their citizens into oblivion and generally expect supranational organisations to administer life on earth to an extent that everybody can look forward to a prehistoric lifestyle …
I would ask a more learned person to weigh in, but…
Keith Minto says:
May 24, 2011 at 12:39 am
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
On many of the papers, is there not enough that a Statistician or Physicist could, and maybe should, participate in the peer review process? If so, the field is certainly not small.
Get serious, it always comes back to Mann, Jones, Briffa and a couple of others. It’s not even a clique, they could all fit in a closet.
Nick Stoke said:
Only a few, if any, of the reviewers will be from that list. And of course authors will nominate people they have worked with.
Weelll – that form for authors where potential peer reviewers should be suggested is always difficult for me. because I’m trying to find someone with whom I did not work, or at least do not work for several years. More fool me…
“Jeff says:
May 24, 2011 at 4:29 am
Kieth,
but doesn’t the IPCC claim membership in the thousands ? also we are talking about less than 2 dozen of the top names who where participating in this nonsense …”
Good observation Jeff. The alarmists seldom see the knife that they cut their own throats with.
“An important distiction to keep in mind is that while the Said et al paper has bee retracted (temporarily we hope) this in no way affects the Wegman report to Congress or its conclusions.”
Looks like a spelling bee stung two words in that sentence, “distinction” rather than “distiction”, and “been” rather than “bee”.
Interesting article.
The Mann Clique… sounds like some deviationalist Marxist-Leninist group. Must notify the NKVD to start making arrests.
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
Isn’t there a code of ethics. I can’t do an unbiased review because Mike and I wrote these papers together…
Dave says:
“It’s not even a clique, they could all fit in a closet.”
True dat. However, Wegman’s statistics definition of a clique is the designated group being studied.
And the definition of a claque is: A group of people hired to applaud (or heckle) a performer or public speaker.
• a group of sycophantic followers
Claque: Romm, Schmidt, Mann, Cook, and other hired alarmist bloggers in particular, and their sycophantic true believer followers in general.
[Sycophant: A person who acts obsequiously toward someone in order to gain advantage; a servile flatterer.]
Norm814 says:
May 24, 2011 at 6:38 am
In such a small field of study, in fact any small field, is a ‘clique’ of reviewers unavoidable ?
Isn’t there a code of ethics. I can’t do an unbiased review because Mike and I wrote these papers together…
It isn’t a small field; it is actually far broader than the “team” pretends it is. They us data and -more or less- theory from numerous disciplines, but fail badly to actually consult within those disciplines on a regular basis to insure their thinking is in a trap they may be unaware of, but which practitioners of the borrow discipline would be well aware of.