The madness of 97% 98% consensus herds

UPDATE: comments welcome on Dr. Richard Tol’s draft paper on this issue, see below. This will be a top post for a day, new posts will appear below this one – Anthony

“Men, it has been well said, think in herds; it will be seen that they go mad in herds, while they only recover their senses slowly, and one by one.”

That is from Charles Mackay in his book, Extraordinary Popular Delusions and the Madness of Crowds first published in 1841.

I think it is an apt description of the process that led to Cook et al. (2013) Quantifying the consensus on anthropogenic global warming in the scientific literature because that paper is in fact, a product of a crowd evaluating a crowd. As an example, Dr. Richard Tol has just discovered that using Cook’s own data, the consensus number Cook should have published is 98%, rather than 97%.

Dr. Tol writes in a critique of the Cook et al. paper:

In fact, the paper by Cook et al. may strengthen the belief that all is not well in climate research. For starters, their headline conclusion is wrong. According to their data and their definition, 98%, rather than 97%, of papers endorse anthropogenic climate change. While the difference between 97% and 98% may be dismissed as insubstantial, it is indicative of the quality of manuscript preparation and review.

He shows the Cook data as he compiles it: 

Tol_table1

You’d think such simple elementary errors in data would have been caught in peer review, after all, that is what peer review is for.

I think that there was a goal by Cook and his crowd, and that goal was to match the 97% number that has become a popular meme in the literature and the media. This intent seems confirmed by a recent statement by one of the co-authors, Dana Nuccitilli in a media argument that 97% global warming consensus meets resistance from scientific denialism

However, we have used two independent methods and confirmed the same 97% consensus as in previous studies.

It is that branding of “denialism” by Nucciltelli to Dr. Tol, who is hardly a “denier” on climate change even by the loosest definition, that has given Tol incentive to now start systemically deconstructing the paper. It also lends a window into the mind of the coauthor Nucitelli, who can’t seem to assimilate useful criticisms, no matter how valid, but instead publicly attributes discovery of real errors in the Cook et al. paper to “denialism” rather than the self-correcting process of science. Nuccitelli’s actions suggest to me, a mindset of zealotry, rather than one of discovery. His actions of branding Dr. Tol’s and others valid criticisms, seem to fit the textbook definition of the word:

zealotry_definition

As an aside, it seems truly laughable that the Guardian has created an entire regular opinion column based and named on this 97% number, and it supports that idea that this was the “target number” rather than the number that the actual data would report. Richard Tol has just proven their own data doesn’t even match the title of their paper. Will the Guardian now correct the title?

98_pct_Guardian

Tol goes on to say this about the crowd-sourcing:

The results thus depend on the quality of the volunteers. Are they neutral observers, or are they predisposed to endorsing or rejecting anthropogenic climate change? Did they suffer from fatigue after rating a certain number of abstracts? 12 volunteers rated on average 50 abstracts each, and another 12 volunteers rated an average of 1922 abstracts each. Fatigue may well have a problem. This level of effort by a volunteer could indicate a strong interest in the issue at hand.

Indeed, and he backs this up by saying it is evident in the data:

WoS generates homoskedastic data. Rating made the data heteroskedastic. Sign of tiredness or manipulation.

So which is it? Tiredness or manipulation, or perhaps both? Based on what has been observed so far, I’d say there is a combination, but given the obvious 97% target, more likely it is an unconscious manipulation by the chosen crowd of volunteer reviewers, which included no climate skeptics and consisted of mostly insiders for Cook’s antithetically named website, “Skeptical Science”. Tol goes on to comment:

No neutral person would volunteer to do 1922 tasks. Cook’s data duly show bias: 35% of abstract were misclassified, 99% towards endorsement.

http://twitter.com/RichardTol/status/341086919930830848

To support the idea that bias played a role in reaching the conclusions of the Cook et al. paper, there seems to be a systemic sloppiness in the sampling process, as Tol points out in his critique:

In fact, 34.6% of papers that should have been rated as neutral were in fact rated as non-neutral. Of those misrated papers, 99.4% were rated as endorsements. It is therefore reasonable to assume that the volunteers were not neutral, but tended to find endorsements where there were none. Because rater IDs were not reported, it is not possible to say whether all volunteers are somewhat biased or a few were very biased.

Tol also says this about the 97% scientific consensus claim:

It is a strange claim to make. Consensus or near-consensus is not a scientific argument. Indeed, the heroes in the history of science are those who challenged the prevailing consensus and convincingly demonstrated that everyone thought wrong. Such heroes are even better appreciated if they take on not only the scientific establishment but the worldly and godly authorities as well.

Well known examples of this include the challenges to the theory that Earth was the center of the universe, that infection was spread by surgeons who didn’t wash their hands, that the Earth’s crust had plates that moved, and that gastric ulcers were caused by a bacterial infection, and not stress as physicians once widely believed. As William Briggs writes:

There was once a consensus among astronomers that the heavens were static, that the boundaries of the universe constant. But in 1929, Hubble observed his red shift among the stars, overturning that consensus. In 1904, there was a consensus among physicists that Newtonian mechanics was, at last, the final word in explaining the workings of the [universe]. All that was left to do was to mop up the details. But in 1905, Einstein and a few others soon convinced them that this view was false.

Consensus can also cause disaster, as NASA proved with a consensus of management that solid rocket booster O-rings affected by unusual cold weren’t worth worrying about or that a foam strike during launch wouldn’t damage the wing of the space shuttle and were “not even worth mentioning”.

Clearly, the power of thousands in agreement on scientific consensus can’t stand up to stubborn facts and that is the self-correcting process of science which sometimes works slowly, other times dramatically quickly. Given that consensus by itself means nothing in the face of such facts, it seems to me that consensus is just another manifestation of herd-like thinking as illustrated by Mackay.

From the Amazon summary of Mackay’s insightful book on crowds:

First published in 1841, Extraordinary Popular Delusions and the Madness of Crowds is often cited as the best book ever written about market psychology. Author Charles Mackay chronicles many celebrated financial manias, or ‘bubbles’, which demonstrate his assertion that “every age has its peculiar folly; some scheme, project, or fantasy into which it plunges, spurred on by the love of gain, the necessity of excitement, or the mere force of imitation.” This still holds fast today! Among the alleged ‘bubbles’ described by Mackay is the infamous Dutch tulip mania, the South Sea Company bubble and the Mississippi Company bubble. And what do bubbles do? Why they burst of course.

The Cook et al. paper bubble is about to burst.

UPDATE: Read the draft paper Tol is working on here, comments welcome:

https://docs.google.com/file/d/0Bz17rNCpfuDNM1RQWkQtTFpQUmc/edit

Advertisements

  Subscribe  
newest oldest most voted
Notify of
Jimmy Haigh.

CAGW is about to burst I would say…

Edohiguma

This is just one of the reasons why the term “social sciences” makes me roar in anger. We’re not scientists. We’re scholars. What far too many of us sell as “science” is more often than not highly biased statistics with ridiculously low sample groups and faulty methods.

Gary Pearse

How’s this for a measure of the bias. 12 were given 50 abstracts each and 12 were given 1922 abstracts each to evaluate. The point that anyone who happily accepts 1922 tasks has to be biased zealot is supported by the arithmetic: (1922-50)/1922 = 0.974, or 97%!!! The zealots were given the abstracts that could possibly be interpreted as pro AGW and instructed to do so, and the others were given abstracts that didn’t mention “global warming” per se.

Camburn

This is a confirmation of Skeptical Science Sydrome. Plain and simple.

Twelve anonymous reviewers, of unknown scientific background, review 1922 pre-screened abstracts for buzz words required to get into the AGW grant gravy train….and this is presumed as science.
Most curious is Dr John Abrahams rabid support, given he is a professor of mechanical engineering and should have adequate training in thermodynamics to at least have serious doubts about the AGW hypothesis. However, his college is a religiously sponsored institution, and leaders of that religion did invest the institution pension fund heavily in the Carbon credit market. Many on the pro-warming side have no higher motive for promoting the fraud than to preserve their portion of the misspent Carbon endangerment grant funding. The money dog is wagging the science tail.

The same things still run down hill as always.
At the top the suff if made by The Two Party Evil Money Cult in Washington D.C..
The Taxes and the Spending are just the creeks, gullies, streams leading to the rivers of the “stuff” that still runs down hill.
It is not about facts, the rule of law, freedom, individual liberty, it is about Taxing and Spending nothing else.
Just a new method of the con.

Ashby

That’s one of my favorite books. It’s worth picking up this version:
Extraordinary Popular Delusions and the Madness of Crowds & Confusión de Confusiones (Wiley Investment Classics)

Trond A

The famous swiss psychiatrist C. G. Jung said something like: When men act in (as) a crowd the consciousness is reduced.

Mike Bromley the Kurd near the Green Line

It still amazes me how wall-eyed the nonsensus crowd is.

pokerguy

“It is a strange claim to make. Consensus or near-consensus is not a scientific argument. Indeed, the heroes in the history of science are those who challenged the prevailing consensus and convincingly demonstrated that everyone thought wrong. Such heroes are even better appreciated if they take on not only the scientific establishment but the worldly and godly authorities as well.”
Of course this sounds convincing in principal. However, the fact remains that a very strong consensus…one in the high 90’s…must generally be taken by a layperson unsure what of to believe, as likely reflective of scientific validity.
I’m a full on skeptic, but if it were really shown that 97 (or 98!) percent of scientists were convinced of the validity of CAGW, I’d seriously have to step back and take another look. I don’t think that makes me some sort of easily swayed nincompoop. I think it’s simply common sense.

JJ

“I think that there was a goal by Cook and his crowd, and that goal was to match the 97% number that has become a popular meme in the literature and the media.”
That much is bltantly obvious. And they not only matched it, they matched it twice – with the abstract ratings and the author’s self assessments.
And think about that match in the light of the degree of the reported disagreement between those two assessments. They get 97% AGW support from the amateur abstract ratings. But according to C(r)ook, the authors’ self-assessments disagree with those ratings by a minimum of 50% for one group (“no opinion”) and by a minimum of 70% for another group (“reject AGW”). Yet when all the machinations cease, they arrive at 97% AGW support for the author’s self assessments as well.
One gets the distinct impression that if you fed C(r)ook red noise or baseball scores, he’d somehow arrive at the number 97.

rustneversleeps

“Men, it has been well said, think in herds; it will be seen that they go mad in herds, while they only recover their senses slowly, and one by one.”
That is from Charles Mackay in his book, Extraordinary Popular Delusions and the Madness of Crowds first published in 1841.

Hmmm,
For what it’s worth, it also appears to be from Sting’s song “All This Time”:
“Men go crazy in congregations. They only get better one by one.”

I guess the real question is “which is the herd that has gone mad”? Is it those that acknowledge the overwhelming consensus in the scientific literature? Or is it the herd that stamps its little feet and sticks it fingers in its ears trying to avoid that simple fact?
Inquiring minds want to know…

Nik Marshall-Blank

@Mike Bromley – I don’t understand your comment. who is the non consensus? The AGW is not a consensus…
The heat is in the oceans
The CO2 is causing the heat.
The Sun is not causing the heat, Trenberth says 10-15% is. Oops
Burning CO2 releases plant fertilisers, greening and therefore CO2 is removed from the atmosphere. But burning CO2 is bad, supposedly. Oops, oops.
Where’s the consensus? expect against those who know it is a pack of mis-representations of natural events.

DirkH

Edohiguma says:
June 2, 2013 at 7:50 am
“This is just one of the reasons why the term “social sciences” makes me roar in anger. We’re not scientists. We’re scholars. What far too many of us sell as “science” is more often than not highly biased statistics with ridiculously low sample groups and faulty methods.”
Don’t worry. For me you are social engineers.

It is time for a refesher course on what the Scientific Method is something Dana and Cook needs to see in this media presentation:
Feynman on Scientific Method.
http://www.globalwarmingskeptics.info/thread-2184.html

marissa

The 97% is obviously a fiction, the scientists need to stand up and express their opposition

Henry Galt

As penance for my being double-dumb on a previous thread I reiterate what I posted earlier. Please note the dates as well as the claims:
Thu, 2012-11-15 10:26
“” I searched the Web of Science for peer-reviewed scientific articles published between 1 January 1991 and 9 November 2012 that have the keyword phrases “global warming” or “global climate change.” The search produced 13,950 articles. “”
“” I read whatever combination of titles, abstracts, and entire articles was necessary to identify articles that “reject” human-caused global warming. To be classified as rejecting, an article had to clearly and explicitly state that the theory of global warming is false or, as happened in a few cases, that some other process better explains the observed warming. Articles that merely claimed to have found some discrepancy, some minor flaw, some reason for doubt, I did not classify as rejecting global warming. Articles about methods, paleoclimatology, mitigation, adaptation, and effects at least implicitly accept human-caused global warming and were usually obvious from the title alone. John Cook and Dana Nuccitelli also reviewed and assigned some of these articles … “”
What I couldn’t achieve with reading skills such as these gods amongst mortals display.
This shyster is claiming is that the three of them read “” … whatever combination of titles, abstracts, and entire articles was necessary to identify articles that “reject” human-caused global warming. “” out of @13,900 articles……
In less than a week.
I base my estimate on the dates and allow some slack for the ‘writing’ of the piece of garbage surrounding the central strawman.
Link to follow for the interested (I wont add it here as it, quite rightfully, triggers the shit-fliter)

pokerguy

“The 97% is obviously a fiction, the scientists need to stand up and express their opposition.”
Many have, but it does no good. I keep hammering away on the need for an actual, statistically valid survey to counter this 97 percent canard, but no one seems interested. More enjoyable I guess, and far easier to sit back and complain about biased surveys, and gullible warmists, and to quote noble sounding observations about the nature of science.
The truth is that 97 percent number would be compelling if real. It needs to be efficiently and legitimately countered.

Ryan

Anthony(or Tol), what do you think the real number is? Why not do a quick survey or two of your own? If the 97% number really is so far off then it should be fairly easy to demonstrate that instead just taking potshots from the sidelines.

sparky

dear rustneversleeps, perhaps you should go outside and ask a member of the public they might be able to tell you…..

Henry Galt

rustneversleeps says:
June 2, 2013 at 8:37 am
Again with the misnomers.
Your inquiring mind would find that nearly ALL the ‘papers’ show evidence of warming or produce claims from modeling.
A very few use ‘simple, centuries old physics’ to show a greenhouse effect may warm the planet as CO2 is increased.
NONE produce empirical evidence to show that our addition to the carbon budget caused any of that warming.
It really is that simple. Billions upon billions spent and we are no nearer being able to blame CO2 for the small, benign warming during the 20th century.
What a waste. What a crying shame. What a travesty.

John M

Why not do a quick survey or two of your own?

Could you imagine Ryan as the owner of a trucking company where one of his drivers just drove into a school bus? A reporter starts investigating and Ryan says “Hey, why don’t you buy your own trucks and learn how to drive them!”

megawati

97 pct, 98 pct; such figures definitely ring a bell…
Ah yes, those official Soviet bloc election results that always made us laugh back then, just before their whole criminal system imploded.

Latitude

,,,and 97% of scientists said these drugs were safe…………
http://en.wikipedia.org/wiki/List_of_withdrawn_drugs

Tony

Politics uses the word “consensus” and suspecting number from polls to backed their position … is science heading there too?

Myrrh

[snip – off topic slayers junk science – mod]

Jay

This is just a case of people doing what they have been paid to do.. Its like hit men complaining that killing people is illegal.. Its true, but has nothing to do with that envelope stuffed full of money and that job that has just been paid for..
So you double down on the idea that you are a professional and do what you have to do, knowing full well that your name could be the next name if you dont watch your step..
The underworld has peer / pal review as well..

– don’t start off with trying to get 97%
“You don’t use science to show that you’re right, you use science to become right.”
SCIENCE. It works, bitches.
Richard Dawkins actually quoting the 2008 words in XKCD Webzine by Randall Munroe

RACookPE1978

Ryan says:
June 2, 2013 at 9:12 am

Anthony(or Tol), what do you think the real number is? Why not do a quick survey or two of your own? If the 97% number really is so far off then it should be fairly easy to demonstrate that instead just taking potshots from the sidelines.

See, no skeptic needs to cite such a government-needed, government-mandated majority of government-paid government-workers using government-sorted government-grants …. to suit a government-propaganda effort to control the population’s energy and gain 1.3 trillion in new taxes.
All ANY skeptic needs to disprove the religion of CAGW is one fact or one paper that falsifies the theory. (And, for example, http://www.co2science.org has over 900 papers showing the MWP did in fact exist with temperatures greater than, well before man’s CO2 was a factor in the global.) Einstein, when attacked by government-pleasing “scientists” in the mid-30’s said bluntly just that: “All they need is one fact ..”
BUT, what no skeptic has is the government’s propaganda machinery of its mass media, Hollywood’s exposure, and the liberal/socialist government’s combination of a corrupt, immoral university faculty that is desperate for funding, and a corrupt political body desperate to spend its money to exploit such a desperation..

Nigel S

A London company prospectus from the time of the ‘South Sea Bubble’ (about 1720).
‘A company for carrying out an undertaking of great advantage, but nobody to know what it is.’
It was ever thus…

Russ R.

@pokerguy:
You wrote “I’m a full on skeptic, but if it were really shown that 97 (or 98!) percent of scientists were convinced of the validity of CAGW, I’d seriously have to step back and take another look.”.
The problem is, the “consensus” doesn’t make any mention of the “C” in “CAGW”. It revolves only around the “GW” and the “A”. (And as it happens, I’m in agreement on both those.)
In other words, the “Consensus” is a lot narrower than you presume it to be. It makes no mention of “catastrophic” anything.
So maybe you should “step back and take another look”. Go re-read the “Consensus” literature: Oreskes (2004), Doran and Zimmerman (2009), Anderegg et. al (2010) and now Cook et al. (2013). Then please show us where any consensus was demonstrated relating to A) future warming, B) impacts of that warming, C) policy actions.
You’ll find no consensus whatsoever. The entire “consensus” argument is a supreme shifting of goalposts.

Bruce Cobb

rustneversleeps says:
June 2, 2013 at 8:37 am
No, the real question is; What is it about CAGW that needs to claim consensus, even where none actually exists? If the science were truly sound, wouldn’t it speak for itself?
Inquiring minds want to know….

T&D meet in 98-th century BC
http://www.vukcevic.talktalk.net/TolDana.htm

Mike Haseler

Ryan says: “Anthony(or Tol), what do you think the real number is? Why not do a quick survey or two of your own? If the 97% number really is so far off then it should be fairly easy to demonstrate that instead just taking potshots from the sidelines.”
A much better survey is to assess all papers mentioning the impacts of global warming and look for any that mention the benefits of warming. This is how I decided whether the subject as a whole was biased or impartial. The result was that of the first 50 papers I picked at random, only one partly mentioned a benefit in passing. In other words something like 98% of papers are 100% biased and 2% are very largely biased.

President Gurbanguly Berdymukhamedov of Turkmenistan must be a popular guy http://www.rte.ie/news/2012/0213/312252-turkmenistan/ as he got 97% of the vote in the latest election.
Of course the result of today’s use of consensus science can at times show similar numbers.
No manipulation involved?

Myrrh

The madness of crowds doesn’t let little inconvenient science facts get in the way of the delusion. The delusion is their blanket, their warm thermal blanket wrapping itself around them where from under its cover snuggled tight they argue with great passion against each other on the degree of warming given them by their trace blanket which is practically 100% hole in the atmosphere.

JJ

pokerguy says:
Of course this sounds convincing in principal.

It is not “convincing in principal”. It is the definition of science. “Scientific consensus” is a contradiction in terms.
“However, the fact remains that a very strong consensus…one in the high 90′s…must generally be taken by a layperson unsure what of to believe, as likely reflective of scientific validity.”
Nonsense. It is not true that the layperson must accept that the opinions of scientists are representative of scientific validity, and they very frequently do not. In fact, they frequently counter the overwhelming majority opinion of scientists to reject good science, let alone the politicized crapfest that is ‘climate science’.
I don’t think that makes me some sort of easily swayed nincompoop. I think it’s simply common sense.
Common sense is not science. Science was created specifically to correct the common failings of common sense. Reverting to common sense is to abandon science. That is OK, so long as you understand that is what you are doing. Appealing to “scientific consensus” is to abandon science, while pretending not to. You are not a “full on skeptic” if you do this.
I keep hammering away on the need for an actual, statistically valid survey to counter this 97 percent canard, but no one seems interested.
Apart from legitimate lack of interest, it simply is not possible. One cannot perform a statistically valid fallacy. In order to complete the survey you suggest, for example, one would first have to define the population to be surveyed. On what scientifically valid basis would that delineation proceed?
There is none. What you are suggesting is an exercise in politics. If you are going to do that while invoking the imprimatur of science, then stick to the science. The typical layperson is perfectly capable of understanding that it is not the opinions that scientists hold that matter, but the reasons why the scientists hold those opinions. And they are also quite capable of understanding that “scientific consensus” is frequently wrong, as well as the specific ways in which ‘global warming’ is both non-scientific as well as demonstrably wrong.
We do not need to teach the layperson to accept anti-science and logical fallacy in order to prevail politically, and irrespective of that we should not.

Bruce Cobb

If the claim was that 100% of scientists supported the idea of the existence of the Great Flying Spaghetti Monster, I would expect the evidence to be readily available, and I would still look at said evidence, if nothing other than out of sheer curiosity.

Mike jarosz

Who do you believe? Alarmists are supported by big government progressives who believe in redistribution of wealth(carbon taxes don’t do squat for the planet) and the skeptics who are searching for the truth under peril, with no financial reward at the end. Mercenaries versus the volunteers.

Nick in Vancouver

97% is just like “42” they knew its was the right answer they just didn’t know what the correct question was. Apologies to Douglas Adams who is a far superior writer of fiction.

mark fraser

Or like our friend Norm, who kept nodding off during a game of charades. Each time he awoke, he’d say “Missouri!” as a guess. You may have been past the Cascade motel on the way to Mt. Baker – the site of this most memorable weekend party.

Gary Pearse says:
June 2, 2013 at 7:59 am
(1922-50)/1922 = 0.974, or 97%!!!
===========
You are onto something. 1922/(1922+50) = 97%
This strongly suggests that the reviewers were purposely divided into two groups.to arrive at the 97% answer. The answer of 97% was COOKed from the start.

Latitude

pokerguy says:
June 2, 2013 at 9:08 am
The truth is that 97 percent number would be compelling if real. It needs to be efficiently and legitimately countered.
===============
no it doesn’t….LOL
The fact is after all these decades….they are still trying to convince people that it’s real

I have now uncovered the difference between 97% and 98%. The reported data have a rate 4 (7970 abstracts). Apparently, the original data have rates 4a (7930) – no position – and 4b (40) – uncertain. They shifted the 40 into the denominator.
Meanwhile, Dana N is loudly complaining that I unfairly accuse them of not fully reporting their data.

Jeff L

Although I appreciate the deconstruction of this 97 % rubbish, it does take to focus away from the fact that science isn’t done by consensus. Even if it was 100%, it still doesn’t mean the science is right.

Reich.Eschhaus

Richard Toll,
can I ask you what the order is of the abstracts that you show rolling statistics on? Is it the Year-Title-order from Cook et al’s data file?
Thanks!

Jim G

Lemmings operate by some unknown type of consensus when they swim out and drown. The “lemming instinct”, as we always called it, appears to be alive and well in climate science. Even rats, another rodent, will depart a sinking ship. Lemmings, maybe not so much. I think we need to change the operative word from “warmists” to lemmings.

Reich.Eschhaus

“Reich.Eschhaus says:
June 2, 2013 at 10:55 am
Richard Toll,”
Oops! Sorry Richard Tol for miswriting your last name!

pokerguy

“You’ll find no consensus whatsoever. The entire “consensus” argument is a supreme shifting of goalposts.”
You completely miss my point. . No kidding the entire consensus argument is bogus. But try convincing a NYT’s reading, MSNBS watching, Greenpeace contributing left winger of that. The best way to counter this b.s. is to fund and design a statistically valid survey with meaningful definitions. Most of you are either too lazy, or too cynical, or perhaps too dense to understand that.
Philosophy of science considerations and lofty, misty-eyed rhetoric about heroic renegade researchers are fine. IN the real world, you fight in any way you can. Once more, this is a PR war. It it were really just about the science, the skeptics would already have been declared the winners.