UPDATE: comments welcome on Dr. Richard Tol’s draft paper on this issue, see below. This will be a top post for a day, new posts will appear below this one – Anthony
“Men, it has been well said, think in herds; it will be seen that they go mad in herds, while they only recover their senses slowly, and one by one.”
That is from Charles Mackay in his book, Extraordinary Popular Delusions and the Madness of Crowds first published in 1841.
I think it is an apt description of the process that led to Cook et al. (2013) Quantifying the consensus on anthropogenic global warming in the scientific literature because that paper is in fact, a product of a crowd evaluating a crowd. As an example, Dr. Richard Tol has just discovered that using Cook’s own data, the consensus number Cook should have published is 98%, rather than 97%.
Dr. Tol writes in a critique of the Cook et al. paper:
In fact, the paper by Cook et al. may strengthen the belief that all is not well in climate research. For starters, their headline conclusion is wrong. According to their data and their definition, 98%, rather than 97%, of papers endorse anthropogenic climate change. While the difference between 97% and 98% may be dismissed as insubstantial, it is indicative of the quality of manuscript preparation and review.
He shows the Cook data as he compiles it:
You’d think such simple elementary errors in data would have been caught in peer review, after all, that is what peer review is for.
I think that there was a goal by Cook and his crowd, and that goal was to match the 97% number that has become a popular meme in the literature and the media. This intent seems confirmed by a recent statement by one of the co-authors, Dana Nuccitilli in a media argument that 97% global warming consensus meets resistance from scientific denialism
However, we have used two independent methods and confirmed the same 97% consensus as in previous studies.
It is that branding of “denialism” by Nucciltelli to Dr. Tol, who is hardly a “denier” on climate change even by the loosest definition, that has given Tol incentive to now start systemically deconstructing the paper. It also lends a window into the mind of the coauthor Nucitelli, who can’t seem to assimilate useful criticisms, no matter how valid, but instead publicly attributes discovery of real errors in the Cook et al. paper to “denialism” rather than the self-correcting process of science. Nuccitelli’s actions suggest to me, a mindset of zealotry, rather than one of discovery. His actions of branding Dr. Tol’s and others valid criticisms, seem to fit the textbook definition of the word:
As an aside, it seems truly laughable that the Guardian has created an entire regular opinion column based and named on this 97% number, and it supports that idea that this was the “target number” rather than the number that the actual data would report. Richard Tol has just proven their own data doesn’t even match the title of their paper. Will the Guardian now correct the title?
Tol goes on to say this about the crowd-sourcing:
The results thus depend on the quality of the volunteers. Are they neutral observers, or are they predisposed to endorsing or rejecting anthropogenic climate change? Did they suffer from fatigue after rating a certain number of abstracts? 12 volunteers rated on average 50 abstracts each, and another 12 volunteers rated an average of 1922 abstracts each. Fatigue may well have a problem. This level of effort by a volunteer could indicate a strong interest in the issue at hand.
Indeed, and he backs this up by saying it is evident in the data:
WoS generates homoskedastic data. Rating made the data heteroskedastic. Sign of tiredness or manipulation.
So which is it? Tiredness or manipulation, or perhaps both? Based on what has been observed so far, I’d say there is a combination, but given the obvious 97% target, more likely it is an unconscious manipulation by the chosen crowd of volunteer reviewers, which included no climate skeptics and consisted of mostly insiders for Cook’s antithetically named website, “Skeptical Science”. Tol goes on to comment:
No neutral person would volunteer to do 1922 tasks. Cook’s data duly show bias: 35% of abstract were misclassified, 99% towards endorsement.
To support the idea that bias played a role in reaching the conclusions of the Cook et al. paper, there seems to be a systemic sloppiness in the sampling process, as Tol points out in his critique:
In fact, 34.6% of papers that should have been rated as neutral were in fact rated as non-neutral. Of those misrated papers, 99.4% were rated as endorsements. It is therefore reasonable to assume that the volunteers were not neutral, but tended to find endorsements where there were none. Because rater IDs were not reported, it is not possible to say whether all volunteers are somewhat biased or a few were very biased.
Tol also says this about the 97% scientific consensus claim:
It is a strange claim to make. Consensus or near-consensus is not a scientific argument. Indeed, the heroes in the history of science are those who challenged the prevailing consensus and convincingly demonstrated that everyone thought wrong. Such heroes are even better appreciated if they take on not only the scientific establishment but the worldly and godly authorities as well.
Well known examples of this include the challenges to the theory that Earth was the center of the universe, that infection was spread by surgeons who didn’t wash their hands, that the Earth’s crust had plates that moved, and that gastric ulcers were caused by a bacterial infection, and not stress as physicians once widely believed. As William Briggs writes:
There was once a consensus among astronomers that the heavens were static, that the boundaries of the universe constant. But in 1929, Hubble observed his red shift among the stars, overturning that consensus. In 1904, there was a consensus among physicists that Newtonian mechanics was, at last, the final word in explaining the workings of the [universe]. All that was left to do was to mop up the details. But in 1905, Einstein and a few others soon convinced them that this view was false.
Consensus can also cause disaster, as NASA proved with a consensus of management that solid rocket booster O-rings affected by unusual cold weren’t worth worrying about or that a foam strike during launch wouldn’t damage the wing of the space shuttle and were “not even worth mentioning”.
Clearly, the power of thousands in agreement on scientific consensus can’t stand up to stubborn facts and that is the self-correcting process of science which sometimes works slowly, other times dramatically quickly. Given that consensus by itself means nothing in the face of such facts, it seems to me that consensus is just another manifestation of herd-like thinking as illustrated by Mackay.
From the Amazon summary of Mackay’s insightful book on crowds:
First published in 1841, Extraordinary Popular Delusions and the Madness of Crowds is often cited as the best book ever written about market psychology. Author Charles Mackay chronicles many celebrated financial manias, or ‘bubbles’, which demonstrate his assertion that “every age has its peculiar folly; some scheme, project, or fantasy into which it plunges, spurred on by the love of gain, the necessity of excitement, or the mere force of imitation.” This still holds fast today! Among the alleged ‘bubbles’ described by Mackay is the infamous Dutch tulip mania, the South Sea Company bubble and the Mississippi Company bubble. And what do bubbles do? Why they burst of course.
The Cook et al. paper bubble is about to burst.
UPDATE: Read the draft paper Tol is working on here, comments welcome:
https://docs.google.com/file/d/0Bz17rNCpfuDNM1RQWkQtTFpQUmc/edit
The Guardian has a science blog today about another historical scientific consensus in the 1940´s regarding genetics ( that was blown out of the water in the 1950´s) which might have been 98%. Read about it here http://www.guardian.co.uk/science/blog/2013/jun/03/oswald-t-avery-genetic-science-dna
Boring bases controlling genetics? No way, it had to be those interesting proteins clearly, who could deny that?
@mt
Fair enough. The pattern in the data may just be caused by the infrequency of anything but 4. Need to think about this.
From Desmog blog (aka Asshat City)
[ http://www.desmogblog.com/2013/05/15/climate-denial-s-death-knell-97-percent-peer-reviewed-science-confirms-manmade-global-warming-consensus-overwhelming ]
Co-author Dana Nuccitelli explains the findings on the consensus:
” … 97.1% … 97.2% …
… We found that about two-thirds of papers didn’t express a position on the subject in the abstract, which confirms that we were conservative in our initial abstract ratings. This result isn’t surprising for two reasons: 1) most journals have strict word limits for their abstracts, and 2) frankly, every scientist doing climate research knows humans are causing global warming. … “
I think Anthony has hit the nail fairly square on the head:
Given the miserable condition of the Cook et al. paper the only real question is how deliberate or how subconsciously self-serving the results are. To put a twist on Naomi Orestes: to what extent was it Cook et al.’s intent to ‘Manufacture Consent~sus’. Why did they not leap to the 98% conclusion? Was it just, as Anthony suggests, to reinforce the “97% of climate scientists” meme that has been floating around for years; to suggest that original ‘97% piece of junk’ was correct too; are they afraid that reaching for the ‘98%’ figure sounds a little too like shooting for the Moon; or are the holding the 98% figure in reserve for a later date, “It’s worse than we thought!” We’ll probably never know unless someone wants to cough up their emails.
It’s a pretty sad situation for Science to be in.
The ‘consensus crowd’ is very unlikely to be shifted out of their current position, basically because they will not formally consider the counter position because they have convinced themselves that there IS NO reasonable cause for doubt. In other words the ‘consensus’ side need make almost NO EFFORT to reinforce their own position while feeling utterly confident in being correct; while on the other hand the skeptical position has to take hour and hours of research to digest, analyze, evaluate, and if necessary refute a single ‘consensus’ point in a scientifically convincing way. Then to add insult on top of personal injury, your ‘consensus’ interlocutor will simply declare you “wrong” without having bothered to do the research necessary to evaluate and refute your position.
Intellectual laziness.
The reason I find it impossible to digest Cook et al.’s cooked up 97% figure is that it just does not seem to match reality AT ALL. In the last six years I have read at least fifty peer reviewed journal articles in full, partials and the abstracts for a couple hundred more, hundreds of other scientific articles, encyclopedia entries and essays [as opposed to ‘news’ articles about the science], and several books. To be a skeptic takes hours and hours of time. I know exactly how much work it is to read an make sense of all of this. It’s really hard to keep up. [don’t we all know]
The 97% figure just doesn’t describe the reality I have experienced – it might if you live in a ‘consensus’ bubble and your only sources of information of the subject are: NPR, the New York times, Scientific American, and Science News. Even if you are a die-hard: RealClimate, Deltoid, Tamino, SkS, consensus type you’ve got to know the 97% figure has no basis in reality because you spend so much of your life refuting the papers of deniers’ “pseudo-science” – tell me does that feel to you like only 2-3% of all that’s out there?
One thousand-nine-hundred-twenty-two abstracts rated by a single individual??? [sure sounds like a lot when you write the number out] Given a mere three minutes to: carefully read for comprehension, analyze, rate, and tabulate each scientific abstract, that sums to a minimum of ninety-six hours of effort by that ‘volunteer’, not counting breaks, to complete the project – more than two full work weeks – almost three work weeks if you live in France. [actually if you consider that the average French factory worker really ‘labors’ three hours a day: 32 work weeks] How good could the quality of that individuals efforts be? forget about bias. Did the Cook et al. methodology have any mechanism for screening out people who were skimming though the abstracts, or trying to cram in 96 hours of reading into one 24 hour day?
Even if the 97% figure were true, or close to being true, it would still prove nothing – except that you can stack cow-pies pretty high if you try, and that skeptics’ assertions that the peer reviewed science has been systematically skewed in favor of ‘consensus’ may be true.
The only real solution is to hunt out and eliminate bias in the science, where ever it is found, and not to put up with intellectual laziness.
W^3
On a slightly different note.
Here’s an interesting link to a study of the behaviors of different herds’ thinking over the years by Charles Murray back in 2009.
http://www.aei-ideas.org/2009/08/the-white-house-and-the-pauline-kael-syndrome/
The ‘divergent herd’ shown in the diagram probably feel very proud of themselves for their cleverness in diverging so sharply from the rest of society – fine – but the might ask themselves if maybe they have been in their own little bubble too long and their groups’ thinking has become a little too self-referential, and even if they don’t think everyone in America thinks as they do on Global Warming they ought to.
W^3
Konrad says (June 2, 2013 at 7:51 pm): “A basic question. Are radiative gases critical for convective circulation in the troposphere? Yes or no?”
Ignoring the ambiguity of the question (e.g. define “critical”), I think that Konrad, like Cook, has misclassified some of his “respondents”. For example, I bet Stokes, Shore, Folkerts, Hoffer, and maybe even Cotton would agree with Dr. Spencer on that one. BTW, did Konrad poll these people directly with the above question, or did he deduce their responses from previous discussions?
“In science, it does not matter what 97% believe, even about the basics.”
If by “basics” we mean “experimentally demonstrable”, then it certainly does matter. Anyone who D-Nyes (sorry, mods) replicated experimental results, i.e. facts, is an obvious crackpot, unless he/she can supply the “extraordinary proof” required. Mere assertions, however passionate, don’t count.
On the other hand, for systems not amenable to controlled experiments, e.g. the Earth’s climate system, indeed “consensus” is no guarantee of correctness, especially when “consensus” is manufactured by statistical manipulation, fraud, and–perhaps most important–political manipulation, e.g. government funding. Another dead giveaway is hype, e.g. “coal trains of death”, and (from an idiot on a blog) “fossil fuels kill, period”.
Going with a true consensus can be useful, though, because it saves a lot of time investigating crackpot ideas. To quote Isaac Asimov, “Though many of the products of genius seem crackpot at first, very few of the creations that seem crackpot turn out, after all, to be products of genius.”
http://thethunderchild.com/Sourcebooks/Asimov/AisForAsimov/FactandFancy.html
“Another basic question. In a non radiative atmosphere that has gone isothermal, will average atmospheric temperature will be set by surface Tav or surface Tmax.”
Konrad’s “poll” includes only two respondents and excludes, for example, the entire IPCC, prominent climate scientists not part of the IPCC, etc. Quoting Asimov again, “People without training in a particular field do not know what to doubt and what not to doubt; or, to put it conversely, what to believe and what not to believe. I am very sorry to be undemocratic, but one man’s opinion is not necessarily as good as the next man’s.”
http://www.strbrasil.com.br/English/Res/fact.htm
Anthony, Ed Davey just played the 97% consensus trick in front of Lawson on BBC R4
Which 97%?
@richard Tol
https://twitter.com/dana1981/status/341629760155295745
and
http://discussion.guardian.co.uk/comment-permalink/24047444
Carry on!
Gary Hladik says:
June 3, 2013 at 1:24 pm
——————————————————————————————————————-
“For example, I bet Stokes, Shore, Folkerts, Hoffer, and maybe even Cotton would agree with Dr. Spencer on that one.”
How much? 😉
“Konrad’s “poll” includes only two respondents and excludes, for example, the entire IPCC, prominent climate scientists not part of the IPCC, etc. Quoting Asimov again, “People without training in a particular field do not know what to doubt and what not to doubt; or, to put it conversely, what to believe and what not to believe. I am very sorry to be undemocratic, but one man’s opinion is not necessarily as good as the next man’s.”
This reads as the classic “call to authority” argument. This is one of the errors in reasoning the scientific method seeks to circumvent. In establishing the role of “Climate Scientist” at the start of the hoax, those involved have sought to establish themselves as the “authority”. Call to authority only works on those activists, journalists and politicians with poor scientific literacy. (why has it worked on you, Gary?) Science is not a religion, institution or qualification. It is a method, a system for advancing working knowledge using repeatable empirical observations and repeatable empirical experiments. Consensus has nothing to do with it.
The use of any number ending with 7 is a retail sales technique.
For some reason, many punters, supposedly, are more attracted to a figure with 7 at the end of it than other numbers.
“Systematic sloppiness” is an interesting oxymoron. Sounds like “planned and deliberate error”, to me.
Konrad says (June 3, 2013 at 6:51 pm): “How much? ;-)”
Short answer: probably about as much as Dr. Spencer does.
Long answer: If you want to know “how much” someone agrees with you, your “poll” should have multiple choice options (e.g. “strongly agree”, “partly agree”) instead of just “yes/no”.
“This reads as the classic ‘call to authority’ argument.”
Not surprisingly you’ve totally misread it. When it comes to science, all men are most assuredly not created equal. If Dr. Spencer, for example, were to claim that so-called “greenhouse gases” on balance cool the Earth’s surface, then I’d at least pay attention to his evidence. If a kitchen climatologist who’s just discovered convection claims the same thing, well, he should come back when he has more. Do you, Konrad, actually listen to a potential crackpot with no track record as seriously as you do to a reputable scientist?
Again, a genuine scientific consensus (not a political one) saves a lot of time. If you haven’t read Dr. Asimov’s essay yet, I highly recommend it.
“Call to authority only works on those activists, journalists and politicians with poor scientific literacy.”
Sez who? I reject your authority on the subject. Provide evidence for your claim, and perhaps I’ll listen. 🙂
“why has it worked on you, Gary?”
For the same reason it has worked on you. Do you verify every scientific fact you’ve been taught? Have you verified the IR absorption spectra of CO2, methane, water vapor, etc.? Have you dissected a dichroic incandescent lamp and verified its advertised method of operation? Scientific “authority”/”consensus” saves a lot of time. Sure, it raises the bar for genuine consensus-busting breakthroughs, but it also raises the bar against far-more-abundant quackery.
“Science is not a religion, institution or qualification.”
Actually it’s all of the above (“I have faith in the scientific method”), but scientists can be and have been persuaded by evidence.
Gary Hladik says:
June 4, 2013 at 11:44 am
———————————————-
My “How much?” was actually my response to your “I bet”. Screenshots for the warmist, slayer and sleeper responses to the question upon receipt of stamped envelope. 😉
“If Dr. Spencer, for example, were to claim…” Call to authority again?
“kitchen climatologist” Ad Hominum argument. You have not challenged the physics demonstrated by the experiments. Are you suggesting that if they were conducted in the lab, lounge or patio that they would produce alternate results? (actually only experiment 3 was conducted in a kitchen.)
“just discovered convection” Ad Hominum again? No. Experiment 3 was simply to illustrate to less informed that energy loss as well as energy input can drive convection in a fluid. Many, including most AGW believers, don’t understand this.
“potential crackpot with no track record”. Should I tell you which technology museum to find my work displayed in? That may be construed as a call to authority. 😉
You argue everything but the science. You have made ad hominum arguments. You have used arguments from authority. You have claimed consensus. These are the very reasons the AGW hoax failed. The use of these very techniques ignited the global sceptic movement that destroyed the hoax and yet you persist?
The AGW hoax is all but over, but the fallout is only just beginning. In the age of the Internet it is going to be savage. The activists, journalists, subsidy farmers and politicians who sought to promote or profit by this hoax are not just up against a few hundred thousand sceptics. They will be facing literally billions of people seeking vengeance who also have instant access to the permanent Internet record of the hoax and its players. It is far, far too late to engineer any “soft landing”.
June 2, 2013 at 10:58 pm
[snip – more Slayers junk science from the banned DOUG COTTON who thinks his opinion is SO IMPORTANT he has to keep making up fake names to get it across -Anthony]
Anthony! Nooo! Please, please put that one back.
The post at June 2, 2013 at 10:58 pm is where Doug Cotton tries to “re-brand” PSI by saying –
“Back radiation can indeed slow radiative surface cooling.”
That one is a keeper 😉
Konrad says (June 4, 2013 at 9:12 pm): “You have not challenged the physics demonstrated by the experiments.”
Once again, for the reading-impaired: I don’t challenge the “physics” of your experiments (and I don’t think the IPCC or Dr. Spencer do either). I challenge the extrapolation of such small-scale experiments to the whole earth/atmosphere system in support of your assertion. Your claims are extraordinary, you have not provided extraordinary evidence.
“You argue everything but the science. You have made ad hominum arguments. You have used arguments from authority. You have claimed consensus. These are the very reasons the AGW hoax failed.”
1) It hasn’t “failed”. It’s still going strong, unfortunately.
2) The problem with the alamists’ “consensus” is that a manufactured consensus isn’t one.
“The use of these very techniques ignited the global sceptic movement that destroyed the hoax and yet you persist?”
I would argue that it wasn’t the methods of the warmists that provoked the skeptics, it was both the lack of scientific evidence for their claims and the extraordinarily destructive “remedies” they proposed.
“The AGW hoax is all but over, but the fallout is only just beginning. In the age of the Internet it is going to be savage. [snip hopelessly naive dreams of retribution]”.
We’ve been here before. I think your vision is…unlikely.
Look, Konrad, I’m flattered by all the attention, and I do enjoy our discussions (no, really). But again, I’m not the guy you should be spending your time on. Suppose you convert me to your view. So if/when you submit your paper and add, “Furthermore, since I’ve convinced the arch-skeptic GARY, you should fast-track this submission for publication!”, do you think they’ll cut you any slack? You could probably spend your time more profitably gathering that extraordinary evidence I mentioned.
Gary Hladik says:
June 5, 2013 at 12:03 am
————————————————————————————————–
“Your claims are extraordinary, you have not provided extraordinary evidence.”
No not extraordinary. Just boring gas conduction and fluid dynamics really. The experiments provided therefore don’t need to be extraordinary. Speaking of extraordinary claims, how does this sound? –
“Adding radiative gases to the atmosphere will reduce the atmospheres radiative cooling ability”
– not hard to guess why “Travesty” Tremberth wanted the null hypothesis reversed in the case of the AGW claims now is it? 😉
“It hasn’t “failed”. It’s still going strong, unfortunately.”
– The heat is hiding deep in the oceans! It will be back!
– The warming is being masked by aerosols!
– Natural variability greater than previously estimated!
– Climate sensitivity may be less than previously estimated!
– Did we say 6C? No, no we meant 2C!
All within the last couple of months? That sounds like the panicked squealing of warmist weasels to me. How does it sound to you?
“I’m flattered by all the attention”
As you would be now aware, I don’t waste much time on “Assault Clowns” (PSI), “Snowstormers” or “Popcorn Warriors”. “Sleepers” however…
“arch-skeptic GARY”
From a previous thread –
“I suspect you misunderstand my motivation and entertainment. […]Sceptics are self motivated. They observe each other and choose an area in which they can contribute. I do not waste energy on debating PSI Assault Clowns. Identifying and outing “Sleepers” is so much more fun ;-)”
Come on Gary, I even typed it in the clear!
“and I do enjoy our discussions”
As do I Gary. As do I. 😉
Konrad says (June 5, 2013 at 1:14 am): “Identifying and outing ‘Sleepers’ is so much more fun ;-)”
What’s a “Sleeper”?
I can’t find Monckton’s quotes in the “Environmental Research Letters” or in “Skeptical Science.” Assuming the texts have been altered, were the originals archived? Where? –AGF
Dr. Craig Loehle made a good comment objecting to the whole approach to classifying all “climate change” related papers ala the Cook et al. study:
http://climateaudit.org/2013/05/24/undercooked-statistics/#comment-420907