UPDATE: comments welcome on Dr. Richard Tol’s draft paper on this issue, see below. This will be a top post for a day, new posts will appear below this one – Anthony
“Men, it has been well said, think in herds; it will be seen that they go mad in herds, while they only recover their senses slowly, and one by one.”
That is from Charles Mackay in his book, Extraordinary Popular Delusions and the Madness of Crowds first published in 1841.
I think it is an apt description of the process that led to Cook et al. (2013) Quantifying the consensus on anthropogenic global warming in the scientific literature because that paper is in fact, a product of a crowd evaluating a crowd. As an example, Dr. Richard Tol has just discovered that using Cook’s own data, the consensus number Cook should have published is 98%, rather than 97%.
Dr. Tol writes in a critique of the Cook et al. paper:
In fact, the paper by Cook et al. may strengthen the belief that all is not well in climate research. For starters, their headline conclusion is wrong. According to their data and their definition, 98%, rather than 97%, of papers endorse anthropogenic climate change. While the difference between 97% and 98% may be dismissed as insubstantial, it is indicative of the quality of manuscript preparation and review.
He shows the Cook data as he compiles it:
You’d think such simple elementary errors in data would have been caught in peer review, after all, that is what peer review is for.
I think that there was a goal by Cook and his crowd, and that goal was to match the 97% number that has become a popular meme in the literature and the media. This intent seems confirmed by a recent statement by one of the co-authors, Dana Nuccitilli in a media argument that 97% global warming consensus meets resistance from scientific denialism
However, we have used two independent methods and confirmed the same 97% consensus as in previous studies.
It is that branding of “denialism” by Nucciltelli to Dr. Tol, who is hardly a “denier” on climate change even by the loosest definition, that has given Tol incentive to now start systemically deconstructing the paper. It also lends a window into the mind of the coauthor Nucitelli, who can’t seem to assimilate useful criticisms, no matter how valid, but instead publicly attributes discovery of real errors in the Cook et al. paper to “denialism” rather than the self-correcting process of science. Nuccitelli’s actions suggest to me, a mindset of zealotry, rather than one of discovery. His actions of branding Dr. Tol’s and others valid criticisms, seem to fit the textbook definition of the word:
As an aside, it seems truly laughable that the Guardian has created an entire regular opinion column based and named on this 97% number, and it supports that idea that this was the “target number” rather than the number that the actual data would report. Richard Tol has just proven their own data doesn’t even match the title of their paper. Will the Guardian now correct the title?
Tol goes on to say this about the crowd-sourcing:
The results thus depend on the quality of the volunteers. Are they neutral observers, or are they predisposed to endorsing or rejecting anthropogenic climate change? Did they suffer from fatigue after rating a certain number of abstracts? 12 volunteers rated on average 50 abstracts each, and another 12 volunteers rated an average of 1922 abstracts each. Fatigue may well have a problem. This level of effort by a volunteer could indicate a strong interest in the issue at hand.
Indeed, and he backs this up by saying it is evident in the data:
WoS generates homoskedastic data. Rating made the data heteroskedastic. Sign of tiredness or manipulation.
So which is it? Tiredness or manipulation, or perhaps both? Based on what has been observed so far, I’d say there is a combination, but given the obvious 97% target, more likely it is an unconscious manipulation by the chosen crowd of volunteer reviewers, which included no climate skeptics and consisted of mostly insiders for Cook’s antithetically named website, “Skeptical Science”. Tol goes on to comment:
No neutral person would volunteer to do 1922 tasks. Cook’s data duly show bias: 35% of abstract were misclassified, 99% towards endorsement.
To support the idea that bias played a role in reaching the conclusions of the Cook et al. paper, there seems to be a systemic sloppiness in the sampling process, as Tol points out in his critique:
In fact, 34.6% of papers that should have been rated as neutral were in fact rated as non-neutral. Of those misrated papers, 99.4% were rated as endorsements. It is therefore reasonable to assume that the volunteers were not neutral, but tended to find endorsements where there were none. Because rater IDs were not reported, it is not possible to say whether all volunteers are somewhat biased or a few were very biased.
Tol also says this about the 97% scientific consensus claim:
It is a strange claim to make. Consensus or near-consensus is not a scientific argument. Indeed, the heroes in the history of science are those who challenged the prevailing consensus and convincingly demonstrated that everyone thought wrong. Such heroes are even better appreciated if they take on not only the scientific establishment but the worldly and godly authorities as well.
Well known examples of this include the challenges to the theory that Earth was the center of the universe, that infection was spread by surgeons who didn’t wash their hands, that the Earth’s crust had plates that moved, and that gastric ulcers were caused by a bacterial infection, and not stress as physicians once widely believed. As William Briggs writes:
There was once a consensus among astronomers that the heavens were static, that the boundaries of the universe constant. But in 1929, Hubble observed his red shift among the stars, overturning that consensus. In 1904, there was a consensus among physicists that Newtonian mechanics was, at last, the final word in explaining the workings of the [universe]. All that was left to do was to mop up the details. But in 1905, Einstein and a few others soon convinced them that this view was false.
Consensus can also cause disaster, as NASA proved with a consensus of management that solid rocket booster O-rings affected by unusual cold weren’t worth worrying about or that a foam strike during launch wouldn’t damage the wing of the space shuttle and were “not even worth mentioning”.
Clearly, the power of thousands in agreement on scientific consensus can’t stand up to stubborn facts and that is the self-correcting process of science which sometimes works slowly, other times dramatically quickly. Given that consensus by itself means nothing in the face of such facts, it seems to me that consensus is just another manifestation of herd-like thinking as illustrated by Mackay.
From the Amazon summary of Mackay’s insightful book on crowds:
First published in 1841, Extraordinary Popular Delusions and the Madness of Crowds is often cited as the best book ever written about market psychology. Author Charles Mackay chronicles many celebrated financial manias, or ‘bubbles’, which demonstrate his assertion that “every age has its peculiar folly; some scheme, project, or fantasy into which it plunges, spurred on by the love of gain, the necessity of excitement, or the mere force of imitation.” This still holds fast today! Among the alleged ‘bubbles’ described by Mackay is the infamous Dutch tulip mania, the South Sea Company bubble and the Mississippi Company bubble. And what do bubbles do? Why they burst of course.
The Cook et al. paper bubble is about to burst.
UPDATE: Read the draft paper Tol is working on here, comments welcome:
https://docs.google.com/file/d/0Bz17rNCpfuDNM1RQWkQtTFpQUmc/edit
“P. Hager says:
June 2, 2013 at 1:15 pm
Mackay’s book is available on Project Gutenberg for download.
http://www.gutenberg.org/ebooks/24518”
Thanks! I have been meaning to read it for some time. I downloaded a copy and it is high-quality, quite readable, with excellent engravings.
***********************************
That Mankind has some affect on climate is trivially true – so do termites, bacteria, insects, and all the rest of God’s creatures and flora. And if our affect on climate is trivially true, it is trivial. Interesting to ponder to be sure, interesting to attempt to tease-out just what our climate signal is; but trivial.
What Cook and Co have done is come up with a marketing gimmick – call it an advert.
Here’s a nice (but old fashioned) advert for the cigarette Doctors recommend:
http://tinyurl.com/lz9hmbu
Same baloney appeal to authority; just a different time, a different fad, with differing motivations and methods of funding. Yawn.
But returning to our affect on climate and on the earth:
http://earthsky.org/science-wire/elevated-carbon-dioxide-making-arid-regions-greener
Maybe we aren’t all bad us humans?
@ur momisugly P. Hager, thanks for the Gutenberg link…
As I’ve said in other threads on this topic, a simple step would be to fund George Mason U. to re-do their two prior surveys of AGU & AMU members. Probably additional questions should be asked.
Also, papers that are not “attribution” papers but impact or policy papers should be marked separately and given less weight.
Also, more recent papers should be given more weight.
And of course the “C” question should be distinguished from the AGW question, and the “A” question from the GW question.
PS: Solomon’s book, The Deniers, made a big point of the fact that its “deniers” mostly accepted AGW theory, except in the fields of their own expertise. They took the consensus elsewhere on trust, IOW. So any survey should ask questions about belief in AGW generally vs. in their own field.
rogerknights says:
June 2, 2013 at 3:59 pm
PS: Solomon’s book, The Deniers, made a big point of the fact that its “deniers” mostly accepted AGW theory, except in the fields of their own expertise. They took the consensus elsewhere on trust, IOW. So any survey should ask questions about belief in AGW generally vs. in their own field.
That’s something that I noticed early on in investigating this – it’s primarily because the general fake fisics memes have been introduced into the education system, from where people specialise they will learn real physics, but unless they have a reason to question the other memes in the varied fields they will take these as ‘background basics’.
This has the added confusion of people arguing at cross purposes because they think they’re arguing about the same physics. Hence the often acrimonious discussions between those who are talking about real gases and the AGW/CAGW’s who think the atmosphere is empty space populated by ideal gas, without either side understanding that these are two completely different sets of basics which produce completely different effects.
I’d say they should have published 98% simply because that’s what 97.6% rounds to. But, yeah, here’s a statistical scientist telling you there are so many things wrong with this conclusion to begin with, this this becomes an almost mute point…
From Google Books
Memoirs of extraordinary popular delusions and the madness of crowds By Charles Mackay
Climate Change grants cause prostitution.
Someone made the point in an earlier discussion about taking information from the abstracts was that in several, iirc, that he checked out the abstract contained ‘confirmation’ of AGW in some form or other, while the body of the research didn’t.
I’ve read counless papers over the last years which detail real science research and will then add a paragraph or even a line at the end making some reference to its posssible importance in ‘global warming’ or now, ‘climate change’
In producing this “study”, John Cook’s idiocy is writ large. Richard Tol has revealed the extent of Cook’s mendacity and incompetence in his flawed methodology. Cook’s idiocy however, is in not understanding that it was the use of terms such as “consensus”, “the science is settled” and “the debate is over”, that sewed the seeds of the inevitable destruction of the hoax.
Those who understand the importance and purpose of the scientific method know that a primary function is providing a workable method for challenging consensus. Claiming “consensus” was in effect waving a red flag to a bull. The methods used to influence activists, journalists and politicians of limited scientific literacy had the side effect of igniting a global sceptical movement amongst those with higher scientific literacy. This was never going to end well.
John Cook is not trying to save the AGW hoax through this study. He is not quite that foolish. This appears to be part of an attempt to establish an exit strategy for some of the fellow travellers in the AGW hoax. “We got it wrong, but in our defence there was a consensus.” Sadly, consensus carries no scientific weight. Its political value is also limited if the claimed consensus is only amongst those who started self identifying as “climate scientists” after the start of the hoax. Global warming has been in effect a global IQ test with results permanently recorded on the internet. For those that failed, this claimed consensus will be no defence.
Perhaps they should change the title of their blog from “Skeptical Science” to “Cooked Science”
Jay says:
June 2, 2013 at 9:42 am
“This is just a case of people doing what they have been paid to do.. Its like hit men complaining that killing people is illegal.. Its true, but has nothing to do with that envelope stuffed full of money and that job that has just been paid for..
So you double down on the idea that you are a professional and do what you have to do, knowing full well that your name could be the next name if you don’t watch your step..
The underworld has peer / pal review as well..”
Well said!!
DrJohnGalan says:
June 2, 2013 at 11:34 am
“Another excellent example of this sort of bubble is in the field of cold fusion or low energy nuclear reactions. Since 1989 almost the whole of the scientific community “herd” has thought that “cold fusion” is junk science despite many hundreds of replications of the original experiment that produced anomalous heat.”
Unfortunately, much like a cheap nearly 100% cure for cancer, workable cheap LENR energy technology would rapidly upset the economic applecart inspired by corporate and world power energy structure. I only fore see these kinds of revolutionary/rapid improvements being (effectively) squished before they can have much effect. The technologies “might” quietly slip in temporarily while being accused of fakery/fraud but even then it is hard to believe that they would be allowed to be what would be seen as a ruinative part of the economic pie. I don’t believe that it is ever about the people at the low end but only those ruling corporate and political elites, the rest of us are just along for the ride. As many historical scandals/other examples (and J.Edgar Hoover) have shown, ethics & morality don’t really apply to the upper echelons. I think that those receiving grants in CAGW realize this and many are just going along on the “Right Side”. It’s all rather sad really………..
johanna says (June 2, 2013 at 2:47 pm): “Frankly, given the early stage that climate science is at, I’d be surprised if 97% of climate scientists agreed on very much at all, if you got down to specifics.”
My guess is that at least 97% of both “warmist” and “skeptic” climate scientists agree on the basics that have been experimentally demonstrated, despite the occasional debates that break out on climate blogs.
The Earth as a whole, however, is a very difficult* system to explore experimentally, so once you get past the basics, opinions differ, so to speak.
* unless of course you have one or more Earths to use as controls
Gary Hladik,
“agree on the basics”
and
“once you get past the basics”
Just like to know what you are talking about. Basically.
pokerguy says:
June 2, 2013 at 9:08 am
“Many have, but it does no good. I keep hammering away on the need for an actual, statistically valid survey to counter this 97 percent canard, but no one seems interested. More enjoyable I guess, and far easier to sit back and complain about biased surveys, and gullible warmists, and to quote noble sounding observations about the nature of science.”
A serious survey to find consensus must be based on a set of hypotheses that everyone agrees are the fundamental hypotheses of AGW, CAGW, or whatever interests us. Asking whether manmade CO2 contributes to rising temperatures cannot provide useful information. Learning from a survey based on a series of key hypotheses that seventy percent believe that cloud cover is a negative feedback would provide us with a great deal of information. Learning that ninety percent believe that climate sensitivity is less than one degree would provide us with important information.
Albert Einstein sat in his small office thinking of a street car going faster and faster away from the clock on the city hall across..
Took him years and years to do the math and it took a solar eclipse to find the red shift and prove it all up.
Michael Mann sat in his small office thinking first of his “red shift” and the need to spend tax money faster and faster to spread the wealth around hard pressed for facts he dreamed up the hockey stick graf.
Sort of like that.
Dream up the lie, Rig the data to fit the lie. Trying to save the time it took Albert etal.
I prefer Tol’s observation that consensus isn’t science, except I’d say it stronger…it’s ANT-SCIENCE.
It has been historically used by scientists to maintain the status quo…to prevent new observations or theories from interfering in careers that have been established based on the status quo. It’s a political concept, not a scientific one, and it can’t be used to find scientific truths. Instead, it is essentially scientific bullying…the science complex’s equivalent of high school peer pressure.
Based on the material and commentary I’ve read by Dana, it’s the perfect tool for him. Look at how his first response was to try to bully Tol into following the “Consensus” by hitting him with the “D” word. I suspect he was absolutely shocked when Tol didn’t instantly cave.
This should be the meme: consensus is anti-science.
I think the term “ochlocracy” would be a good fit.
Poems of Our Climate says (June 2, 2013 at 6:14 pm): “Just like to know what you are talking about. Basically.”
Some non-controversial examples would be the spectra of incoming sunlight at the top of the atmosphere and at the Earth’s surface, the Stefan-Boltzmann Law, the measurements of CO2 at Mauna Loa and other sites, the measured emissivities of various materials, the absorption and emission spectra of atmospheric gases, conservation of energy, and so on.
@ur momisugly pokerguy says: June 2, 2013 at 8:23 am
If indeed those 97 or 98% had done independent research (which Climategate proved was not the case), then I would be just like you – having to re-examine the contradictions. But all we know from this latest escapade from Cook and SkS is that they WANT to create a crowd sourcing. But cannot even do that to convince their own side (not team).
It seems to me that we should agree that 97% of a carefully selected group of papers or authors whose “research” is intended to support the thesis that supports them do in fact support the thesis that supports them. Well, then, this is equivalent to “the science is settled”, particularly since there is no meaningful difference between the 97% reported, and the 100% consensus that could certainly have been claimed with just a little more massaging of the study population.
The obvious result of that consensus is that no more research is needed to prove the thesis, and the obvious, appropriate response to no more need for research is to cancel all funds for all such “researchers”, who are now completely redundant. Maybe they can find jobs in the Indian railway system.
A recent empirical study from Yale looks at views on climate change and concludes, inter alia, that “[m]embers of the public with the highest degrees of science literacy and technical reasoning capacity were not the most concerned about climate change.” http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2193133. I wonder how Cook squares his conclusions with this study?
Gary Hladik says:
June 2, 2013 at 6:04 pm
——————————————————————–
In science, it does not matter what 97% believe, even about the basics.
A basic question. Are radiative gases critical for convective circulation in the troposphere? Yes or no?
Dr. Spencer – “Yes”
Konrad – “Yes”
Doug Cotton – “No”
Nick Stokes – “No”
Joel Shore – “No”
Tim Folkerts – “No”
Davidmhoffer – “No”
TonyB- “No”
There appears to be a consensus. Who is right?
Another basic question. In a non radiative atmosphere that has gone isothermal, will average atmospheric temperature will be set by surface Tav or surface Tmax.
Dr. Spencer – “surface Tav”
Konrad – “surface Tmax”
Who is right?
A primary function of the scientific method is to provide a reasonably safe and reliable method for even a single person to challenge prevailing consensus. 97% of planetary scientists can agree on the basics of luna regolith composition. If those 97% believe it to be green cheese, they are still wrong.
pokerguy says:
June 2, 2013 at 11:05 am
“Philosophy of science considerations and lofty, misty-eyed rhetoric about heroic renegade researchers are fine. IN the real world, you fight in any way you can. Once more, this is a PR war. It it were really just about the science, the skeptics would already have been declared the winners.”
The vast majority of us are here in defense of science. Much of politics today is just about PR. What makes the AGW debate different is that participants must stand or fall depending on the quality of their science. I wish the same were true in the politics of free speech, one of many examples. If it were true, I would be heavily engaged in that debate.
johanna says:
June 2, 2013 at 2:47 pm
“Frankly, given the early stage that climate science is at, I’d be surprised if 97% of climate scientists agreed on very much at all, if you got down to specifics. Broad statements like “human activity is affecting the weather” are trivially true, but meaningless. When it comes to quantifying and weighting the factors that drive climate, I suspect that there is plenty of variation even within the CAGW “consensus” crowd.”
Exactly. How many climate scientists are lined up to embrace Trenberth’s most recent revelation that the heat is hiding in the deep oceans? A scientific survey of opinion on the main hypotheses of climate science would reveal great disagreement.