UPDATE: comments welcome on Dr. Richard Tol’s draft paper on this issue, see below. This will be a top post for a day, new posts will appear below this one – Anthony
“Men, it has been well said, think in herds; it will be seen that they go mad in herds, while they only recover their senses slowly, and one by one.”
That is from Charles Mackay in his book, Extraordinary Popular Delusions and the Madness of Crowds first published in 1841.
I think it is an apt description of the process that led to Cook et al. (2013) Quantifying the consensus on anthropogenic global warming in the scientific literature because that paper is in fact, a product of a crowd evaluating a crowd. As an example, Dr. Richard Tol has just discovered that using Cook’s own data, the consensus number Cook should have published is 98%, rather than 97%.
Dr. Tol writes in a critique of the Cook et al. paper:
In fact, the paper by Cook et al. may strengthen the belief that all is not well in climate research. For starters, their headline conclusion is wrong. According to their data and their definition, 98%, rather than 97%, of papers endorse anthropogenic climate change. While the difference between 97% and 98% may be dismissed as insubstantial, it is indicative of the quality of manuscript preparation and review.
He shows the Cook data as he compiles it:
You’d think such simple elementary errors in data would have been caught in peer review, after all, that is what peer review is for.
I think that there was a goal by Cook and his crowd, and that goal was to match the 97% number that has become a popular meme in the literature and the media. This intent seems confirmed by a recent statement by one of the co-authors, Dana Nuccitilli in a media argument that 97% global warming consensus meets resistance from scientific denialism
However, we have used two independent methods and confirmed the same 97% consensus as in previous studies.
It is that branding of “denialism” by Nucciltelli to Dr. Tol, who is hardly a “denier” on climate change even by the loosest definition, that has given Tol incentive to now start systemically deconstructing the paper. It also lends a window into the mind of the coauthor Nucitelli, who can’t seem to assimilate useful criticisms, no matter how valid, but instead publicly attributes discovery of real errors in the Cook et al. paper to “denialism” rather than the self-correcting process of science. Nuccitelli’s actions suggest to me, a mindset of zealotry, rather than one of discovery. His actions of branding Dr. Tol’s and others valid criticisms, seem to fit the textbook definition of the word:
As an aside, it seems truly laughable that the Guardian has created an entire regular opinion column based and named on this 97% number, and it supports that idea that this was the “target number” rather than the number that the actual data would report. Richard Tol has just proven their own data doesn’t even match the title of their paper. Will the Guardian now correct the title?
Tol goes on to say this about the crowd-sourcing:
The results thus depend on the quality of the volunteers. Are they neutral observers, or are they predisposed to endorsing or rejecting anthropogenic climate change? Did they suffer from fatigue after rating a certain number of abstracts? 12 volunteers rated on average 50 abstracts each, and another 12 volunteers rated an average of 1922 abstracts each. Fatigue may well have a problem. This level of effort by a volunteer could indicate a strong interest in the issue at hand.
Indeed, and he backs this up by saying it is evident in the data:
WoS generates homoskedastic data. Rating made the data heteroskedastic. Sign of tiredness or manipulation.
So which is it? Tiredness or manipulation, or perhaps both? Based on what has been observed so far, I’d say there is a combination, but given the obvious 97% target, more likely it is an unconscious manipulation by the chosen crowd of volunteer reviewers, which included no climate skeptics and consisted of mostly insiders for Cook’s antithetically named website, “Skeptical Science”. Tol goes on to comment:
No neutral person would volunteer to do 1922 tasks. Cook’s data duly show bias: 35% of abstract were misclassified, 99% towards endorsement.
To support the idea that bias played a role in reaching the conclusions of the Cook et al. paper, there seems to be a systemic sloppiness in the sampling process, as Tol points out in his critique:
In fact, 34.6% of papers that should have been rated as neutral were in fact rated as non-neutral. Of those misrated papers, 99.4% were rated as endorsements. It is therefore reasonable to assume that the volunteers were not neutral, but tended to find endorsements where there were none. Because rater IDs were not reported, it is not possible to say whether all volunteers are somewhat biased or a few were very biased.
Tol also says this about the 97% scientific consensus claim:
It is a strange claim to make. Consensus or near-consensus is not a scientific argument. Indeed, the heroes in the history of science are those who challenged the prevailing consensus and convincingly demonstrated that everyone thought wrong. Such heroes are even better appreciated if they take on not only the scientific establishment but the worldly and godly authorities as well.
Well known examples of this include the challenges to the theory that Earth was the center of the universe, that infection was spread by surgeons who didn’t wash their hands, that the Earth’s crust had plates that moved, and that gastric ulcers were caused by a bacterial infection, and not stress as physicians once widely believed. As William Briggs writes:
There was once a consensus among astronomers that the heavens were static, that the boundaries of the universe constant. But in 1929, Hubble observed his red shift among the stars, overturning that consensus. In 1904, there was a consensus among physicists that Newtonian mechanics was, at last, the final word in explaining the workings of the [universe]. All that was left to do was to mop up the details. But in 1905, Einstein and a few others soon convinced them that this view was false.
Consensus can also cause disaster, as NASA proved with a consensus of management that solid rocket booster O-rings affected by unusual cold weren’t worth worrying about or that a foam strike during launch wouldn’t damage the wing of the space shuttle and were “not even worth mentioning”.
Clearly, the power of thousands in agreement on scientific consensus can’t stand up to stubborn facts and that is the self-correcting process of science which sometimes works slowly, other times dramatically quickly. Given that consensus by itself means nothing in the face of such facts, it seems to me that consensus is just another manifestation of herd-like thinking as illustrated by Mackay.
From the Amazon summary of Mackay’s insightful book on crowds:
First published in 1841, Extraordinary Popular Delusions and the Madness of Crowds is often cited as the best book ever written about market psychology. Author Charles Mackay chronicles many celebrated financial manias, or ‘bubbles’, which demonstrate his assertion that “every age has its peculiar folly; some scheme, project, or fantasy into which it plunges, spurred on by the love of gain, the necessity of excitement, or the mere force of imitation.” This still holds fast today! Among the alleged ‘bubbles’ described by Mackay is the infamous Dutch tulip mania, the South Sea Company bubble and the Mississippi Company bubble. And what do bubbles do? Why they burst of course.
The Cook et al. paper bubble is about to burst.
UPDATE: Read the draft paper Tol is working on here, comments welcome:
https://docs.google.com/file/d/0Bz17rNCpfuDNM1RQWkQtTFpQUmc/edit
It is a great mystery to me why the number 97% seems to have this almost mystic importance for alarmists. Is it to defend the bad old Doran-paper?
It looks quite silly to return again and again to this number. Much more convincing if these surveys had varied between for instance 80 and 93%. The propaganda point of massive CAGW-support would have been better made that way. The insistence on 97 only serves to stir up memories of referendums in totalitarian states, where a puny minority was allowed to deviate from the party line, in order to create an illusion of free elections.
But then, some alarmists have suggested that CAGW is too important to suffer the whims of a democracy.
Latitude says:
June 2, 2013 at 10:44 am
pokerguy says:
June 2, 2013 at 9:08 am
The truth is that 97 percent number would be compelling if real. It needs to be efficiently and legitimately countered.
===============
no it doesn’t….LOL
The fact is after all these decades….they are still trying to convince people that it’s real
—————————————————–
I consider myself a skeptic when it comes to man made CO2 being the one and only major problem which must be tackled ….. now. The story is fabulous and the solution is a lie.
I’m not overly concerned if the price we pay for abundant cheap energy (and the happiness this energy brings to humans) is an increase of couple of degrees in temperature and swapping some newly unfrozen usable land for a new coastline. A reasonable man would look at this from a global perspective and say, “If this is their “C” in cagw then it sounds like a win – win situation.”
I consider myself a reasonable skeptic.
cn
StephenP –
Daily Mail has, perhaps, the most hilarious coverage and Cook’s 97% Consensus gets Davey’s nod!
3 June: UK Daily Mail: Matt Chorley: ‘Blinkered’ climate change deniers accused of ‘dangerous, publicity-seeking, bloody-mindedness’ by Energy Secretary Ed Davey
The Lib Dem Cabinet minister will use a major speech to condemn those who ‘deny the reality of climate change itself’.
He will accuse people who argue the planet is not warming are ‘absolutely wrong and really quite dangerous’.
Mr Davey’s speech comes as he faces fierce criticism from his own party for dropping targets for cutting carbon emissions by 2030…
Speaking at the launch of a new Met Office climate change service, Mr Davey will argue that ‘healthy scepticism’ about the science of climate change is part of the scientific process.
But he will blame ‘some sections of the press’ for giving an ‘uncritical campaigning platform’ to people and campaigners who reject the idea that climate change is a result of human activity and some who ‘even deny the reality of climate change itself’.
Mr Davey will say: ‘This is not the serious science of challenging, checking and probing.
‘This is destructive and loudly clamouring scepticism born of vested interest, nimbyism, publicity seeking contraversialism or sheer blinkered, dogmatic, political bloody-mindedness.’
He claims this ‘tendency’ seize on any scientific uncertainty as proof that green policy and investment in renewable power is ‘hopelessly misguided’…
Mr Davey will add: ‘By selectively misreading the evidence, they seek to suggest that climate change has stopped so we can all relax and burn all the dirty fuel we want without a care.
‘This is a superficially seductive message, but it is absolutely wrong and really quite dangerous.’
He will highlight a recent survey of over 12,000 peer-reviewed climate science papers which he says ‘provides a startling picture of the consensus that exists in our scientific community’…
Tory MP Tim Yeo and Labour’s Barry Gardiner have tabled an amendment to set a target to that the power industry will be carbon-free by 2030.
Businesses, environmental organisations, faith groups and trade unions are all calling on MPs to back the move in a vote.
However, Chancellor George Osborne has expressed support for a new ‘dash for gas’ instead of a drive toward renewables and other low-carbon technology in the 2020s.
The Government’s own climate advisers have backed the target to cut emissions by 2030 and said that investing in low-carbon power such as wind farms and nuclear reactors in the 2020s could save consumers billions of pounds compared to relying on gas.
Greenpeace executive director John Sauven said that the amendment was very important because without it the Energy Bill became ‘a bit aimless’.
‘This is the point the investors and business people are making: you’ve got to be really clear about what your route map is and where you want to get to,’ he said…
http://www.dailymail.co.uk/news/article-2335100/Blinkered-climate-change-deniers-accused-dangerous-publicity-seeking-bloody-mindedness-Energy-Secretary-Ed-Davey.html?ito=feeds-newsxml
almost as funny as Davey:
3 June: ABC PM: Environmental investment campaign targets fossil fuels
BILL MCKIBBEN: It’s gone quickly, also, to churches in the United States, but also in Australia, where parts of the Uniting Church have divested their stock in coal companies. Just saying these companies, you know, they’re running Genesis backwards, you know, they’re de-creating this planet, we can’t keep, in good conscience, profiting from the wreckage of the climate…
SARAH CLARKE: So the maths of climate change is what you’re also saying is not being listened to?
BILL MCKIBEEN: That’s right. Climate change in essence is a kind of math problem…
SARAH CLARKE: Looking at Australian politics and an election in September; the Coalition, if it does win as suggested, it will roll back the carbon price. How would that be looked upon internationally, and do you think that really will make a difference?
BILL MCKIBEEN: I do. I think the fact that Australia has taken a good, bold step on carbon pricing, is something that the rest of the world has noticed. And in the months since, both the Koreans and the Chinese have emulated this and set up carbon pricing schemes of their own.
So it’s been important, and it would be a clear step backwards, a step out of the 21st Century and back into the 20th, or maybe even the 19th, to just say we can keep pouring carbon into the atmosphere for free…
http://www.abc.net.au/pm/content/2013/s3773610.htm
——————————————————————————–
Second draft is up. Lucia L showed that there is a validity test (which Cook et al. failed spectacularly) and I got to the bottom of the difference between Web of Science and Scopus.
I expect someone has already posted this – sorry I don’t have time to check but I’m putting it up in case they didn’t. I despair of our idiot politicians, who do no scientific research before inflicting this madness on us all
http://www.telegraph.co.uk/earth/energy/10095188/Ed-Davey-attacks-papers-who-report-destructive-climate-sceptics.html
Michael Crichton on the Eugenics movement:
…Its supporters included Theodore Roosevelt, Woodrow Wilson, and Winston Churchill. It was approved by Supreme Court justices Oliver Wendell Holmes and Louis Brandeis, who ruled in its favor. The famous names who supported it included Alexander Graham Bell, inventor of the telephone; activist Margaret Sanger; botanist Luther Burbank; Leland Stanford, founder of Stanford University; the novelist H. G. Wells; the playwright George Bernard Shaw; and hundreds of others. Nobel Prize winners gave support. Research was backed by the Carnegie and Rockefeller Foundations. The Cold Springs Harbor Institute was built to carry out this research, but important work was also done at Harvard, Yale, Princeton, Stanford and Johns Hopkins. Legislation to address the crisis was passed in states from New York to California.
These efforts had the support of the National Academy of Sciences, the American Medical Association, and the National Research Council. It was said that if Jesus were alive, he would have supported this effort.
. . .
As Margaret Sanger said, “Fostering the good-for-nothing at the expense of the good is an extreme cruelty … there is not greater curse to posterity than that of bequeathing them an increasing population of imbeciles.” She spoke of the burden of caring for “this dead weight of human waste.”
Such views were widely shared. H.G. Wells spoke against “ill-trained swarms of inferior citizens.” Theodore Roosevelt said that “Society has no business to permit degenerates to reproduce their kind.” Luther Burbank” “Stop permitting criminals and weaklings to reproduce.” George Bernard Shaw said that only eugenics could save mankind.
. . .
Eugenics research was funded by the Carnegie Foundation, and later by the Rockefeller Foundation. The latter was so enthusiastic that even after the center of the eugenics effort moved to Germany, and involved the gassing of individuals from mental institutions, the Rockefeller Foundation continued to finance German researchers at a very high level. (The foundation was quiet about it, but they were still funding research in 1939, only months before the onset of World War II.)
Since the 1920s, American eugenicists had been jealous because the Germans had taken leadership of the movement away from them. The Germans were admirably progressive. They set up ordinary-looking houses where “mental defectives” were brought and interviewed one at a time, before being led into a back room, which was, in fact, a gas chamber. There, they were gassed with carbon monoxide, and their bodies disposed of in a crematorium located on the property.
. . .
After World War II, nobody was a eugenicist, and nobody had ever been a eugenicist. Biographers of the celebrated and the powerful did not dwell on the attractions of this philosophy to their subjects, and sometimes did not mention it at all. Eugenics ceased to be a subject for college classrooms, although some argue that its ideas continue to have currency in disguised form.
The 97% argument in action used by UK Energy Secretary
“‘Blinkered’ climate change deniers accused of ‘dangerous, publicity-seeking, bloody-mindedness’ by Energy Secretary Ed Davey
Lib Dem minister launches extraordinary attack on his opponents
Claims climate change deniers are ‘absolutely wrong and dangerous’
He faces pressure for watering down emissions targets in Energy Bill”
……..
He will highlight a recent survey of over 12,000 peer-reviewed climate science papers which he says ‘provides a startling picture of the consensus that exists in our scientific community’.
Target: Tory MP Tim Yeo MP, chairman of the Commons Environmental Audit Committee, wants to write tougher targets for 2030 into the Energy Bill
Target: Tory MP Tim Yeo MP, chairman of the Commons Environmental Audit Committee, wants to write tougher targets for 2030 into the Energy Bill
It found 97 per cent of the climate experts who expressed an opinion ‘agree that human activity is driving global warming… just three per cent question man’s contribution’.
The extraordinary attack on his opponents comes ahead of the Energy Bill facing a crucial vote in the Commons.
http://www.dailymail.co.uk/news/article-2335100/Blinkered-climate-change-deniers-accused-dangerous-publicity-seeking-bloody-mindedness-Energy-Secretary-Ed-Davey.html
(My bolding)
The interesting part is going to the comments and selecting ‘best rating’ the Energy Secretary does not appear to be getting a lot of support.
It is well known in the marketing world that an uneven number contains more intrinsic authority and is more beleivable than an even number. So 97% is more beleivable than 98%.
What’s disappointing are skeptics who cling to the naive belief (or public persona on the matter, no matter how inwardly false this actually is) the AGW debate is essentially “science”. That many are unwilling to identify with the “right-wing” in their skepticism clearly is an enabler of AGW fanaticism.
Aside from the fact “97%” can’t quantify any threat of AGW there is the essentially ignored fact that the core AGW science community are largely like minded, academic left (of the Hansen/Mann variety) and there isn’t an objective authority (for example the left-wing MSM) who are willing to discipline the bias culture represented by Cook.
The “consensus” should be called out on its politics and technical skeptics are essentially weak and in denial for not doing so (many other complex reasons exist for this skeptic reality as well).
Pointman, you said this well.
I applaud Dr. Tol for doing this. I for one am sick of pseudo scientific PR efforts made by the likes of Cook and intended to cause stampedes. Climate scientists need to understand that the late great Dr. Steven Schneider hadn’t just possibly lost his moral compass, he was flat out wrong. If you want to motivate me, you’d darn well better quit worrying about being ‘effective’ and tell me the truth, the whole truth, and nothing but the truth, no matter how far over my head you think that truth is. I’m tired of people trying to manipulate me.
Why not harness the might of WUWT and go further than Cook? I’m sure we could drum up maybe 100 times the number of reviewers. Expand out to the “non Global” dataset. Take just a reasonable bundle of abstracts each. Establish clear and unambiguous rules. Yes, I know it’s probably too late (MSM already wringing the most out of Cook), but if done right, could perhaps right some of the wrongs…
rustneversleeps says:
June 2, 2013 at 8:37 am
“Men, it has been well said, think in herds; it will be seen that they go mad in herds, while they only recover their senses slowly, and one by one.”
That is from Charles Mackay in his book, Extraordinary Popular Delusions and the Madness of Crowds first published in 1841.
Hmmm,
For what it’s worth, it also appears to be from Sting’s song “All This Time”:
Sting was born the latter half of the 20th C, more than 100 years after McCay published Popular Delusions. Anthony’s alllusion is explicitly to McCay. The “97%” has been shot down repeatedly, because among other things, scientists never benefit from “consensus.” If they did, research grants would dry up. “Consensus” would equal “proven,” and in the case of AGW, its clear that short of a human catastrophe, if CAGW is “proven” the only way to prevent it would be to hold World War III. The real consensus lies among non-scientifically trained individuals in the media and easily frightened – I have to include a great many who purport to be social scientists – neurotics who are likely also certain that some specific food or other is the real evil in human health. Any opinion that is so strongly held that it is impervious to empirical reality can be classed as a delusion, so there are undoubtedly many delusionary on both sides of the debate’s fence.
Dana N points out that the main difference between draft 1 and 2 is that “it now references discussions stolen from the hacked SkS private forum”.
Good research can stand the light of day.
@richard Tol. “Good research can stand the light of day.”
If SkS would simply release the data gathering methodology you are requesting, the need for using the forum references ends, does it not?
Poems of Our Climate says (June 2, 2013 at 8:41 pm): “That nonsense talk Gary.”
Eh? You don’t think all climate scientists agree on, for example, conservation of energy?
I can think of a few things all climate scientists don’t agree on, if that will help: so-called “climate sensitivity”, the probability that most measured warming is anthropogenic, the validity of “adjustments” to temperature records, the “robustness” of the so-called “hockey stick”, etc.
Better?
@richard Tol
Comment1: Assuming that the reported increase in Neutral (very high) and Endorsing (high) papers over the years is correct as opposed to Rejecting papers (not much change), is the increasing negative skewness you report not to be expected since the data file is ordered by year of publication?
Comment2: “that the sample of papers that were rated twice is not representative for the large sample that was rated once”. Note in this respect that the response percentage of authors is greater in later years. If the character of papers changes over the years, then unrepresentativeness results.
Carry on!
@Anthony
We would learn a lot if only they released all the data.
The SkS forum discussions, however, reveal that there were complaints about fatigue, that the supposedly independent raters discussed their work with one another, that authors were raters, and that John Cook both ran the survey and participated in it. The last three points violate standard protocols for surveys.
Why do you keep knocking the idea of a 97% consensus among climate scientists. Since AGW is the current belief in the profession and to pass the examinations to become a climate scientist one has to accept this as right surely the only question is why the consensus is not 100%
@RichardTol, I’m a little concerned about your windowed analysis. You state that the papers were rated in a random order, how can structure pulled from a windowed analysis be attributed to the rating process rather than the (undefined) sorting from the Web of Science result set you’re working from. I don’t see how the two clusters of AGW rejections you found arose from fatigue or bias during randomly ordered rankings, or that the “unexpected” heteroskedasticity somehow reflects on the raters. For instance, sorting by year would explain skewness result (early papers showed less “consensus”) and possibly the rejection clusters (were they around the IPCC publishing deadlines?)
Richard Tol on p.4 of his second draft points to what I think has been entirely missing from the extremely misleading discussions from Oreskes to Cook: the vast majority of “climate change” related papers take no position and offer no evidence on the genuine ***CAUSAL*** hypothesis of AGW. For the Cook data, Tol reports that corrected analysis shows that ***95%*** of the papers take no position on the AGW hypothesis itself.
Hype artists from Oreskes to Cook/Nuccitelli are manufacturing a level of “consensus” from the fact that funding and political pressures lead many people to pay lip service to “climate change” and then discuss mitigation or adaptation issues. It is highly misleading to pretend that all the papers which assume AGW are adding anything to the evidentiary, scientific basis. (As a “lukewarmer” I don’t worry too much about whether there might be *some* AGW, but the magnitude and effects may well be exaggerated, misunderstood, and/or hyped for socio-political purposes)
Gary Pearse says:
June 2, 2013 at 7:59 am
How’s this for a measure of the bias. 12 were given 50 abstracts each and 12 were given 1922 abstracts each to evaluate. The point that anyone who happily accepts 1922 tasks has to be biased zealot is supported by the arithmetic: (1922-50)/1922 = 0.974, or 97%!!! The zealots were given the abstracts that could possibly be interpreted as pro AGW and instructed to do so, and the others were given abstracts that didn’t mention “global warming” per se.
The numbers seem to have multiple problems in the original Cook paper. I can’t seem to find 110 ratings assuming each abstract really was rated twice as the authors explain in the “not” Methodology.* The author say that a final sample of 11,944 abstracts were rated, and that each was categorized twice, once by each by two different raters, but the reported rating numbers indicate that a total 11,834 ratings were made (or some abstracts were really only rated once).
*Methodology is not merely a catalog of “we did’s,” Those are just methods. A methodology should explain the reasons that specific methods were employed. It should also justify them in the light of expected biases and data collection and quality problems, and place them within a specific scientific perspective. None of that information is there except implicitly, once. They did reportedly screen author names, which might have been to reduce assumptions about “big names,” but that is not stated. Neither do the authors explain why they are doing “crowd source” science, what they understand it be, or what strengths it might have over “standard” science. As science goes, a tossed salad has more methodology.
97% is the 21st century religious icon of the Church of Global Warming:
Throughout history, various religious cultures have been inspired or supplemented by concrete images, whether in two dimensions or three. The degree to which images are used or permitted, and their functions — whether they are for instruction or inspiration, treated as sacred objects of veneration or worship, or simply applied as ornament — depend upon the tenets of a given religion in a given place and time.
http://en.wikipedia.org/wiki/Icon
Richard Tol says:
“The last three points violate standard protocols for surveys.”
That is true. But this ‘survey’ was produced for propaganda, not as science, or even as a legitimate statistical study. The usual gang will pick it up and run with it, hoping the public will just read the headlines.
But as science/statistics, it is typical John Cook crap.
As far as I can tell, Cook, et al., 2013 does not provide information about the training of the raters. In any study of this sort, a rubric or protocol for rating should be provided and the people doing the rating tested for consistency. The Zooniverse crowd-sourcing projects (https://www.zooniverse.org/), for example, provide a bit of training for each of their numerous and very successful projects. This is another failure of the Cook project that Dr. Tol might add to his other criticisms.