An oopsie in the Doran/Zimmerman 97% consensus claim

David Burton writes:

I just realized the obvious answer to a question that has been nagging in the back of my mind for nearly a year and a half.

In 2008 Margaret Zimmerman asked two questions of 10,257 Earth Scientists at academic and government institutions. 3146 of them responded. That survey was the original basis for the famous “97% consensus” claim.

For the calculation of the degree of consensus among experts in the Doran/Zimmerman article, all but 79 of the respondents were excluded. They wrote:

 
“In our survey, the most specialized and knowledgeable respondents (with regard to climate change) are those who listed climate science as their area of expertise and who also have published more than 50% of their recent peer-reviewed papers on the subject of climate change (79 individuals in total). Of these specialists, 96.2% (76 of 79) answered “risen” to question 1 and 97.4% (75 of 77) answered yes to question 2.”

The basis for the “97% consensus” claim is this excerpt:

[of] “the most specialized and knowledgeable respondents (with regard to climate change)… 97.4% (75 of 77) answered yes to question 2.” 

But that is a false statement.

The two questions were:

Q1: “When compared with pre-1800s levels, do you think that mean global temperatures have generally risen, fallen, or remained relatively constant?”   76 of 79 (96.2%) answered “risen.”

Q2: “Do you think human activity is a significant contributing factor in changing mean global temperatures?”   75 of 77 (97.4%) answered “yes.” 

My nagging question was, why did different numbers of people (79 vs. 77) answer the two questions? What happened to the other two respondents?

The answer to that question is not in the Doran article.

But it is in the Zimmerman report, a copy of which I bought back in March, 2012. The reason I feel stupid is that I read it and even quoted the relevant part way back then, and it still took me until now to realize the obvious answer to my nagging question.

This was the full set of questions that Zimmerman asked in their survey:

Q1. When compared with pre-1800's levels, do you think that mean global temperatures have generally risen, fallen, or remained relatively constant?
1. Risen
2. Fallen
3. Remained relatively constant
4. No opinion/Don't know
 
Q2. Do you think human activity is a significant contributing factor in changing mean global temperatures?  [This question wasn’t asked if they answered “remained relatively constant” to Q1]
1. Yes
2. No
3. I'm not sure
 
Q3. What do you consider to be the most compelling argument that supports your previous answer (or, for those who were unsure, why were they unsure)? [This question wasn’t asked if they answered “remained relatively constant” to Q1]
 
Q4. Please estimate the percentage of your fellow geoscientists who think human activity is a contributing factor to global climate change.
 
Q5. Which percentage of your papers published in peer-reviewed journals in the last 5 years have been on the subject of climate change?
 
Q6. Age
 
Q7. Gender
 
Q8. What is the highest level of education you have attained?
 
Q9. Which category best describes your area of expertise?

Do you see it?  If a respondent answered “remained relatively constant” to the first question, then he wasn’t asked the second question!

That’s obviously why only 77 answers were reported to the second question. Two of their 79 top climate specialists had answered “remained relatively constant” to the first question, and those two were not asked the second question, and were not included in the calculation of the supposed 97.4% agreement.

That means only 75 of 79 (94.9%) of their “most specialized and knowledgeable respondents” actually gave them the answers they wanted to both of their questions.

So, despite asking “dumb questions” that even most skeptics would answer “correctly,” and despite excluding over 97% of the responses after they were received, they still did not find 97% agreement. They actually found only 94.9% agreement.

I’ve updated my  http://tinyurl.com/Clim97pct  page to reflect that fact.

I’ve also emailed the editor of Eos, which published their article back in 2009, asking that they run a correction.

Advertisements

  Subscribe  
newest oldest most voted
Notify of

“Humans have had a “significant impact” on warming is pretty vague.
Also, why does anyone care what a survey conducted in 2008 show?
A lot of water under the bridge since then.

I personally made the Sun rise this morning. And this had never happened before.

charles stegiel

Lies, damn lies and statistics.

Severian

Given the results of the Surface Station project, and the frequent and ill-advised manipulation of historical and current temperature data, I’d say that warming definitely has a strong human component, just not in the way they think.

How they arrived at it is merely an exercise in Marketing. That it is a worthless number that has no meaning is what is scientifically valid.

Is the underlying data of ALL responses actually available? So, of the 3000+ responses, how did the data break down?

Dodgy Geezer

@Harold Ambler says:
…I personally made the Sun rise this morning. And this had never happened before…
Quite right. Yesterday, it was me…

EthicallyCivil

Also, the answer to the second question would include people that thought temperature was lower and that it was caused by humans. It doesn’t seem likely that any would answer this, but there’s nothing in the question or data *as* *reported* above.

“I’ve also emailed the editor of Eos, which published their article back in 2009, asking that they run a correction.”
Why waste time doing that?
To any audience says 95% of scientists …. blah, blah, is just the same as saying 97%.
You asking them to correct it , implies that you would consider it better if it showed the “correct” number of 95% , when in fact the whole thing is BS, game playing that should be retracted in its entirety.
The unpublished question number 0 was probably the most important one:
Q0 Would you like more money and public funds to be directed to you area of research in the future?
Those who replied “No” or “Don’t care” were obviously not to be considered experts in relevant fields of study. and were not asked the rest of the questions.

“I’ve also emailed the editor of Eos, which published their article back in 2009, asking that they run a correction.”
Why waste time doing that?
To any audience says 95% of scientists …. blah, blah, is just the same as saying 97%.

Quite a scientific study. Science by consensus, I’m impressed.

Richard

That really feels like a cherry picked bunch of yes men, with a couple of dissenters just for effect to make it feel like a proper survey.
What happened to the other two , what happened to the other three thousand plus.

Taphonomic

I’ve always been appalled that a Masters degree was awarded for this torturing of data.

steverichards1984

The 97% is still being quoted in the MSM, it would be good to get it changes, pointing out the mistake, now that the lame questions and methods are out in the open.
A more thorough revisit with publicity could be worthwhile.

DirkH

EthicallyCivil says:
December 10, 2013 at 8:14 am
“Also, the answer to the second question would include people that thought temperature was lower and that it was caused by humans. It doesn’t seem likely that any would answer this, but there’s nothing in the question or data *as* *reported* above.”
During the cooling 70ies and up to 1988 the scare scientists postulated that an increase of CO2 due to human activity was the cause of the cooling; arguing that CO2, being IR-active, was emitting more energy to space; leading to an “ice age”, as the media reported (the journalists meant to say “glaciation”, but were too stupid for that).
The scare scientists switched to warming over time, as a glaciation became impossible to market, arguing CO2 emitted more IR to the surface; the switch was complete in 1988, when Hansen performed his infamous A/C stunt. The scare scientists remained the same; each one making his choice between being efficient, and being honest, as Schneider demanded.

AnonyMoose

Why did they ask Q2 and Q3 if the answer to Q1 was: “4. No opinion/Don’t know”?

TomL

The first question was trivial. 1800 was in the midst of the Little Ice Age, so OF COURSE it’s warmer now.
The second question is also trivial if you keep in mind that, as used by the UN and the IPCC, “Climate Change” is BY DEFINITION anthropogenic. It’s hardly surprising that anybody who publishes papers on “Climate Change” believes it is anthropogenic, or they would have used different terminology.

Harold Ambler and Dodgy Geezer go on about how they made the sun rise.
Will you knuckleheads please knock it off? Do you have any idea how hard I work to make the Sun set?!

wayne

“Q2. Do you think human activity is a significant contributing factor in changing mean global temperatures?”
This says nothing of ‘warming’. Land use may have very well kept the temperatures more moderate duing the LIA recovery, keeping it slightly lower than it would have been keeping all else kept constant. This question is ambiguous and fails the survey. Also, few think humans have directly affected ocean temperatures so this would imply global land-only temperature records, again ambiguous.

GlynnMhor

It’s not just that being warmer than during the Little Ice Age is trivially true, but the use of the term ‘significant’ is ambiguous.
Most lay people use ‘significant’ to mean ‘major, important, big, predominant’ or other such, whereas those accustomed to dealing with measurement and the statistics thereof use ‘significant’ as meaning simply ‘detectable above the noise’.
So I, for example, would have to say yes to both questions, even though I’m not a believer in catastrophic AGW.

leon0112

I want to know how many of them predicted the coldest temperature ever measured.

Thanks for this post David B. I hadn’t seen the full questions yet, and neither was I aware of the exact phrasing om Q2:

“Do you think human activity is a significant contributing factor in changing mean global temperatures?”

The obvious follow-up question, in my opinion, would be:
What exaclty is meant by “a significant contributing factor”? And how would this be quantified? To me thos looks like a advertently vague ambiguous question, that could easily be misintepreted by a casual reader or a referring churnolist.

Resourceguy

Humm, 3,146 responses reduced to 77 in the final segment on human caused global warming. Evidently, this filtering process ended up in the innards of the Health Exchange website insurance application process generating the same 77 completed applications on the first day. You doubt this possibility? Prove it as wrong, go ahead.

SionedL

Why it is important to bring this up again is that not only MSM still quote it, but Obama has used the 97% argument again recently.

75 scientists say the science is settled!

T Montag

These survey’s all seem to be phrased to achieve a “desired” outcome. I would like to see a survey that asked how much anthropogenic forcings have contributed to global warming. Break it down in fifths (0-20%, 20-40%….) or even something less definitive, like (some, about half, most or nearly all). The recent Cook et al study was prime example of over-complicating the question and playing fast and lose with what defines “the consensus”.
Why not just ask those in the climate filed what they think?

Darn!
There goes that neat 97% propaganda number,now it is a boring 94.9% number.
Back to the drawing board again.

Edim

Good comments! lol

JimS

What shocks me is that of the thousands who completed the survey, only 79 were chosen in order to come up with the alleged “97% consensus”, thus 77 our of 79. Does the media know this? This was not a survey determining a consensus, when an oligarchy was formed first from which the 97% was then derived.

Jeff L

94.9% or 97%, either way it was still a ridiculous survey & manipulated to get the desired results. It was any thing but objective.

DC Cowboy

“Do you think human activity is a significant contributing factor in changing mean global temperatures? ”
I would bet that if you asked that question of known skeptical scientists, you’d get a high % of ‘yes’. “Human activity” covers a lot of ground and “significant” factor is very difficult to quantify, for some it could mean ‘more than 50%’, for others it could mean 10% if, in the opinion of the scientist, the other factors (solar, Milankovitch cycles, PDO/AMO oscillation, etc) contributed similar or smaller % to the total.
The question is so vague as to be meaningless

dp

I also think this revelation is too little, too late. The more important discovery is that 97% of scientists who’s livelihood (writings) depends on climate study grant money agree, and that their numbers (79) are a small part of the entire population of interested scientists who responded.

petermue

For those 77 I’d like to see the answers to
Q9. Which category best describes your area of expertise?
May be there were some more cartoonists or unqualified “scientists” like Cook amongst them.
I shouldn’t wonder.

The real question to ask – these are (partly climate) scientists after all:
If you had to bet on the actual value of the ‘climate sensitivity’ what you bet on? (ref: http://en.wikipedia.org/wiki/Climate_sensitivity)
Temperature increase per CO2 doubling:
a) Don’t know
b) It does not matter
c) 6
I wonder how the graph on the wikipedia page (models) would compare to a poll run by Anthony, vs a poll at from AGU members, vs a poll run by some IPCC hired gun.
To me this is THE question, as the entire spend of $20 trillion or whatever, plus all the fear is based not on the Earth’s temperature rise, but on its relation to CO2.

thisisnotgoodtogo

The most interesting question is why is there about 3 % in disagreement?
You wouldn’t get 3% of working scientists who say the earth is flat.
It ruins the comparison.
Even the Flat Earth Society president responded to Obama and furiously endorsed GLOBAL warming.
Who were those masked men/women?

Mark

I’d say yes man is responsible, a large part is the UHI…

Harold Ambler says December 10, 2013 at 8:03 am
I personally made the Sun rise this morning. And this had never happened before.

Fits in there well with ‘Dreams of my Father.’ (Does narcissism/do narcissists really know any bounds?)
http://survivingnarcissism.com/2011/02/04/narcissism-knows-no-bounds/
.

jim southlondon

Just a quirky
So exactly how much extra CO2 did Obama ,Cameron, Prince Charles and the rest of the world leaders generate all jetting into South Africa for Mandela,s funeral.

Jeff L

For those who think the 94.9 number should be put out for public consumption , I would personally be worried about a Striesand effect as 94.9 sounds pretty much like 97 to the general public – dont want to reopen that wound !

Stacey

I agree with an earlier poster whether its 94% or 97 % is irrelevant the question and and the observations to be made are as follows:-
1 Out of the 10,257 people asked to respond only 3146 took the survey ie 30%
2 Out of a sample of 3146 only 79 respondents were considered ie 2.5%
This survey is absolute garbage and the following statement is true almost 100% of alarmist scientists agree with themselves 🙂

Ken G

Of all the things quite obviously wrong or misleading with this survey, I think this 94.9% instead of 97% is by far the least of them. What do you expect the response to be? “Oh my, you’re right! 94.9% of all scientists agree!”…. and you’ve accomplished what exactly?

pappad

Anybody care to explain to me how CO2 can allow IR to REACH the surface but somehow “traps” it there and won’t allow it to reflect back into space??? Is it one-way reflective?

CodeTech

That stupid survey was identical to asking Holistic practitioners if Holistic medicine is better than “western” medicine. Once you weed out all of the non-believers, you will be left with a consensus.
4 out of 5 dentists who expressed an opinion recommend patients chew sugarless gum. The rest still need to pay for something expensive.
The only thing amazing about the “97%” thing is that enough people are stupid enough to fall for it. Really, the bottom line is, 97% of self-proclaimed “climate scientists” who have an interest in promoting their reason for existing agree that they should exist. The rest are still trying to determine what the meaning of “is” is.

rogerknights

Ken G says:
December 10, 2013 at 10:04 am
Of all the things quite obviously wrong or misleading with this survey, I think this 94.9% instead of 97% is by far the least of them. What do you expect the response to be? “Oh my, you’re right! 94.9% of all scientists agree!”…. and you’ve accomplished what exactly?

That warmists are sloppy and/or “stretchers.”

Louis LeBlanc

I haven’t read all the comments, but has anyone asked if the 79 “experts” were identified before or after the questionnaires were returned? I assume that Zimmerman had pre-qualified the 10,257 “scientists” etc. who were asked the questions, so why were the 7,111 who didn’t think climate change or global warming was worth a response excluded from the advertised data? Te sad part of all this is the duplicity of the “scientists” propagating this malarkey and the gullibility of the public to allow the grant money researchers and the liberal press to pull this scam off.

EthicallyCivil

Also, Q2 is misrepresented as “significant, in a way likely to have strong net negative effects on humanity” as opposed to “significant, meaning measurable and non-trivial”. Certainly when one includes elements like UHI in fraction of global temperature signal that includes urbanized and suburbanized land, and including all factors (per Pielke Sr. rainforest destruction, soot, et. al. it can reasonably be argued as measurable and non-trivial.as a whole.
Even 94.9% didn’t agree that burning fossil fuels will cause any number of catastrophes or tipping points.

DirkH says: at 8:28 am
The scare scientists remained the same; each one making his choice between being efficient, and being honest, as Schneider demanded.
Good points. Not to quibble, but Schneider, though, at least in the quote that I have, made the choice between being honest and being effective:
“We have to offer up scary scenarios… each of us has to decide the right balance between being effective [lying] and being honest [ineffective].” -Stephen Schneider, lead ipcc author, 1989
Other similar quotes:
“Unless we announce disasters no one will listen.” -Sir John Houghton, ex ipcc chair
“The only way to get our society to truly change is to frighten people with the possibility of a catastrophe.” -Daniel Botkin, ex Chair of Environmental Studies, UCSB
“Only sensational exaggeration makes the kind of story that will get politicians’ attention.” -Monika Kopacz, Atmospheric Scientist

Berényi Péter

Well, the logic behind the Doran/Zimmerman study is mind boggling. To see this, first of all let’s substitute climate science with another field of investigation, for logic is not supposed to be sensitive to semantic details, just truth values assigned to propositions. Therefore the subsample considered is defined this way:

In our survey, the most specialized and knowledgeable respondents (with regard to homeopathic remedies) are those who listed homeopathy as their area of expertise and who also have published more than 50% of their recent peer-reviewed papers on the subject of homeopathic remedies

Let the questions asked be as follows:
1. Does homeopathic remedies have any beneficial effect?
2. Does a solvent retain “memory” of the active substance even if the solution is diluted until not a single molecule of it is left there?
Q2 is only asked from those in the subsample described above who answered “yes” to Q1.
Now, I would bet a fortune the consensus calculated this way is pretty close to 100%. Does it imply anything about the memory of water?
What have we learnt here? If the merit of the basic paradigm of an entire field is in question, genuine experts of said field are the first to be excluded from the survey, that’s what.

Tom O

I agree, David, that it is important. What I would like to know, of course, is WHO were the 77, and how many of those “experts” were, in fact experts.
Why does it matter? I think that since it is only those in the :skeptical” community that recognize that the mythic 97% is just that, mythic, we still need to keep hammering the “truth” so it gets out to the growing community of “I’m not sure any longer” people. Keep up the good work.

captainfish

I’ve conducted a survey by mail before from a university setting. This whole Zimmerman survey is trash. First off, the total realm of N is 10,257. The number of respondents is 3146. The number who responded yes to question 2 is 77 our of 3146. Then, you have to compare that value to the Non-Respondents. You have to survey those who did not reply. Are they a different population? How do they differ to those who responded.
Sure, you can drill down in to the survey to cherry pick out a limited view of the data, like:
“..those who listed climate science as their area of expertise and who also have published more than 50% of their recent peer-reviewed papers on the subject of climate change “
Why send out surveys to 10,257 scientists if you are only going to take 79 of them? Don’t we have more than 79 declared “climate scientists”?
More importantly, how can “climate science” be a specialty when it is a general science? Geomorphology is a specialty. Space Weather is a specialty. Climate Science is a vast generalized field of hundreds of other sciences.