What else did the '97% of scientists' say?

Guest Post by Barry Woods

I wonder just how many politicians, environmentalists or scientists who use the phrase ‘97% of scientists’ (or those who more carefully use ‘active climate scientists’) to give weight to their arguments regarding climate change to the public, have any idea of the actual source of this soundbite.  

Perhaps a few may say the ‘Doran Survey’, which is the one of the most common references for this ‘97% of active climate scientists’ phrase. In fact, the Doran EoS paper merely cites a MSc thesis for the actual source of this 97% figure and the actual survey.

“This was a very simplistic and biased questionnaire.”

(‘Doran Survey’ participant)

In a world where politicians (UK) went to war in Iraq based on a ‘sexed’ up dodgy dossier plagiarised from a 12 year old PhD thesis. I wonder how confident they would be lecturing the public about the need for radical decarbonising economic climate polices, if they were aware that the ‘97% of active climate scientists’ quote/soundbite actually comes from a students MSc thesis, that the Doran EoS paper cites?

Here are but just a few of many responses from scientists that actually took part in the survey, taken from the appendi of the MSc thesis:

“..scientific issues cannot be decided by a vote of scientists. A consensus is not, at any given time, a good predictor of where the truth actually resides..”

“..The “hockey stick” graph that the IPCC so touted has, it is my understanding, been debunked as junk science..”

“..I’m not sure what you are trying to prove, but you will undoubtably be able to prove your pre-existing opinion with this survey! I’m sorry I even started it!..” (Doran/Zimmerman feedback)

I wonder just how many politicians or environmentalists (or scientists) that have used the phrase ‘97% of climate scientists, have actually read the original source of the cited survey.

The Doran paper  has been criticised by many sceptics in the past, where a survey of 10,256 with 3146 respondents was whittled down to 75 out of 77 “expert” ‘active climate researchers’ (ACR) to give the 97% figure, based on just two very simplistic (shallow) questions that even the majority of sceptics might agree with. Lawrence Soloman made one of many critiques of the Doran Paper here and offers a very good summary, some other reviews here, here and here

A closer look at ‘The Consensus on the Consensus’

Yet, I’m not aware of anyone having a detailed look at the actual reference for the ‘97%’ quotation cited in the Doran EoS paper –  (link and press release), this was a students MSc thesis entitled “The Consensus on the Consensus” – M Zimmermann  (download here for £1.25 / ~$2), who was Peter Doran’s graduate student (and the EoS paper’s co-author)

“..and I do not think that a consensus has anything to do with whether a hypothesis is correct. Check out the history of science…you will find that scientific discovery is generally made by ignoring the ‘consensus..’” (Doran/Zimmerman feedback)

As this MSc thesis was the original source of the oft cited Doran paper  97%  quote, I tracked it down (sometime ago now) and discovered in the appendi that there was a great deal of  email feedback and answers to write in questions from the scientists that actually participated in the survey, much of it critical and sceptical of the survey itself, the methodology and the questions asked. Additionally, amongst those environmental scientists that responded, were some very sceptical sounding scientists with respect to man made climate change being the dominant driver of climate change.

“..Science is based on scepticism and experimental proof. Whereas human GHG emissions certainly have a warming effect, the breakdown between natural and anthropogenic contributions to warming is poorly constrained.

Remember that the warming since 1650 AD (not 1900) is part of a real ‘millennial cycle’ whose amplitude cannot yet be explained by any quantitative theory.

Also, the computer climate models are both too complex to be readily understood and too simple to describe reality.

Believing their results is an act of faith…”

(Doran/Zimmerman survey participant – App F)

There are also a number of additional problems I think, with the methodology that comes to light, that the previous critiques of the Doran paper are not aware of and some other interesting facts.

97% of the world’s scientists?

One fact that is not obvious (ie missing) from the Doran EoS paper and that surprised me, is that over 96% of the scientist that responded were from North America (90% USA, 6.2% Canada), with 9% from California alone.

90% (2833) of respondents were from the United States, while the remaining 10% (313) came from 22 other countries (Figure 1). Respondents from Canada accounted for 62% of the international responses. (Zimmerman)

What is the opinion of the worlds scientists?

Are the public aware when they are lectured that ‘97% of scientists’ agree based on the Doran paper, by their media, lobbyists, activist scientists and their politicians justifying climate action, that the UK, Germany, Spain, France, Australia, New Zealand respondents made up less than 3% of the survey in total. China had 3 scientists respond (three not 3%), Russian and India zero.

Perhaps if I was a western politician trying to persuade the public West to decarbonise and to extend or go beyond the Kyoto agreement I might think carefully about telling the public about the 97% of ALL scientists agree, when pushing for radical climate policies? As those countries outside of Kyoto agreement (China, India, Russia, etc) made it very clear at Copenhagen that  reduction in their own emissions is just not going to happen and at the recent Rio 20 plus conference I’m not even really aware that ‘climate change’ was mentioned that much at all.

What might I ask are those countries scientists telling their leaders about ‘climate change’ that may appear to many of them as a peculary western obsession (not many environmental lobby groups in China in the last 30 years). Perhaps those countries scientists are just not that concerned about a catastrophic interpretation of climate change,

I’ll just provide a ‘small’ anecdote to back up that hypothesis, just for fun, from China’s lead climate negotiator at Copenhagen (and Durban) no less.

Telegraph

“..China’s most senior climate change official surprised a summit in India when he questioned whether global warming is caused by carbon gas emissions and said Beijing is keeping an “open mind”

Xie Zhenhua was speaking at a summit between the developing world’s most powerful countries, India, Brazil, South Africa and China, which is now the largest emitter of carbon dioxide, the gas believed to be responsible for climate change.

But Mr Xie, China’s vice-chairman of national development and reforms commission, later said although mainstream scientific opinion blames emissions from industrial development for climate change, China is not convinced.

“There are disputes in the scientific community. We have to have an open attitude to the scientific research. There’s an alternative view that climate change is caused by cyclical trends in nature itself. We have to keep an open attitude,” he said…” (Telegraph)

Guardian

“..China’s most senior negotiator on climate change says more research needed to establish whether warming is man-made

China’s most senior negotiator on climate change said today he was keeping an open mind on whether global warming was man-made or the result of natural cycles.  Xie Zhenhua said there was no doubt that warming was taking place, but more and better scientific research was needed to establish the causes.

Xie’s comments caused consternation at the end of the post-meeting press conference, with his host, the Indian environment minister, Jairam Ramesh, attempting to play down any suggestions of dissent over the science of climate change…”(Guardian)

This only made the few column inches on the inside pages of the Guardian and the Telegraph, (by their Indian correspondents) perhaps an inadvertent unguarded comment by a senior diplomat let slip at a non-western conference expressing China’s real thinking perhaps?

Perhaps, unsurprisingly none of these newspapers UK environment journalists picked up on this ‘revelation’ on Chinese thinking, I wonder why, after all Xie was only China’s  lead negotiator (he was also at Durban). For further thoughts on this topic, Jo Nova has a very interesting article on Chinese, Russian and Indian thinking on climate change. (here)

But perhaps we should get back on to the topic of ‘The Consensus of the Consensus’

The ‘expertise’ of the 97%

On occasion when challenged about the 97% figure depending on 75 scientists from a survey of 10,000, it is usually met with a response that these were the experts in the field  of climate science and this is what maters not the number that took part. A closer look at the methodology perhaps raises some concerns about the ‘expertise’ and selection bias as this as his result depends on 2 additional questions in the survey that were used to identify expertise in climate research (not an unreasonable goal) within the respondents

Q5 Which percentage of your papers published in peer reviewed journals in the last 5 years have been on the subject of climate change?   

A:  1) less than 50% 2) 50% or more 3) not applicable

Q9 Which category best describes your area of expertise?

1) Hydrology/Hydrogeology  2) Geochemistry 3) Geophysics

4) Paleontology 5) Economic Geology (coal/metals/oil and gas)

 6) Soil Science 7) Oceanography/MarineGeology

8) Environmental Geology 9) Geology/Planetary Science

10) Climate Science 11) Geomorphology  12)General Geology

 13) Structure/Tectonics* 14) Petrology*

15) Sedimentology/Stratigraphy 16 Atmospheric Science*

17) Quaternary Geology*  18) Meterology*

19) Geography/Archeaology/GI 20 Engineering (Envr/Geo/Chem)*

21 Ecology/Biogeochemistry* 22) Glacial Geology*

23) Mineralogy*  24) Volcanology*  25) Other (*write in description)

(Zimmerman)

The survey used the answer to Q5 narrow down the expertise of the respondents, not unreasonably perhaps, and defined these as ‘active climate researchers’ (ACR), there was also criticism of the framing of this question in the feedback. This subset of respondents were then contacted to check the these claims and once verified, there were 244 respondents that met this criteria.   This categorisation gave positive responses to Q1 – 95% and Q2 – 92%

The survey used the answer to Q9 to define those as identifying as in the category of climate science as having more expertise than the other listed categories. Question 9 resulted in 144 respondents self identifying in the category of climate science. This categorisation gave positive responses to Q1 – 95% and Q2 – 88.6%

Finally  a category of experts was defined as those that responded as publishing more than 50% of papers AND self identifying in the survey as climate scientists, resulting in a group of 77

This categorisation gave positive responses to Q1 – 96.2% and Q2 – 97.4%

So is Zimmermann defining expertise or introducing a selection bias here ? It has not gone unnoticed that perhaps those scientists that self identify as climate scientists, are perhaps those that are more activist minded for a consensus.

It is quite possible for example, in this survey for scientist or even colleagues with identical qualifications, to self identify differently. Thus in this survey respondents could even be co-authors of a paper, but this survey would categorise one as more expert than the other.  Who knows if this happened or not, the fact that it is possible demonstrates the flaws in the thinking.

Additionally those that are in the 97% group are deemed to be more expert in climate science, keeping more abreast of the ‘whole’ field than the others.

“..The participants in this group are actively publishing climate scientists, and those most likely to be familiar with the theory and mechanisms of climate change, as well as have a thorough understanding of the current research and be actively contributing to the field..” (Zimmermann feedback)

This I think is a huge assumption, ‘climate science’ is a huge multidisciplinary field.

Is a geologist that identifies as a ‘climate scientist’ any more an expert on astrophysics, atmospheric physics, statistics, etc than those classified as have less expertise in the categories identified above.

Additionally the responses may merely capture (only the last 5 years publishing Q5) those junior more activist post docs, etc that self identify as climate scientist, where perhaps the older more published ‘expert’ colleagues describe themselves by the qualifications, not as climate scientists.  And of course, by the very nature of the survey, (which was commented on in the feedback) surveys of this type are potentially self selecting by the probability that those that are most concerned are more willing to take part.

Finding a consensus

In the introduction of the Zimmerman thesis, it describes criticisms of many other papers that have attempted in the past to establish what is the ‘consensus’ amongst scientists on climate change and the survey’s purpose was to address these criticisms.  However the introduction raised concerns for me that the author is not perhaps without there own biases (subconscious or otherwise). Perhaps judge this for yourself (here)

“..I did complete your survey. However, no matter how important, no matter how apparently obvious the combination of facts and theory, scientific issues cannot be decided by a vote of scientists. A consensus is not, at any given time, a good predictor of where the truth actually resides..” (Zimmerman feedback)

“..Science is not based on votes or consensus. Irrelevant question. Besides, which scientists do you regard as relevant?..” (Zimmerman feedback)

“..Science is based on scepticism and experimental proof. Whereas human GHG emissions certainly have a warming effect, the breakdown between natural and anthropogenic contributions to warming is poorly constrained..” (Zimmerman feedback)

Why does this matter, don’t other survey give similar results?

In the introduction, the Zimmerman thesis describes the earlier papers attempting to establish what the consensus is in the field of climate science and the thesis describes the criticisms made of these papers. And that the Zimmermann thesis survey is intended to meet some of these criticisms.

All too often in an article or presentation the phrase/soundbite ‘97% of scientists say’ is used to justify or imply certain climate policies, or that there is a consensus amongst climate scientists that policy action must be taken, or agreement of dangerous climate change, or any other thing that need the weight of authority this statement gives to an argument.

The later ‘Anderegg survey’ is perhaps the next most often cited survey, often alongside the ‘Doran Survey’, as producing a 97% figure for a consensus of climate scientists. Anderegg has also receive criticism as it seemed to be little more than a black/white document count of papers giving a percentage of numbers on each side. This of course gives no consensus on any of the above issues either. But again is often used to give the weight of authority to an argument.

An example perhaps, of this ‘use’ was by Scott Denning recently at the Yale climate forum, with a very critical response from Paul Matthews (Reader of Mathematics, Nottingham University

Scott Denning: “Let’s be clear: there is in fact an overwhelming scientific consensus on climate change. No peer-reviewed science disputes the expectation that rising CO₂ levels will cause major climate change in the coming decades.

Survey data have shown more than 97 percent agreement among professional climate scientists (Anderegg et al, 2010, PNAS), and every major professional society has issued supporting statements. (Yale – here)

I raised my own concerns about the nature of the Anderegg survey (here and here), but I think Professor Paul Matthews is more to the point and eloquent than I was.

Paul Matthews: “Scott Denning needs to be more careful if he and his fellow climate scientists are to be taken seriously by scientists from other fields such as myself. 

He loses credibility by referring to the ridiculous Anderegg et al study, in which the authors put scientists into two different pigeon-holes. 

Worse still, he misrepresents the claims of that paper (he implies the 97% believe CO2 will cause major climate change in the coming decades, while Anderegg et al say 97% agree that most of the warming of the 20th C was very likely due to man-made greenhouse gases – two very different statements). (Yale – here)

At the time, Joseph Romm at Think Progress gave his own interpretation of what the Anderegg survey showed us.

“..The issue is whether folks are actively spreading disinformation, especially disinformation that has been long debunked in the scientific literature.  As I’ve said for many years now, it is time for the media to stop listening to, quoting, and enabling those who spread anti-science and anti-scientist disinformation. (Think Progress)

It is interesting to compare the Think Progress response to the Anderegg survey to that of scientists. Dr Roger Pielke junior was very critical of the Anderegg survey (link) referring to it as a blacklist, this brought about I think a very appropriate response from Real Climate’s Dr Eric Steig (quite a contrast to Climate Progress – Joe Romm)

“Wow. Roger, you know I disagree with you on many things, but not on this. What the heck where they thinking? Even if the analysis had some validity — and from a first glance, I’m definitely not convinced it does — it’s not helpful, to put it mildly. I’m totally appalled.” (Dr Eric Steig)

Keith Kloor also has a very good article with various responses to the PNAS Anderegg survey and the comments / discussion also make very interesting reading (Collide a Scape – The Climate Experts)

Concerns about ‘consensus surveys’

I am concerned that the conclusions made by Doran EoS paper and the Zimmerman MSc thesis seems to go beyond the results warranted by the survey and motivated by activism more than science.

“..the debate on the authenticity of global warming and the role played by human activity is largely nonexistent among those who understand the nuances and scientific basis of long-term climate processes.”

The challenge now, they write, is how to effectively communicate this to policy makers and to a public that continues to mistakenly perceive debate among scientists..” (Doran press release)

But, I would like to put aside any criticism of the methodology or conclusions the scientists behind the Doran, Anderegg or any other similar paper make and reserve my strongest criticism to others that misrepresent them, or go much further than the conclusions. My strongest criticism is not for those politicians, environmentalists, journalists or scientists, that use the soundbite of ‘97% of scientists’ in complete ignorance of its source, or do not check the citation for themselves in Zimmermann.

No, I reserve my strongest criticism for those activist scientist that know full well the source of the ‘97% of scientists’ soundbite and use it anyway, usually very carefully worded along the lines of 97% actively researching in their field, and then use it to imply that there is some consensus of future dangerous or catastrophic risk, or that certain policies that must be taken, because of this consensus.

In my mind this is misusing the authority and goodwill most of the public still hold for scientists, when attempts are made to justify claims of policy action with a soundbite, or to try to silence any dissenting voice as a denier or holding extreme questionable views (implying others not mainstream respectable scientists) It also raises the very real concern that other  activists response to sceptics will assume motives of malign intent (greedy fossil fuel deniars, with the same morals of holocaust deniers, for example) if they seeing leading scientist making these strong claims.

As in the activists worldview, surely only those with questionable malign and/or greedy motives would disagree with  ’97 of scientist agree’ that future climate change is a catastrophic danger.

An example being this extreme reaction by Steve Zwick at Forbes.

“..We know who the active denialists are – not the people who buy the lies, mind you, but the people who create the lies.  Let’s start keeping track of them now, and when the famines come, let’s make them pay.  Let’s let their houses burn until the innocent are rescued*.  Let’s swap their safe land for submerged islands.  Let’s force them to bear the cost of rising food prices…” (Steve Zwick – Forbes)

And he cites the authority of a consensus of scientists which support in his mind, this statement of certainty about future climate.

“..If the shirkers and deniers actually believe their propaganda, they’ll go along with this – because they only have to pay if they’re wrong and 98% of all climate scientists are right. (And what are the odds of that happening – nudge nudge, wink wink?)..” (Steve Zwick – Forbes)

Another example being when a number of climate scientists (community leaders) responded in a letter to the Wall Street Journal, to the 16 scientist that signed an opinion piece entitled – No Need To Panic About Global Warming – in the Wall Street journal.

The climate scientists response (extract)

“..Research shows that more than 97% of scientists actively publishing in the field agree that climate change is real and human caused.

It would be an act of recklessness for any political leader to disregard the weight of evidence and ignore the enormous risks that climate change clearly poses.” (Trenberth et al – WSJ)

The authors of the original Wall Street Journal opinion piece duly responded making the same complaint about the misuse of the ‘97% of scientists’ phrase as mine:

“.. The Trenberth letter states: “Research shows that more than 97% of scientists actively publishing in the field agree that climate change is real and human caused.” However, the claim of 97% support is deceptive. The surveys contained trivial polling questions that even we would agree with. Thus, these surveys find that large majorities agree that temperatures have increased since 1800 and that human activities have some impact.

But what is being disputed is the size and nature of the human contribution to global warming. To claim, as the Trenberth letter apparently does, that disputing this constitutes “extreme views that are out of step with nearly every other climate expert” is peculiar indeed.” (Wall Steet Journal)

I did show a copy of ‘The Consensus on the Consensus’ to a well known writer on the environment,(over a very nice lunch at Brasenose College, Oxford University) who was very interested and whose first response was why are they all so sceptical! And to his credit admitted he was not aware of it, and had not  looked at the primary source and he even suggested to me:

‘If I were a sceptical journalist I would make hay with it!”

To be very fair to him, Zimmerman only came online in September 2011, I’m sure I went looking for it before that and could not find it anywhere. Additionally when faced with a paper with multiple citation who of us, actually goes and reads all those citations to see if the conclusions are correctly used in the paper?

All this said and done, I’m a sceptical blogger, writing for a major sceptical blog, please don’t take my word for anything, download it yourself and form your own views. (here) there are at least 80 pages of responses, my selections are but a fraction of the whole.

Some further examples  of feedback to the survey below:

Problems with questions 1 and 2 and the word ‘significant’

“Questions 1 asks if I think temperatures are warmer than the 1800s, but doesn’t indicate if I’m supposed to compare to today, the last 10 years, the last 50 years, or… Without telling me what I’m comparing to, I cannot answer the question.

Q2 then asks if I think that humans are “a significant” contributor to warming temperatures, but I can only answer yes or no. I happen to think that we are one among many contributing factors, so I answered yes, but I couldn’t explain this. The third question then asks me why I think humans are a major contributor, but is phrased in such a way that it’s implicit that I’m now listing them as THE significant factor. They are not the primary cause, but I had to stop the survey at this point because it was forcing me to answer queries about why I think they are.

As constructed, your responders will be unable to indicate that there are multiple causes to climate change, that climate change is the norm on Earth and has been going on throughout geologic time, and that there is strong evidence to indicate that climate change not only occurred before humans existed, but also was probably more extreme than the event we are living in today.”

And:

Your use of the word ‘significant‘. It seems clear that human activity has caused an increase in CO2 levels. That, in theory, might have caused an increase in global temperature. However, did it? If so, was it the only cause? If it was a cause, was it a significant cause?

And:

Not Fair: You changed the question from ‘significant’ to ‘contributing’ Significant= 25%. Contributing=75%

“What defines significant? If 1-2 degrees F is considered significant then I would agree that human input is significant

“what do you mean by significant? Statistically? A player in the total rise? sure we are! How much? I am not sure.

What is meant by significant? A major contribution, yes, but what is human activity compared with increased solar activity. So far, it is lost in the statistical models. While it certainly seems likely that human activity is at least partly responsible, I am not aware of data conclusively proving this. It has been documented that natural earth temperature cycles occur with, or without, human-based effects.

I entered an answer I did not intend. I think human activity is a significant component, but I do not know if it is 10%, 25%, 50% or more. (3c)

 “I appologize, but as an objective scientist I do not communicate “opinions” or “attitudes”. These do not belong on the scientific agenda and certainly not in the classroom. Thus I decline to contribute to your survey.” (Zimmerman feedback)

Appendix G – Emails received (lots of interesting responses)

I found the very first email response to be quite amusing (ref ‘team’)

“I am on the team. Your survey is most appropriate and I am honoured to have been asked to participate.” (Zimmermann -App G)

The third response provides a counter:

“I’d be happy to participate. This is a great idea. We were talking about this just yesterday and I’m guessing you’ll find less consensus that the media tend to suggest.”(Zimmerman – App G)

Appendix F – Write in questions for 3c (reasons sceptical)

 “I am not absolutely convinced, however, that carbon dioxide is the culprit. I think that remains to be proved. Carbon dioxide is complicated, and I believe that there could be other both human induced and natural causes for global warming.”

And:

“After thinking a while about the questions, I wish that I had not participated in the survey because of the way that the questions could be misconstrued.”

And:

“I study glaciers. Earth has had hundreds of continental scale glacier events during its history. Glaciers will continue to experience cycles where they expand and then contract, and then expand again, as they have done many times before, prior to humans evolving. They will also continue to do so long after our species is extinct.”

I’m glad I’m not a young scientist in the USA:

“I believe this global warming scare is a hoax designed to raise taxes and fill the pockets of the likes of Gore and those who do research in the topic, etc. I am not the only one who feels this way. One of our professors, XX, paleontologist, Antarctic specialist, agrees with me. He said he is treated like a pariah here at XX.”

I will finish on the following piece of feedback, as it  highlights and sums up a concern of mine, that all scientists might want to consider with respect to the public trust in science.

“As I indicated in my survey responses, every scientist I work with is convinced that human activity is a factor influencing global warming, but it is also well known that the causes extend beyond human activity to include astronomical cycles which we had no part in creating and which we are powerless to stop. I have not found anyone who could tell me what percentage of the warming we’ve seen so far is attributable to natural vs. human causes, however.

I feel that the scientific community has not been totally forthcoming in public statements about acknowledging the dual causes of global warming, and that someday people will realize that no matter what we do, we will never stop global warming entirely because a good fraction of the causes are natural and not anthropogenic.

I’m afraid that at that point people will feel misled by scientists and politicians who have implied, essentially, that “we caused it, by cleaning up our act we can stop it.”

I feel that this is a recipe for public disillusionment with the science community, and is a mistakeon our part. (Zimmerman feedback – App F)

It is my personal recommendation that if anyone should publically claim because ‘97% of scientists agree’ and are attempting to use this phrase as a soundbite to close down any criticism, going beyond the conclusions of these surveys.

My recommendation is to ask them politely if they are aware of the source of this phrase. And then quote to them an example of the feedback by scientists that took part in the survey itself, any then perhaps it will be possible to have a debate about any issue or claim being made.

References/Links:

MSc Thesis – The Consensus on the Consensus – M Zimmerman (download Cost £1.25)

Eos Abstract – EoS Examining the Scientific Consensus on Climate Change Citation: Doran, P. T. and M. K. Zimmerman  (2009), Examining the Scientific Consensus on Climate Change,  Eos Trans. AGU,  90(3), 22, doi:10.1029/2009EO030002.

EoS Paper –  Examining the Scientific Consensus on Climate Change -Doran/Kendall Zimmerman

UIC Press Release – Survey: Scientists Agree Human-Induced Global Warming is Real

Related articles

• About that overwhelming 98% number of scientists consensus (wattsupwiththat.com)
• By its actions, the IPCC Admits Its Past Reports Were Unreliable (wattsupwiththat.com)
• Climate Science – The Abuse Of Science For A Global Political Agenda (toryaardvark.com)