Where Consensus Fails – The Science Cannot Be Called 'Settled'

There appears to an inconistency in the respondents.
On the one hand we have, “86.5% agreed or strongly agreed that ‘climate change is occurring now’ and 66.5% agreed or strongly agreed that future climate ‘will be a result of anthropogenic causes.'”
On the other hand we have the response that indicate a strong lack of confidence in the key feedbacks that go to support the above statement: the cloud-water vapor feedbacks.
Without confidence in the weay the feedbacks are dealt with, how can there be confidence in the role of humans in future climaet change?

I liked this article much better than your last posting I read. I also really enjoyed you and Steven’s CRU Tape Letters book. Thanks for you time and effort.

Their insecurity doesn’t stop them from calling for a total clampdown on our energy infrastructure, though… A majority supports the IPCC’s “findings”, after all.

From page 9;
“The survey employed a non-probability convenience sample.”
http://en.wikipedia.org/wiki/Nonprobability_sampling
“Sampling is the use of a subset of the population to represent the whole population. Probability sampling, or random sampling, is a sampling technique in which the probability of getting any particular sample may be calculated. Nonprobability sampling does not meet this criterion and should be used with caution. Nonprobability sampling techniques cannot be used to infer from the sample to the general population. Any generalizations obtained from a nonprobability sample must be filtered through one’s knowledge of the topic being studied. Performing nonprobability sampling is considerably less expensive than doing probability sampling, but the results are of limited value.”
“Examples of nonprobability sampling include:
Convenience, Haphazard or Accidental sampling – members of the population are chosen based on their relative ease of access. To sample friends, co-workers, or shoppers at a single mall, are all examples of convenience sampling.
…”
http://en.wikipedia.org/wiki/Convenience_sampling
“Accidental sampling is a type of nonprobability sampling which involves the sample being drawn from that part of the population which is close to hand. That is, a sample population selected because it is readily available and convenient. The researcher using such a sample cannot scientifically make generalizations about the total population from this sample because it would not be representative enough. For example, if the interviewer was to conduct such a survey at a shopping center early in the morning on a given day, the people that he/she could interview would be limited to those given there at that given time, which would not represent the views of other members of society in such an area, if the survey was to be conducted at different times of day and several times per week. This type of sampling is most useful for pilot testing.”
Nuff said.

only 32% agreed or strongly agreed that the current state of scientific knowledge is developed well enough
===============
I’m really surprised at that. 1/3 of them think they are smarter than they really are….

This may be the worst summary of data I have ever seen. You might want to go back to the source data and recalculate some of your points above. Also, the method by which you discard respondents answering “4”, or “5” is a little strange.
Only 25% agree or strongly agree that “Data collection efforts are currently adequate,” while 16% disagree or strongly disagree.
As an example, the above was not the question asked. Secondly, 16% is not correct even by the incorrect way you are interpreting the data.
This whole analysis should be re-written and then double-checked for math and a little more time spent on what the different responses indicate in relation to the question asked.

After looking at the text on pages 9-11 it would appear that the use of the term “convenience sampling” is a misnomer or somewhat misleading as the selection criteria is a closer fit to Judgmental sampling or Purposive sampling;
“The researcher chooses the sample based on who they think would be appropriate for the study. This is used primarily when there is a limited number of people that have expertise in the area being researched.”
http://en.wikipedia.org/wiki/Nonprobability_sampling
However, this does not change the nature of the survey one iota, and still casts serious doubts on the statistical accuracy of said survey results in total.
Also, while you posted a link to the original survey results, it would appear that most of the text above was “cherry picked” and does not give an entirely objective overview of the ENTIRETY of the survey results over all subject catagories.
Everyone should download and read the PDF and judge for themselves firsthand PRIOR to reading the above post’s somewhat limited focus points.

The cry is about:
Yes, we have no bananas.
And silence deafens.
===========

Why should we be pouring over the results of a survey made in 2008? It is worthless.

“The politicians, propagandists and the converted acolytes haven’t gotten the message. If this survey does not educate them, nothing will”.
Of course they haven’t gotten the message because they invented the scam and or earn their fortunes and living maintaining it.
And they will do until we are out of money or the risks get too high.
The so called Consensus is an integral part of the AGW scam that rests on at least 7 basic strategies and lies.
1. demonize oil
2. peak oil has arrived
3. CO2 is a powerful green house gas
4. The climate is warming in a unprecedented manner
5. 1.000 scares
6. All scientists agree AGW is real, any (scientific)opposition is useless (consensus)
and public opposition is demonized (by the argument of consensus).
7. Drastic action is needed, a radical change of our life styles, our energy infra structure, Taxes and the introduction of sustainable energy, solar wind, bio fuels and nuclear.
Of course the “inventors” of this scam knew their carefully planned scam long before we started to discuss the subject at the skeptic blogs.
So Thomas Fuller, do you really think you can educate these people, make them change their mind?
Do you really live with the illusion this will bring anything?
Don’t you get the feeling you’re kicking in open doors?
Really I am very pleased there are significant doubts about this scam among scientists but during the past years all of the AGW claims have been debunked and the scam has been unmasked.
But as long as the gravy train continues and government grants exist we have to live with the direct results: wind parks are build, electricity prices are going through the roof, bio ethanol is mixed with gasoline in many countries, food prices are on the rise, 1.2 billion people live from less than 1700 calories per day, the AGW propaganda machinery is running full steam ahead and we have to endure one crazy scheme after another to save our climate.
Right, education is required to wise up and inform the public.
But I am afraid we seriously need to think about much more pressing way’s to stop this fraud before it bankrupts us.
And it’s not education.

But this is 2 years old. Before Climategate. My guess is that the results are meaningless today. And in any case, I doubt whether a mere survey will educate people.

Anthony I have a suggestion prompted by this topic of who believes what.
You have on the right columns with web links to other discussion sites. It seems to me that to assist the advance of common sense and to reduce the polarisation so evident in public climate discussion, the list needs to be further sub-divided.
There should be a ‘catastrophists’ category where the lunatic fringe hang out. Content would obviously be the main criterion. Six metres of sea rise and a Venusian runaway heating of the atmosphere would be reasonable qualification for listing.
Then a ‘Climate Taliban’ list where different opinions are routinely repressed and polite discussion of alternative viewpoints is not allowed. It would be easy to construct a pass/fail test for those sites.
Then the better known ‘warmist’ sites edited by those who tend to believe preferrentially and enjoy doing so and are probably open to new information undermining or supporting their favourite prejudices.
Next is the neutral zone where all polite discussion is encouraged (what major science publications used to be). This is where the majority of so-called skeptics are.
The Last and Least category would be the agenda-driven sites where people rage against the One World Domination of satan they heard about in church or the Black UN Helicopters and the partyists seeking to remove this or that president of some union or country – in other words UUNET et al. The fringe sites: ‘it’s all a giant plot’ are not skeptical, they are just chillist versions of the Climate Taliban. Interesting how all extremists look the same from a distance.
I see messages here that appear to be written by Climate Taliban pretending to be skeptics, seeking to undermine the common sense reputation of this list. That is, I am sure, unavoidable but a reasonable person can recognise and ignore them. It is the editorial policy and skill that defines a site. Skilled scientists and the skilled public will always be attracted to the places where meaningful conversations are held, which of course tend to the center.

Wait, there’s more, from page 10;
“The combined invitation list numbered a potential 2677 respondents; defunct email addresses reduced the valid mail out to 2059. Invitations to participate in the survey were distributed by email, providing a link to the on-line survey. Provisions were made so that should someone submit a duplicate form the form identifier resulted in the original being over written.”
So it was an ONLINE survey, yes?
Well then, how do we know the TRUTH of the respondents?
How do we even know that only actual climate scientists responded to this ONLINE survey?
Were the responses validated in some way?
I would hope so, but this PDF does not go into those details.
Email verification AFTER completion of said survey?
Or reverse IP checking?
Who knows?
Who really knows?

By 1850, the Bible was being replaced Archaeology studies and findings.
In 1850, certain people began to replace a cooling Earth born in 7 days in 4004 BC. With a warming Earth from an Ice Age.
In the same period man from God was replaced by a man from an Ape.
We have come so far in the last 160 years in being like are new maker.

Only 86.5% percent believe the climate is changing. That means 13.5% of the repondents don’t know what they are talking about. The climate has always been changing. Mostly in a cyclic fashion. It is one of the proofs that man is not the cause of climate change. It has been warmer than is now and it has been a lot colder than it is now and man was not the cause, nor is man the cause now.

I think most conspiracies are done haphazardly and after the fact for the most part. To think that when the global warming scare begun that bankers had this idea to create a new “carbon” economy is kind of far-fetched considering the number of years needed in order to be patient…People with money do not tend to want to wait 20 years to make money to say the least.
What is more likely is that eco-nuts like Hansen dive-bombed the scientific process in order to get their agendas across, slowly convinced other nuts and politians (is there a difference between the two?) and slowly the bad science permeated everything it touched.
When the funding for science and reputations where based on bad science, in order to make their names be well-known, the scientists fudged a litte more and more until we had the hockey stick, the IPCC and then it became a political madhouse. The bankers saw the investment scheme and the dollars to be made to make money trading air, and then everyone was involved to the point that pulling out would be political/reputation suicide.
So now we have people who have no choice but to stick to what they believe, said etc. Its not a conspiracy in the sense that they thought this up 20 years ago, its a conspiracy after the fact since they have no choice but to work together to try to save their names/jobs/money etc. It is funny or ironic that in most cases conspiracy in general just makes things worse for those involved, and by trying to cover their reputations/asses in general they have came to the breaking point now where most of them will be thrown out as a bum.
Some people will lose large amounts of money. And some politicians will lose their careers. Some people might even go to jail. If you are willing to accept the consequences of your actions, you will never be in a position to be involved in a conspiracy…hence the irony. These eco-nuts such as Hansen though will only go down when they are thrown into jail or on the streets for being worthless. Until then, they will fight it out until the bitter end. This is why its important to fight the non-sense that these nuts thought up and throw them out into the streets. Only then can we begin to remake science.

I was a little puzzled by some of the response distributions and I did read the entire paper. Then it struck me, way back at the top it listed some of the demographic information. “78.31 of the respondents are involved in Physics-modeling”, that explains many of the skewed distributions > 3.5.

Dennis Nikols,
lysdexia strikes again, 78.13 % not 78.31% 🙂 yes, I know it should be dyscalcula but that would ruin the joke
I wouldn’t read too much into that number, the categories were such that the physics-modeling group includes most who study climate from any hard science sort of perspective. Some of the people who study impacts might also be hard science types but I doubt many of the policy or communications groups are. Anyone who isn’t in the physics-modeling group can be considered not to have a handle on most of the questions.

Tom Fuller says:
September 25, 2010 at 7:08 pm
EFS Junior,
Online surveys are not perfect–as you point out, nobody knows you’re a dog on the internet. However, a lot (and I mean a lot) of money has been invested comparing the results of surveys done on the internet with surveys conducted with older technologies, such as telephones and face to face interviews.
The conclusions are that no methodology is perfect, that people can game internet surveys, but they can also game other types of survey as well. But internet surveys when properly done can successfully predict winners in political elections, products that will perform, movies and tv shows that will be hits, just as well (and sometimes better) than other forms of getting data from respondents. Internet surveys can get close to the actual results–sometimes as close as other surveys. (There is the occasional foul-up, but they are just that–occasional.)
Internet surveys are not perfect. But they do okay.
_____________________________________________________________
Have you ever watched “The Ed Show” on MSNBC? If not, not to worry, but during each show he asks the viewing audiance a “polling” question, and usually 90+% give the desired answer. I’m sure the same can be said of Faux Noise (or Fixed News).
I can state with very high probability (p > 0.99) that this survey is entirely bogus.
So how can I be confident in the statement above, you ask?
See page 19 (figure) where 78.13% (293 out of 375) are categorized as “physics-modelling” which seems to be an awfully high number, since when I see “modelling” I think numerical climate science modelling (having been a hydrodynamic numerical modeler myself).
Just above that graph “physics-modelling” includes the following;
“8. The nature of your work is best described as being concerned with
physics of the climate system (modelling, model development, data acquisition, theory development, etc
…”
Etceteras must be a pretty large catagory!
But seriously, of the four explicit areas mentioned, three of them deal with GCM’s in some way (modelling, model development, and theory development).
Now I go to The Union of Concerned Scientists (UCS) 2006 survey (BTW very similar polling methodology) of federal (and NCAR) climate scientists;
http://www.ucsusa.org/scientific_integrity/abuses_of_science/federal-climate-scientists.html
Where 308 climate scientists responded out of 1630 (18.9% again quite similar to this Bray and von Storch (BS) survey response rate), but only 21% (or 65 out of 308) self-proclaimed themselves as “Modeling”.
So which survey should I believe, the 2008 one with the ambigious definition of “physics-modelling” of 78.13% (hint BS), or the more direct definition of 21% as “modeling” from the 2006 survey (UCS)?
The answer should be obvious, but it would be the 21% from the 2006 UCS survey.
78.13% vs 21% (almost a factor of four difference), now do you really think 78.13% of climate scientists are, in fact, “in-the-trench” numerical climate science modelers?
I know I don’t.
Something is seriously wrong with the 2008 BS survey to get almost 80% in a SINGLE catagory (hard science vs all others (social sciences, policy, health, communication, administration, and others)). You either go for the hard sciences alone, than break down that group into subcatagories (with numerical modelers being one explicit group).
Where I work, it goes something like this: theoretical, experimental, prototype, observational, empirical, statistical, numerical, physical, etceteras.
IMHO very poor survey design.

Watts Up With That? does not need “fillers”.

EFS_Junior says:
September 25, 2010 at 6:32 pm
So it was an ONLINE survey, yes?
Well then, how do we know the TRUTH of the respondents?
How do we even know that only actual climate scientists responded to this ONLINE survey?
Were the responses validated in some way?
I would hope so, but this PDF does not go into those details.
Email verification AFTER completion of said survey?
Or reverse IP checking?
Who knows?
Who really knows?

You’re grabbing at straws now. Many objective studies and most opinion polls are done ONLINE today. Most of the issues you raise are easily controlled for by an experienced study developer. Irrespective, it’s an opinion poll, not a study or trial. The TRUTH of the respondent isn’t a quality control consideration as an opinion isn’t based on fact, value, or TRUTH. The study’s methodologies are reasonably acceptable for its intended purpose.

EFS_Junior says:
September 25, 2010 at 10:20 pm
Did you just poke fun at Fox’s slant on news just after admitting to watching MSNBC? I guess that explains why you are so desperately trying to find fault with the survey. Junior, if you’ve ever taken one of those types of polls, then you’d know they don’t often have job descriptions that entirely match the person’s function. So they checked a box and went on. What does a climate scientist do if he/she isn’t involved in modeling, data acquisition or theory development?
Junior, its just a poll, it isn’t like we didn’t already know there wasn’t ever a consensus.

You buried the lede. If you look at the study, question no. 74, the study asks for an open-ended answer on the most pressing problem facing humanity. Almost no one said AGW. Clean drinking water and overpopulation appeared to be popular answers.

Thomas Fuller,
I don’t think you should get carried away interpreting this survey. Scientists are perfectionists and your “adequate” is quite likely to be different from your or Joe Bloggs “adequate”.
For example if you surveyed physists about the “adequacy” of Newton’s Law of Gravity you might not get consensus there either.
The whole “science is settled ‘NOT” meme was bogus from the start. The IPCC was pretty explicit about the uncertainty in it’s conclusions. So it did not think the science was settled. The scientists did not stop studying climate and find another career. So they did not think all uncertaintly was removed.
“The science is settled” phrase is short for the “the science is sufficiently settled to start making political decisions”. This is distinct from the alternative position “we need to understand the climate perfectly before we make a decision”. The latter is equivalent to “do nothing until it’s too late, then blame people for not warning us”.
Maybe people here should ask themselves how they would decide in the following 2 scenarios:
1. The Army Corp of Engineers recommends that the missisipi levees need to be reinforced, just in case we get a hurricane, but we are unsure if this us ever going to happen.
2. The CIA reckons some Muslim nutter has tried to blow to blow up the WTC before and there is reason to believe he is going to try again, but we are not 100% sure.
In each case I am going to suggest someone produced a report recommending action. And some bureaucrat denied it.
What would you do?

So the consensus among climate scientists is largely the climate is changing, human activities impact it, the impact could be bad, we don’t know enough about it, give us money so we can learn more.
The consensus among profiteers in alternative energy and carbon reducing technologies is rate and increase of fossil fuel consumption is unsustainable and wreaking the environment, send us money so we can fix it.
The consensus among politicians is that there’s a huge opportunity to exert more control over the plebs and increase tax revenues by taking over how energy is produced and consumed.
I think that about wraps it up. The common denominator is money. Follow the money. It never fails.

I expect that “climate scientists” are almost all environmentalists. Asking environmentalists whether there is good evidence that humans are causing great damage to the environment is like asking Democrats if Barack Obama is a good president. Scientists who don’t think the evidence for AGW is solid are probably not going to be “climate scientists”. A consensus of climate scientists is not a scientific consensus.

wws says:
September 25, 2010 at 4:22 pm
“’Consensus’ is a purely political word – it has no place in science at all.”
I ‘strongly agree’ with wws’ statement. I’d like to add that the expression “settled science” is a shibboleth for recognizing the scientifically illiterate, including His Goreness. Real scientists do not use that particular buzz-phrase, because real science is seldom settled.
Example. A minority of physicists have been sniping at General Relativity for many years. Some non-scientists take a jaundiced view of the sniping. But to me, the sniping is evidence for the robustness of the scientific approach to gaining knowledge.
Even after all of these years, you can get your hands dirty, improve upon existing measurement techniques, retest GR, and get published in a peer-reviewed physics journal. And that’s true even if you don’t demolish GR. Being able to say that GR is consistent with your measurements, within a smaller-than-ever-before experimental error, is good enough. GR is a very strong theory. But it would be grossly inaccurate to describe it as “settled science”.
Sometimes it’s very easy for scientists to recognize BS about science in the popular press. For example, when the author demonstrates that he has zero understanding of the concepts behind the technical terms that he uses in his writing. Heretical views are not necessarily evidence that justifies the classification of an idea as junk science, or that justifies the classification of an individual as a ‘denier’.
True Believers are the ones who engage in hate speech against heretics, blacklist heretics, and burn heretics at the stake. True Believers and real scientists are mutually exclusive categories of people.
Nonstandard thinking (NT) has an important and ‘settled’ place within real science. Without NT, we’d still be living in mud huts, and sacrificing virgins to the volcano gods whenever the weather isn’t to our liking.

September 26, 2010 at 12:53 am LazyTeenager says:
“The science is settled” phrase is short for the “the science is sufficiently settled to start making political decisions”. This is distinct from the alternative position “we need to understand the climate perfectly before we make a decision”. The latter is equivalent to “do nothing until it’s too late, then blame people for not warning us”.
Maybe people here should ask themselves how they would decide in the following 2 scenarios:
1. The Army Corp of Engineers recommends that the missisipi levees need to be reinforced, just in case we get a hurricane, but we are unsure if this us ever going to happen.
2. The CIA reckons some Muslim nutter has tried to blow to blow up the WTC before and there is reason to believe he is going to try again, but we are not 100% sure.
In each case I am going to suggest someone produced a report recommending action. And some bureaucrat denied it.
What would you do?
——-
I’d say that you’ve got your examples backwards. Our present global response to AGW is the panicky gut reaction to a perceived but poorly understood threat #2. What we should be doing is sensible spade work for a potential problem, forming sensible plans, doing win, win activities and putting money aside for a rainy day #1.
Much of the public backlash against CO2 reduction schemes is because they make no sense. Corruption has already set in. Scientific arguments have been more about political egos than actual knowledge. The lies and half truths told about AGW have damaged its credibility. The hypocrisy and pocket lining from key proponents is sickening. Large amounts of money and public good will have been frittered away.
Much of this could have been avoided if a more measured approach had been taken.

‘E.M.Smith says:
September 25, 2010 at 8:55 pm
In 2008 Al Gore said “the entire north polar ice cap will be completely gone in five years.”’
You refer to an interview on German TV that had to be removed from YouTube:
http://wattsupwiththat.com/2008/12/14/gore-entire-north-polar-ice-cap-will-be-gone-in-5-years/
Rick Sharp (Dec 14, 2008, 5:56 pm) commented then that he had seen him saying the same thing in 2007. So “the arctic is expected to be ice free in 2012”.
Of course Gore today still uses the phrase “in 5 years” or even “in 5 to 7 years”, thus continuously moving the goal posts.

There once was claimed that the so called MWP (Medieval Warm Period) is not even a Northern hemispheric phenomenon. I startet a blog on this question, interviewed 8 scientists until yet and talked/mailed with 20 others. The tenor was, there seems to be no real dissence that the “temperatures in High Medieval time were warmer than during the subsequent Little Ice Age” (Bradley et al.: Climate in Medieval Time, in: Science, Vol 302, 405) or in other words “that mean temperatures during this interval were warmer than the subsequent Little Ice Age” (Crowley and Lowery: How Warm Was the Medieval Warm Period?, in: Ambio Vol. 29, No. 1, 2000, 54) at least for the Northern Hemisphere. As there is now evidence for a global LIA it seems to be very clear that the time before the LIA must have been warmer in all areas. For me the construct of a MWP – the concept of a MCA is not functionally adequate for several reasons – seems to be adequate for the NH and the SH – in spite of that there is less data available -, linked to the appearance of a LIA in this area, varying in extend and time.

LazyTeenager says:
September 26, 2010 at 12:53 am
““The science is settled” phrase is short for the “the science is sufficiently settled to start making political decisions”. This is distinct from the alternative position “we need to understand the climate perfectly before we make a decision”
So let us take the evidence we have and decide if we can make a decision. Has the troposphere warmed as modeled?
“What Do Observational Datasets Say about Modeled Tropospheric Temperature Trends since 1979?­¬«” This paper states that no it hasn’t.
But the stratosphere has cooled right? Just like modeled? “Ozone and temperature trends in the upper stratosphere at five stations of the Network for the Detection of Atmospheric Composition Change ” states that no, that hasn’t happened either.
We can at least point to the oceans and state with certainty that they are warming according to models and there is a huge energy imbalance right? “Recent energy balance of Eartht” states no we can’t.
But ARGO is a new system and not reliable and we need to rely on sea levels as measured by GRACE? “Uncertainty in ocean mass trends from GRACE” this paper does not put the accuracy of GRACE in a particularily good light. And even if it did the sea level attributions are imcomplete with major contributors ignored.
I don’t think most intelligent people would expect a complete and perfect understanding of the climate before we make a decision. I do think that people should be able to expect at least one major line of evidence to be clearly supportive of catastrophic warming. As of right now the only evidence I have seen is that we are warming. This warming is occuring consistant with a climate sensitivity of 1.1C as stated in “Why Hasn’t Earth Warmed as Much as Expected?”.

Interesting that the answers to Q27 “The potential that climate change may have some positive effects for the country where you live and other parts of the world” is pretty balanced whereas to Q28 “How much do you think the potential impact of climate change is one of the leading problems for eco-systems and for humanity” there is a far more positive agreement.
This seems to indicate that a majority of scientists see positive effects as a problem! Its nice to know they care about us.

Lazy Teenager, that CO2 has radiative properties does not prove that man’s release of an ‘unnatural’ aliquot has modified climate. It’s just that simple.
Ben D. @ 7:43 PM yesterday. Say it again. Louder. Louder.
JP Miller @ 3:07 AM. Yes, you point out the fundamental political problem.
And thanks, banjo @ 5:14 AM for the Delingpole link. The globe is cooling, folks; for how long even kim doesn’t know. And apparently the Bilderbergers don’t either.
====================

Those who believe the science is settled should act according to their views and reject as unnecessary any further funding of climate research.

By the way, it’s disappointing that the 2010 Bray and von Storch paper is based on a 2008 survey. We can rest assured that things are quite different than they were two years ago.

E.M.Smith says: September 25, 2010 at 8:55 pm
In 2008 Al Gore said “the entire north polar ice cap will be completely gone in five years.”
I make that about 2013. Given that it’s late 2010, aren’t we about 1/2 way? So how about a graph of the monthly “Gore Consensus” of a 2008 to Zero trend compared with the actual 2008 to date trend? I’d like to see how the “consensus” is measuring up to reality?
E.M., it appears to be faring poorly. Should the trend sudden increase some 73 fold the ‘Gore Line’ will be back on track. See: http://i51.tinypic.com/67us8k.jpg

janama says: September 25, 2010 at 6:45 pm

And 86.5% agreed or strongly agreed that “climate change is occurring now” and 66.5% agreed or strongly agreed that future climate “will be a result of anthropogenic causes.” if they disagreed their departments would close and they’d be out of a job!

That’s the key to the whole survey. Sad that really the survey is fairly useless post-Climategate – UNLESS von Storch redid it to compare. Interesting questions were asked – but still, of little use unless tempered by janama’s insight.
Vidkun Quisling says: September 26, 2010 at 12:02 am

You buried the lede. If you look at the study, question no. 74, the study asks for an open-ended answer on the most pressing problem facing humanity. Almost no one said AGW. Clean drinking water and overpopulation appeared to be popular answers.

Very well spotted. But pleeeeeeeeeease change your handle. I’m only half Norwegian and that’s still enough.

Well, the consensus appears to be so far advanced amongst the Greens that our UK’s only Green MP thinks it’s a job for the army:
http://www.independent.co.uk/environment/climate-change/let-mod-tackle-climate-says-lucas-2089975.html
‘Only if global warming is classified as an issue of “national survival”, like a military threat, will it be treated with the urgency it now needs, she told the Sustainable Planet conference in Lyon.’

Compare the questions on the actual climate science with ones concerning the prediction of problems in the future. The respondents are quite a bit more confident in the future disasters than in the science. Responses to science questions tend to be 4 or less. Responses to conclusion questions tend to be 4 or much greater 5,6.

@Pamela
‘With the advent of CO2 inspectors, we would all, in reality, be police states.’
No problem – here in the UK/EU we pretty nearly are already. And I’m sure that there has to be at *least one US politician who believes much the same as Lucas.

LazyTeenager says:
September 26, 2010 at 12:53 am
“Maybe people here should ask themselves how they would decide in the following 2 scenarios:
1. The Army Corp of Engineers recommends that the missisipi levees need to be reinforced, just in case we get a hurricane, but we are unsure if this us ever going to happen.
2. The CIA reckons some Muslim nutter has tried to blow to blow up the WTC before and there is reason to believe he is going to try again, but we are not 100% sure.
In each case I am going to suggest someone produced a report recommending action. And some bureaucrat denied it.
What would you do?”
========================================================
Maybe you should try to ascertain as to why neither scenario applies. In both scenarios mentioned, we had enough information (history) to make educated guesses about the events reoccurring. As to the WTC scenario, I don’t believe the WTC was a known target, but even if it were, the who, when, how wasn’t known. So, I’ll ask you the same question, what would you have done? Given the porous state of our borders, I can’t help but think Divine intervention has saved us from a re-occurrence. We can blame 9/11 on anyone we want, but in the end the fault lays at the feet of the terrorists and their supporters.
As to the flooding scenario, flooding occurs. An example………The local river around my area floods from time to time. (every 20-30 years) It was determined our levees needed reinforced. Eminent domain was invoked, farmers lost fertile ground and production, levees were reinforced and flooding occurred in areas that weren’t reinforced. Again, I’ll ask you the same question, what would you have done? We can blame the bureaucrats, engineers or politicians, but in the end of the day, it is apparent Nature will do what Nature does.
We could accept that life comes with certain inherent risks, or we can try to eliminate all risks to life and we end up with ridiculous quixotic outcomes such as drowning warning labels on 5 gallon buckets, caps on our CO2 emissions and the elimination of the most reliable and cheap energy sources available.

LazyTeenager,
“Maybe people here should ask themselves how they would decide in the following 2 scenarios:
1. The Army Corp of Engineers recommends that the missisipi levees need to be reinforced, just in case we get a hurricane, but we are unsure if this us ever going to happen.”
This is, of course, an example of the precautionary principle. I don’t think most people would have a problem with sprinkling fairy dust that would magically transform the world into a sustainable, efficient and carbon neutral source of energy. Heck, you would have to be crazy not to.
But what happens in reality is a sequence of events that start from alarm, leading to ill considered knee-jerk policy decisions that have no practical mitigation on the problem in hand, and come at a cripling economic cost to-boot. You have governments subsidising biofuels that are leading to rain forest destruction and food price rises while mitigating CO2 by insignificant amounts. You have governments rushing to divert billions of taxpayers money into building legions of wind farms that consume enormous quantities of steel, copper and concrete to build and erect – and then have to have fossil fuel powerstations built to back them up when the wind isn’t blowing at the correct speed, saving negative amounts of CO2 emissions. These policies are doing nothing to mitigate against climate change, even if it were a problem (which you admit is uncertain) and costing exponential increments of public money.
Using your analogy, can you explain why this is the correct action to take?

brian says:
September 26, 2010 at 3:28 am
Sorry O/T.
‘Away match at http://www.leftfootforward.org/2010/09/time-to-shift-the-climate-debate-on-to-renewables/.
Bring your best arguments.’
My comment is awaiting moderation over there. Thanks for the pointer.
Dominic

Hank Hancock says:
September 25, 2010 at 10:57 pm
EFS_Junior says:
September 25, 2010 at 6:32 pm
So it was an ONLINE survey, yes?
Well then, how do we know the TRUTH of the respondents?
How do we even know that only actual climate scientists responded to this ONLINE survey?
Were the responses validated in some way?
I would hope so, but this PDF does not go into those details.
Email verification AFTER completion of said survey?
Or reverse IP checking?
Who knows?
Who really knows?
You’re grabbing at straws now. Many objective studies and most opinion polls are done ONLINE today. Most of the issues you raise are easily controlled for by an experienced study developer. Irrespective, it’s an opinion poll, not a study or trial. The TRUTH of the respondent isn’t a quality control consideration as an opinion isn’t based on fact, value, or TRUTH. The study’s methodologies are reasonably acceptable for its intended purpose.
_____________________________________________________________
Strongly disagree.
I’ve posted links to the accuracy of these types of surveys, and we know a priori, that these types of surveys, targeted as a very select subset of “climate scientists” will not be as accurate, and that meaningfu statistics can’t be, and should never be, derived from such a priori biases going into these types of survers, we need a control group, where’s the control group?
Specificity is critical here, if the goal of this survey was to capture the hard science types (and given the demographic choices and 78.13% self identifying with the first group “physics-modelers, demands that the hard science group be further divided with a more specific demographic subdivision question for the 78.13% of all who fall into said hard sciences group (e. g. I know that 78.13% are NOT numerical climate science modellers, so for this entire group to opine on said subject matter is not an accurate statistical measure of the opinions of those that have direct contact (either as developers or end users) with numerical climate science modelling (GCM’s)).
It’s akin to asking what’s your favorite color, than running the numbers, and claiming that since most people chose “green” then “green” is the de facto best color with a MOE of +/- 5.2% based on a sample size of 375.

Hans von Storch has been polling climate scientists for several years now. After each poll he improves his methodology. I don’t know if his methodology is perfect (though his sample size of 379 seems plenty big enough) but it is most likely good enough to provide useful info on a very interesting subject.
What is surprising and puzzling is why it took two years to release these results.
Here is some interesting stuff from wikipedia about von Storch:
[von Storch said] “Based on the scientific evidence, I am convinced that we are facing anthropogenic climate change brought about by the emission of greenhouse gases into the atmosphere.”[1]
He is also known for an article in Der Spiegel he co-wrote with Nico Stehr, which states that:
“Scientific research faces a crisis because its public figures are overselling the issues to gain attention in a hotly contested market for newsworthy information.”[2]
“The alarmists think that climate change is something extremely dangerous, extremely bad and that overselling a little bit, if it serves a good purpose, is not that bad.”[3]
In December 2009, he expressed concern about the credibility of climate scientists and compared Global Warming alarmism to the German Waldsterben (Forest dieback) hype of the 1980s:[4]
Research about the forest die back in Germany may serve as an example at the other end of the spectrum. The science of forest damages was in the 1980s heavily politicized, and used as support for a specific preconceived “good” policy of environmental protection. The resulting overselling and dramatization broke down in the 1990s, and news about adverse developments in German forests is now a hard sell in Germany. An observer wrote in 2004: “The damage for the scientists is enormous. Nobody believes them any longer.” Of course, the damage was not only limited to the forest researchers, but also to other environmental scientists and politicians as well.”

Steve Koch says:
September 26, 2010 at 1:13 pm
Hans von Storch has been polling climate scientists for several years now. After each poll he improves his methodology. I don’t know if his methodology is perfect (though his sample size of 379 seems plenty big enough) but it is most likely good enough to provide useful info on a very interesting subject.
What is surprising and puzzling is why it took two years to release these results.
_____________________________________________________________
I asked the same question to myself.
Since political polls are turned out within a few days, or even a day after the polls are taken.
The obvious answer?
Bray and von Storch got first dibs for publication purposes;
http://www.climatescienceandpolicy.eu/2010/08/how-do-scientists-assess-the-skill-of-climate-models/
At the bottom;
■Bray, D., 2010a: Consensus among climate scientists revisited. Env. Sci. Policy, in press.
■Bray, D., and H. von Storch, 2009: ‘Prediction’ or ‘Projection’? The nomenclature of climate science. Sci. Comm. 30, 534-543, doi:10.1177/1075547009333698
■Bray, D., 2010c: Baltic Climate Scientists Assessment of Climate Change and Climate Science in the Baltic Sea Basin. BALTEX report, in press
I found the author’s copy of the first citation here;
http://dvsun3.gkss.de/journals/2010/Bray-envscipol.pdf
A 2009 version of the above report can be found here;
http://coast.gkss.de/staff/storch/pdf/CliSci2008.pdf
And The BALTEX report can be found here;
http://coast.gkss.de/staff/storch/pdf/surBACC report v3.pdf
As of yet, I have been unable to find a “free” copy of;
Bray, D., and H. von Storch, 2009: ‘Prediction’ or ‘Projection’? The nomenclature of climate science. Sci. Comm. 30, 534-543, doi:10.1177/1075547009333698
I guess I’ll email the authors to request a copy on Monday.

EFS_Junior says:
September 26, 2010 at 12:20 pm
I’ve posted links to the accuracy of these types of surveys, and we know a priori, that these types of surveys, targeted as a very select subset of “climate scientists” will not be as accurate, and that meaningfu statistics can’t be, and should never be, derived from such a priori biases going into these types of survers, we need a control group, where’s the control group?

A priori what? It’s an adjective, not a noun. Of course the study targets a select demographic or subset as you call it. The study’s results are clearly in context to that demographic and not intended to be representative of a more general population. Regarding priori bias, I don’t see where the questions make pre-existing assumptions, ignore fact, exclude alternative explanations, or fail to challenge theory. If you do, please point out the specific questions that do so and explain why.
I agree that meaningful referential statistics aren’t built on priori knowledge, if that’s what you’re saying. However, the purpose of the survey is to infer opinion or sediment of the sample group. The target audience (those of us concerned with their opinion) is more interested in where the majority of responses center. A scale of minimum graduation, as used in this study, makes perfect sense.
A control group is used mostly in clinical or quantitative studies whereas this would be classified as a demographic or qualitative study. As such, no control group is required. Besides, how do you control for opinion? What would the demographics of the control group look like? How would you justify such demographic as controlling for the study group? I hope you can see where your criticisms of the study methodologies really don’t make much sense.

Whether sample results can be generalized to a larger population depends on the representativeness of the sample, not its size. A large sample can still be quite biased. The usual significance tests or confidence intervals give no protection against biased sampling.
Questions can be asked in such a way as to bias the answers, as well. I was struck by this one, a good example of what honest pollsters mostly try not to do:
“Some scientists present extreme accounts of catastrophic impacts related to climate change in a popular format with the claim that it is their task to alert the public. How much do you agree with this practice?”
The first sentence asks readers to accept a premise described with loaded words, “extreme,” “catastrophic,” “claim.” If I encountered that question on a poll I would see where the pollster was trying to push me, and probably give the answer he doesn’t want, just to push back. It looks like other folks had that reaction, as well.

Hank Hancock says:
September 26, 2010 at 5:03 pm
EFS_Junior says:
September 26, 2010 at 12:20 pm
I’ve posted links to the accuracy of these types of surveys, and we know a priori, that these types of surveys, targeted as a very select subset of “climate scientists” will not be as accurate, and that meaningfu statistics can’t be, and should never be, derived from such a priori biases going into these types of survers, we need a control group, where’s the control group?
A priori what? It’s an adjective, not a noun. Of course the study targets a select demographic or subset as you call it. The study’s results are clearly in context to that demographic and not intended to be representative of a more general population. Regarding priori bias, I don’t see where the questions make pre-existing assumptions, ignore fact, exclude alternative explanations, or fail to challenge theory. If you do, please point out the specific questions that do so and explain why.
I agree that meaningful referential statistics aren’t built on priori knowledge, if that’s what you’re saying. However, the purpose of the survey is to infer opinion or sediment of the sample group. The target audience (those of us concerned with their opinion) is more interested in where the majority of responses center. A scale of minimum graduation, as used in this study, makes perfect sense.
A control group is used mostly in clinical or quantitative studies whereas this would be classified as a demographic or qualitative study. As such, no control group is required. Besides, how do you control for opinion? What would the demographics of the control group look like? How would you justify such demographic as controlling for the study group? I hope you can see where your criticisms of the study methodologies really don’t make much sense.
_____________________________________________________________
I’ve posted all I need to post on this subject matter, the dubious quality of these types of survery exists purely on the grounds of their type catagory. Therefore and forthwith, a priori in nature and with good potential for grossly spurious outcomes.
Meaningful statistical analyses can not strictly be applied to these types of surveys since they are not truly “random” of the entire climate scientists population. A control is necessary and necessarily absent in these types of surveys, can’t be helped, they are what they are.
The authors themselves readily admit to the survey type in their own words, yet offer no reasoning for, or defense fo, the survey method, except for their “ease of use” disclaimer.
I’ve also argued persuasively about the deeply flawed demographics subdivision, particularly when asked to a largely unknown demographic of one hard science catagory to choose from vs several soft science (or otherwise) groups. In other words the demographics lack sufficient granularity to, for instance, distinguish numerical climate science developers modelers and end users, climate science theoreticians, experimentation, observational, the list goes on and on.
Sediment? Sentiment.
You have the entire population (the obvious control group), and you have the sampled population (again not randomly selected in these BS surveys, ergo the basic flaw of these types of surveys, particularly when there is a very specific target population as we would have here, for obvious reasons).
A sample size of only 375 when there must be tens of thousands of practicing climate scientists worldwide, well there are statistics for this (e. g. MOE), when the sample is truly randon, definitely not the case here.
Also what muddies the waters somewhat, are the selectable scale range of 1 thru 7 (most studies I’ve seen use 5, but if you are going to use 7, than why not 9, or why not 11, or why not …), which further increases MOE statistics considering the small non-random sample size (e. g. 375/7 ~ 54 for a uniform distribution (and yes, I realize that the distributions are not uniform)).
The IPCC AR5 is just a few years away, now that would be the most opportunistic time to capture thousands of climate scientists in the heat of battle, as it were, a very well defined control group if you ask me.
Until then, the best we can do is to rely on many existing sources, in addition to these BS surveys, as for example shown here (BS 2008 is included in this list BTW);
http://en.wikipedia.org/wiki/Scientific_opinion_on_climate_change
“This article is about scientific opinion on climate change. For recent climate change generally, see Global warming. For debate on scientific consensus, see Climate change consensus. For opinions of individual dissenting scientists, see List of scientists opposing the mainstream scientific assessment of global warming.”
Take it, or leave it, but it is the best composite we have to date, to judge the overall scientific opinion on climate change. All encompassing, no cherry picking allowed. BS is just one snapshot from very many snapshots. Some snapshots are inherently better than others, and IMHO BS is not one of them. An opinion on opinion polling methodologies. Go figure.
My final words on this matter, you get the final say, as I’ve said all that has been needed to say on this matter at this point ad infinitum, ad nauseam.

The results of the CliSci2008 survey are interesting. Bray and van Storch (BS) utilized a Likert Scale (1 to 7) in their survey and a bucket graph to portray the degree of consensus among the responses to the ~ 100 questions in the survey. BS reported the average Likert Score (LS) and the associated STDEV for each score in their article.
I did not find the bucket graphs very helpful in understanding the degree of consensus among the respondees. Therefore, I 1) normalized the STDEV of the responses to the questions and then 2) rank-ordered the results. The average of the STDEVs is 1.346 and the STDEV about that average is 0.167.
The five questions with the highest degree of consensus, i.e., lowest normalized deviation wrt 1.346, are:
1. Q87 The IPCC reports tend to under estimate, accurately reflect (a value of 4) or over estimate the magnitude of future changes to temperature:
• under estimate 1 2 3 4 5 6 7 over estimate
• LS = 4.00559
• Normalized deviation = -2.66163
2. Q79 The IPCC reports tend to under estimate, accurately reflect (a value of 4) or over estimate the magnitude of future changes to temperatue:
• Not at all 1 2 3 4 5 6 7 very much
• LS = 3.98619
• Normalized deviation = -2.48325
3. Q56 How convinced are you that climate change, whether natural or anthropogenic, is occurring now?
• under estimate 1 2 3 4 5 6 7 over estimate
• LS = 6.44474
• Normalized deviation = -2.20425
4. Q88 The IPCC reports tend to under estimate, accurately reflect (a value of 4) or over estimate the magnitude of future changes to precipitation:
• under estimate 1 2 3 4 5 6 7 over estimate
• LS = 3.83662
• Normalized deviation = -2.20254
5. Q70 In making policy decisions about adaptation decisions about adaptation to climate change, priority should be given to
• Industry and commerce 1 2 3 4 5 6 7 scientific Expertise
• LS = 5.33060
• Normalized deviation = -1.55093
The five questions with the lowest degree of consensus i.e., wrt to 1.346 are:
6. Q12 To what degree do you think climate science has remained a value-neutral science?
• Not at all 1 2 3 4 5 6 7 A great deal
• LS = 3.96226
• Normalized deviation = 1.58676
7. Q65 The potential that climate change might have some positive effects for the country in which you live is
• Very low 1 2 3 4 5 6 7 very high
• LS = 3.88859
• Normalized deviation = 1.76123
8. Q116 There is a great need for immediate policy decisions for immediate action to mitigateclimate change
• Strongly disagree 1 2 3 4 5 6 7 Strongly agree
• LS = 5.505040
• Normalized deviation = 1.84350
9. Q110 Making discussions of climate science open to potentially everyone through the use of blogs on the w.w.w is
• Avery bad idea 1 2 3 4 5 6 7 A very good idea
• LS = 4.58197
• Normalized deviation = 1.92996
10. Q62 If we do not do anything towards adaptation or mitigation, the potential for catastrophe resulting from climate change for other parts of the world :
• Very Low 1 2 3 4 5 6 7 Very high
• LS = 4.61351
• Normalized deviation = 2.17793
I would consider the following ranges of normalized deviation wrt 1.346 to be reasonable for classifying the degree of consensus:
• Very low: less than – 2.5
• Low: -1.5 to-0.5
• Medium: -0.5 to +0.5
• High: +1.5 to +2.5
• Very high: More than 2.5

Tom Fuller,
I do think you should point out that this is also considerable evidence that this survey was contaminated and passed along through a climate skeptic web-list
Deltoid even confirmed this years ago.
http://scienceblogs.com/deltoid/2005/05/bray.php

I’d prefer see a survey of the form that allowed progressive details on each issue, i.e.:
1. Is atmospheric CO2 increasing beyond know norms?
2. If yes, is the increased caused by human activies?
3. If yes, is the increase causing a minor amount of warming?
4. If yes, is the increase causing a major amount of warming?
Same format could be used for other issues.

Being the good, agnostic skeptic that I am, one wonders who funded the study, and who had a hand in the development of the questions.
Skepticism is a two-way street.

Tom Fuller says:
September 27, 2010 at 12:09 pm
—————–
Tom Fuller,
To paraphrase a line from John Masefield’s poem “Sea Fever”;
. . . and all I ask [for] is a strong body of independent thinkers and a star [for them] to steer climate science by . . . .
John

Tom Fuller says:
September 26, 2010 at 9:06 pm
Robert, you do understand the difference between 2003 and 2008, right? If so, let’s continue. Despite the fact that Deltoid is passionately attacking a different survey, and despite the fact that Deltoid and its administrator, Tim Lambert, would cheerfully call white black if it helped spread alarmist fever, let’s continue.
This survey only allowed one response per emailed invitation. Repeat responses over-wrote the previous version. Invitations were emailed to a pre-selected sample of climate scientists.
Lambert (as usual) is wrong in insisting that his interpretation of the results is the only way to look at them. It is not, and his chosen method is not even the standard.
So, apart from this being the wrong survey, with no relevance to Lambert’s expose, is there anything further you’d like to discuss?
_____________________________________________________________
We know this was an ONLINE survey hosted on a website. We do not know EXPLICITLY how the verification process was itself verified, meaning how a completed ONLINE survey was traced back to a valid email/IP address (e. g. the actual people on the original email list). Typing in a valid email address into the ONLINE survey is NOT verification of the actual person answering the ONLINE survey.
Until sometime that the verification process can be ascertained, which SHOULD be part of any background writeup section of the report(s)/paper(s), we are not certain who answered the ONLINE survey.
We don’t, for example, know if the emails sent were forwarded to others, and that those others (and subsequently other others), if fact filled out the survey questionnaire.
This will be a question I ask Bray and von Storch myself to clear up this matter for my own peace of mind.
As to the ad hominem of Deltiod/Tim Lambers, meh, BAU, but they do have a valid criticism of at least one of the two previous surveys, calling into question a comparison between the 2nd with the 1st and 3rd surveys.
Here I am at WUWT displaying some (or a lot) of skepticism with respect to one part of the survey (although I’ve also shown that I do have a lot of skepticism with respect to other aspects of the survey, primilarily the statistical confidence one can extract accurately from these time of choice lists/response choice lists types of surveys and the demographics question), how it was verified.
Why am I THE AUDITOR while TEAM AUDIT has been mostly silent? Hypocrisy? Isn’t TEAM AUDIT supposed to be skeptical with all aspects of climate science? Or only when one thinks something conforms to their own worldviews, also known as confirmation bias.

Re: My 09/26/2010 7:13 PM comment
Whoops!! The last paragraph should read:
I would consider the following ranges of normalized deviation wrt 1.346 to be reasonable for classifying the degree of consensus:
• Very low: greater than 1.5
• Low: 0.5 to 1.5
• Medium: 0.5 to -0.5
• High: -0.5 to -1.5
• Very high: Less than -2.5
Sorry about that …

In my 9/26/2010 7.13 PM comment I said that I did not find the bucket graphs In the BS article to be very helpful in understanding the degree of consensus among the respondees. I then described my preferred methodology wherein I 1) normalized the STDEV of the responses to the questions and then 2) rank-ordered the results. The average of the STDEVs is 1.346 and the STDEV about that average is 0.167. I then stated that I considered the following ranges of normalized deviation wrt 1.346 to be reasonable for classifying the degree of consensus (DC):
• Very low: greater than 1.5
• Low: 0.5 to 1.5
• Medium: 0.5 to -0.5
• High: -0.5 to -1.5
• Very high: Less than -1.5
Applying the same line of reasoning to the LS average scores for the ~ 100 questions yields an average score of 4.184 and a STDEV of 0.956. Since most of the LS scores measure the agreement wrt a specific question, I would consider the following ranges of normalized deviation wrt 0.956 to be reasonable for classifying the degree of agreement (DA):
• Very low: less than – 1.5
• Low: -1.5 to-0.5
• Medium: -0.5 to 0.5
• High: 0.5 to 1.5
• Very high: Greater than 1.5
Based on this classification scheme, only 13 of the 100 questions have a high or very high degree of agreement and a high or very high degree of consensus. Only Q56 How convinced are you that most of recent or near future climate change is, or will be, a result of anthropogenic causes? (not at all 1 2 3 4 5 2 3 4 5 6 7 very much) has both have a very high degree of agreement (DA = 2.365) and a very high degree of consensus (DC = -2.204).
Interestingly, 6 of the 9 questions in the Adaptation and Mitigation section have high degrees of agreement and consensus:
Q70 In making policy decisions about adaptation to climate change, priority should be given to opinions of industry and commerce 1 2 3 4 5 6 7 scientific expertise. (DA = 1.199, DC = -1.551)
Q71 In making policy decisions about adaptation to climate change, priority should be given to political opinion 1 2 3 4 5 6 7 scientific expertise. (DA = 1.410, DC = -1.204)
Q72 In making policy decisions about adaptation to climate change, priority should be given to
Public opinion 1 2 3 4 5 6 7 scientific expertise. (DA = 1.335, DC = -1.019)
Q73 In making policy decisions about mitigation to climate change, priority should be given to
opinions of industry and commerce 1 2 3 4 5 6 7 scientific expertise. (DA = 1.324, DC = -0.981)
Q74 In making policy decisions about mitigation to climate change, priority should be given to
political opinion 1 2 3 4 5 6 7 scientific expertise. (DA = 1.398, DC = -0.865)
Q75 In making policy decisions about mitigation to climate change, priority should be given to
public opinion 1 2 3 4 5 6 7 scientific expertise. (DA = 1.436, DC = -1.122)
Clearly, the respondees believe that scientific expertise should trump opinions of industry and commerce, political and public opinions when it comes to policy decisions about mitigation and adaptation to climate change. My, my …
For the remaining 6 of 13 questions having a high or very high degree of agreement and a high or very high degree of consensus.
Q 32: DA = 0.794 and DC = -0.609
Q 59: DA = 1.128 and DC = -0.659
Q 91: DA = 1.029 and DC = -0.565
Q 107: DA = 1.470 and DC = -1.066
Q 109: DA = 1.265 and DC = -1.465
Q122: DA = 1.537 and DC = -0.713

Ah ok it’s on page 32. Nevermind. Isn’t it something suspicious to be only taking the top two responses in calculating that 32% figure?