Replication of Lewandowsky Survey

Guest post by A. Scott

There has been considerable discussion about the methodology and data regarding the recent paper “Lewandowsky, S., Oberauer, K., & Gignac, C. E. (in press). NASA faked the moon landing – therefore (climate) science is a hoax: An anatomy of the motivated rejection of science” (copy here)

This allegedly peer reviewed paper claims their survey data show climate skeptics are supporters of wild conspiracy theories, such as “NASA faked the moon landing.” The author admits, however, no climate skeptic sites were involved in the survey, that essentially all survey results were obtained thru posting the survey on pro-global warming sites. 

Due to the serious and legitimate questions raised, I have recreated the Lewendowsky Survey in an attempt to replicate and create a more robust set of replies, including from skeptic users.

Please click on the Lewandosky Survey Page above and you’ll be presented the survey. This survey replicates the questions, both the paper, and several sites have indicated were in the original survey, including those questions deleted from the survey results.

The only change was to use a 1 to 5 ranking vs. Lewandowsky’s 1 to 4, which several people with experience have noted should improve the overall responses.

Each visit to the survey is tracked. Access is password protected for an additional layer of tracking.

THE PASSWORD FOR THE SURVEY IS “REPLICATE” (case sensitive)

Please only complete and submit once. Also, please respond to each question with the answer that best reflects your position, even though the question may not be perfectly worded.

This survey is built on the Google Doc’s open access platform. Results are collected automatically. As no significant randomization or counterbalancing was performed on the original survey none is applied here. Data collected will be provided upon request.

A. Scott

0 0 votes
Article Rating

Discover more from Watts Up With That?

Subscribe to get the latest posts sent to your email.

260 Comments
Inline Feedbacks
View all comments
highflight56433
September 9, 2012 10:40 am

“The State kills citizens and congress and the president just ‘let it slide.’”
The “state” let’s it slide because the public is apathetic, uninformed, misled, suffer from “herd” management, are divided by political propagandists, fall victim to “the lies become the truth,” are subjects of a total brainwashing education system, suppressed (voting, Ruby Ridge, Waco) and oppressed (obstructive regulation, IRS, EPA).
Example of insidious condescension toward the conservatives is in the following Google search; however the text in the Google search has been scrubbed from the Wikipedia site:
Voter suppression – Wikipedia, the free encyclopedia
en.wikipedia.org/wiki/Voter_suppression
In the United States, voter suppression was used extensively by Democratic party operatives in most Southern states until the Voting Rights Act (1965) made …
It is the ongoing tactic to rewrite history in favor of lefty Marxist domination to promote the progressive agenda. Just like the survey of discussion. Is it a conspiracy? No, it is the agenda of those who promote their ideological progressive beliefs. That’s what they do.

Bob
September 9, 2012 10:41 am

Here’s my partial summery of the questions on the survey.

5. Free and unregulated markets pose important threats to sustainable development *
6. The free-market system is likely to promote unsustainable consumption *

Nothing is ultimately sustainable. Ignorant questions.
8. A powerful and secretive group known as the New World Order are planning to eventually rule the world through an autonomous world government, which would replace sovereign governments
It is the space aliens who will run things, and they are not here in force, yet. Everybody knows that.
11. US Agencies intentionally created the AIDS epidemic and administered it to Black and gay men in the 1970′s *
This is a popular meme of the Democrat Party, especially as espoused by that brilliant scientist, Van Jones.
16. The US government allowed the 9/11 attacks to take place so that it would have an excuse to achieve foreign (e.g., wars in Afghanistan and Iraq) and domestic (e.g., attacks on civil liberties) goals that had been determined prior to the attacks *
This means that it was Bush’s fault. Who could believe that?
20. The claim that the climate is changing due to emissions from fossil fuels is a hoax perpetrated by corrupt scientists who wish to spend more taxpayer money on climate “research”.
Who is kidding whom, here? DUH! Allow me to follow the money.

22. I believe that burning fossil fuels increases atmospheric temperature to some measurable degree
23. I believe that the burning of fossil fuels on the scale observed over the last 50 years has increased atmospheric temperature to an appreciable degree *

What does the word appreciable mean? How can I measure “appreciable”? Questions not thought out very well. Sloppy work.

24. I believe that the burning of fossil fuels on the scale observed over the last 50 years will cause serious negative changes to the planet’s climate, unless there is a substantial switch to non-CO2 emitting energy sources *
25. I believe that the burning of fossil fuels on the scale observed over the last 50 years has caused serious negative changes to the planet’s climate *

Past tense vs future tense. Sloppy work.
38. Out of 100 climate scientists how many do you think believe that human CO2 emissions cause climate change? *
It doesn’t matter.

Editor
September 9, 2012 10:41 am

The more I think about it, the more I think that there is a deep structural flaw in the basis of the study design.
It is structured largely to force you into choosing WHICH “cliff of conclusion” from which to leap; when IMHO the basic character that sets a skeptic apart from a ‘warming believer’ is the very RELUCTANCE to leap off any cliff of conclusion.
So, for example, on the “Area 51” question: You are asked to choose “space alien studies” or not. Yet it is a secret site where by definition only the folks who work there will know. It is highly likely there are other life forms in space somewhere. To assert either that they have never come here or that they are here and being studied both are leaping off a ‘cliff of conclusion’ for which there is not sufficient evidence.
For the Kennedy Assassination question: You are asked to choose ‘grand conspiracy’ or ‘Warren Report single shooter’; yet there is a lot of evidence for what I’d call the “small conspiracy” approach ( things like recorded history of an attempt in Florida and Mafia involvement, and that the Kennedy clan had a history of conflict with the Mafia reaching all the way back to the Saint Valentines Day Massacre era of rum running, not to mention the Bay Of Pigs when John F. had made promises to the Mafia that he blew off… a bad idea…). Is it ENOUGH evidence? No, not at all. Suggestive at best. But the same can be said of the “Single Shooter Oswald” conclusion. The “correct” assessment is “We don’t know, likely can’t know, but either answer is an insufficiently supported ‘leap'”… THAT is the “skeptic” answer, not choosing one wrong answer over another. So I chose the “slightly agree” with the conspiracy theory, because there are more loose ends to the Single Shooter than to the “small conspiracy with Mafia help” choices; BUT what I really wanted was a “We do not know” or a “strength” metric. I.e. I’d like to have been able to choose “conspiracy” but with a “strength” of 2 (just above weak) out of 5, AND marked the “single shooter” as 1 (weak) out of 5 strength as well. BOTH are possible, but a bit weak and with poor evidence.
So to me, the survey is designed to force a “decline to accept dogma” to show up as a “whacky theory advocate” when the reality OUGHT to be “decline to accept insufficient proof for dogma AND decline to reject ‘whacky theory’ without sufficient proof”. That is, an ‘open but cautious mind’ very aware of when there is simply insufficient information to support an idea, or reject one.
So you may be able to ‘replicate’, but it still isn’t going to say much about the actual “skeptic” axis, as that axis is clouded by the question structure.

Oscar Bajner
September 9, 2012 11:07 am

I just completed the “survey”. I naturally used the same name and email as I use here. So I would also be interested in Mr Scott’s results for folks who used contactable details.
As for the survey itself, LOL. I was in Area 51 last week, and I know for a fact there are no aliens there, remember folks, the aliens have rule number one – always retrieve your wounded.
And the moon landings were not faked in Hollywood, but in Burbank. /LOL

3x2
September 9, 2012 11:12 am

It would be more interesting to survey his department …
(1) Energy in the 15u band is mainly held ..
a) In the rotational and vibrational modes of the CO2 molecule
b) In the words and works of Dr Mann
c) Only in renewable forms of energy and sustainable living
d) In a piss poor internet survey masquerading as science
e) You are a denier funded by a big fuel conspiracy and Agenda 21 is the way forward
f) Do you believe in conspiracies? The clay model I made of you does!
g) I lost the plot a long time ago – what was the question again?

G. Karst
September 9, 2012 11:23 am

It is the peer review system that is failing. Anyone is free to conduct any survey they want, in order to make hay. That doesn’t mean scientific journals should publish their findings as scientific hay. GK

September 9, 2012 11:30 am

Reblogged this on Climate Ponderings and commented:
Better yet – would be to survey Lewandowsky et al…IT IS THEIR INTERPRETATIONS of the Questions that drive the results!!!

Pamela Gray
September 9, 2012 11:58 am

I deal with standardized surveys all the time (what special educator hasn’t?). The scaled response usually includes a “not observed” or “not applicable”, or “do not know” category to allow respondents to state that the question’s focus is new to them, they have no knowledge of its focus, or does not apply to them.
As for the Kennedy assassination, modern ballistics, including trajectories and deflections through various materials demonstrate that it is equally possible that all injuries were caused by a single assassin from a single location. As for multiple assassins, they would have to be located very near each other (as in nearly on top of each other) for that theory to be a possible cause of all injuries. That does not prove the assassin was the only one who knew (IE acted alone) what he was going to do. Criminal research demonstrates it is relatively rare for a killer to plan for a political assassination (versus a mentallly deranged killing) without others having prior knowledge of what he was going to do. While I tend to lean towards a single shooter, I think the commission came to the “acting alone” conclusion without benefit of a thorough investigation. Of course, back then, no one had heard the term “sleeper cell” before.

Skiphil
September 9, 2012 12:19 pm

The “forced choice” approach to survey questions (allowing no neutral or “do not know” option) is a conscious choice by Lewandowsky et al to polarize the answers. Yet, for many of the conspiracy questions “do not know” is the rigorous rational answer since the respondent has no genuine evidence at all. However, it is also rational to reject outlandish conspiracy theories automatically UNLESS one has credible evidence to consider one. I reject virtually all conspiracy theories out of hand, but I am now willing to entertain a theory that Stephan Lewandowsky et al set out to provide evidence that CAGW alarmists are delusional dishonest morons. I admit that Lewandowski and his pals are a tiny sample, but they are more representative of their ilk than anonymous web survey respondents are representative of anything at all.

DirkH
September 9, 2012 12:25 pm

Bob says:
September 9, 2012 at 10:41 am

“6. The free-market system is likely to promote unsustainable consumption *
Nothing is ultimately sustainable. Ignorant questions.”

In a free market system, a resource becomes more expensive as it gets scarcer – this leads to less consumption and replacement of the scarce resource with an abundant one. Therefore, a free market system cannot lead to unsustainable consumption. The Free market system is demonized by the left as an ever growing monster, but that is not the definition; wealth and consumption can shrink when resource scarcity forces adaptation.
See Julian Simon, The Ultimate Resource
http://www.juliansimon.com/writings/Ultimate_Resource/

phlogiston
September 9, 2012 12:27 pm

Here are some questions that should be asked in a survey of believers in global warming:
1. A single stick breaks easily but a bundle of sticks (Italian fascies) bound together is much stronger, true / false?
2. A climate skeptic obstructing action on global warming should be treated as an enemy of the people, true / false?
3. All people are equal but some are more equal than others, true / false?

DirkH
September 9, 2012 12:28 pm

3×2 says:
September 9, 2012 at 11:12 am

It would be more interesting to survey his department …
(1) Energy in the 15u band is mainly held ..

That terminology would be a bit too complicated for a social scientist.

September 9, 2012 12:30 pm

@A.Scott 9/8 10:24 pm reply to Skiphil: may well actually serve to mitigate the poor quality of the questions etc.
You are fooling yourself if you believe this. More answers to poor questions does not add knowledge.
You have strong standing to show that the survey results are not replicable and therefore their statistics are meaningless – a valid sample was not taken can adequately represent the population. Stick with that.
I repeat my suggestion to repeat the survey questions but with responses that only evaluate the quality of the question.

Gary
September 9, 2012 1:14 pm

Sorry, I just can’t respond to such poorly constructed questions. Too many of them conflate ideas making it impossible to answer accurately. Any results tell you nothing except some people are willing to take a survey, no matter how bogus.

September 9, 2012 1:19 pm

I originally tried to take the survey on one of the believer’s sites. I abandoned it partway through, thinking it was a childish parody, badly done satire, not a genuine survey.
I’ve answered it now, but note that my ‘lukewarmer’ position might well be misinterpreted as not being skeptical. I suspect that if Lewandowski had crafted a survey that adequately explored the opinions of his respondants, he would think I’m the archetypical ‘climate denier’.
Many of my responses on another day would differ from those I gave today, because today I guessed at the meaning of some questions. Another day, my guess might be different, and therefore my answers would also differ.
For example, the question about affecting the quality of the environment. If he meant we should leave the world as untouched wilderness, I strongly disagree. If he meant to ask about making areas unfit for life, that’s a completely different answer. Which did he actually mean? There’s no way to tell. There were similar difficulties with many of his ambiguous questions.

davidmhoffer
September 9, 2012 1:21 pm

I’d like to see a second survey but properly worded questions and with a heavy hand toward basic science literacy. Questions like:
What does “logarithmic” mean?
Are the effects of CO2 logarithmic?
What is the commonly accepted baseline CO2 concentration?
What is the CO2 concentration today?
How much additional CO2 would we have to add to the atmosphere to equal the effects of the change from baseline to current?
How much has CO2 concentration increased in the last 10 years?
Based on the change rate of the last 10 years, how long will it take to equal the effects of the change from baseline until now?
The reason I suggest questions such as these as a lot of people, skeptics and alarmists alike, will get the answers wrong. Who will do worse? That’s not the point of these questions. The point of these questions is to educate. When people who come back to see what the right answers were, they get an education in physics in the process, and rather simple physics at that, not something anyone can dispute without looking like an utter fool. In fact, not something that anyone can dispute without being in disagreement with the worst that the IPCC reports have to say on the matter. As far as I am concerned, the fact that CO2 is logarithmic should have been the end of the discussion in the first place. The alarmists twist and turn to avoid that discussion and keep the issue off the table at all costs. Time to put it back front and centre and a well thought our survey would be a great way (I think anyway) to educate a lot of people on this simple and obvious fact of physics that largely negates the while catastrophic meme in the first place.

3x2
September 9, 2012 1:25 pm

DirkH says:
September 9, 2012 at 12:28 pm
3×2 says:
September 9, 2012 at 11:12 am
It would be more interesting to survey his department …
(1) Energy in the 15u band is mainly held ..
That terminology would be a bit too complicated for a social scientist.

Then perhaps they should stay out of real science and get back into the wonderful world of politics and PR.
Then again … we have this …

I simply can’t believe the coverage this muppet has had in a supposedly ‘science based’ blogsphere. What is wrong with you people? Slow news week? The guy is a certified fruit loop obsessed by conspiracies against the lovingly crafted sustainability utopia he wishes you all to live in. PROJECTION much. Should never have left the ivory tower – thar be sharks in them there waters Prof.

theduke
September 9, 2012 1:52 pm

The Iraq War was launched because Saddam Hussein was potentially the world’s most dangerous terrorist, given his unlimited financial resources and a diplomatic corps that was little more than a terrorist planning organization that could operate with near impunity around the world. He also had a maniacal enmity toward the United States which refused to allow him to win the war with Iran and then decimated his military in the Gulf War. Nearly every intelligence organization in the world believed he had WMD, and Saddam conducted misinformation campaigns that were designed to convince all other nations that he did.
Next question.

TinyCO2
September 9, 2012 1:53 pm

Given that Tony Blair has publicly admitted he would have invaded Iraq even without evidence of weapons of mass destruction and would have found a way to justify the war to parliament and the public, is that a conspiracy theory?
http://www.guardian.co.uk/uk/2009/dec/12/tony-blair-iraq-chilcot-inquiry

u.k.(us)
September 9, 2012 2:17 pm

theduke says:
September 9, 2012 at 1:52 pm
The Iraq War was launched because Saddam Hussein was potentially the world’s most dangerous terrorist, given his unlimited financial resources and a diplomatic corps that was little more than a terrorist planning organization that could operate with near impunity around the world. He also had a maniacal enmity toward the United States which refused to allow him to win the war with Iran and then decimated his military in the Gulf War. Nearly every intelligence organization in the world believed he had WMD, and Saddam conducted misinformation campaigns that were designed to convince all other nations that he did.
Next question.
——————————————
Ever wonder why a crediable threat was never given to the U.S.
Smoking holes tell their own story.

Richdo
September 9, 2012 2:22 pm

I took the survey and did the best I could to answer the questions with an honest opinion, though as others have well noted some were catch-22. But then it’s opinion right? You know, “opinions are like A$$ holes…everybody has one.”
The ET questions did prompt a thought though; has anybody else noticed that the reports of alien abductions (not to mention spontaneous human combustion) seem to have declined over recent years as the CAGW rhetoric has increased? Perhaps the crazies have found something new to go on about.

Malcolm Chapman
September 9, 2012 2:34 pm

This may already have been fixed, but:
This survey replicates the questions, both the paper, and several sites have indicated were in the original survey, including those questions deleted from the survey results.
Something missing there. Otherwise, nice post.

September 9, 2012 2:46 pm

Is there any way of knowing which responses are genuine climate skeptics?
Lurid Off Key clearly hates the thought of real science, real truth, real evidence. Why else would he try to imprison us in his spider’s web that (1) is constructed around nothing but irrelevant politically incorrect beliefs, and (2) doesn’t even begin to touch the real climate skeptics’ issues?

Bob
September 9, 2012 3:13 pm

Dirk H said:
” In a free market system, a resource becomes more expensive as it gets scarcer – this leads to less consumption and replacement of the scarce resource with an abundant one. Therefore, a free market system cannot lead to unsustainable consumption. ”
I agree. At the least, you have to pump in resources and energy to keep any process going. Solar cells fail. Windmills wear out. There is always a cost. Physical systems have the property that they will eventually become unstainable.

Francisco
September 9, 2012 3:15 pm

The Lewandowsky paper is remarkably clownish. It’s amazing the author has such an atrophied sense of ridicule.
I am thinking a link to this Lewandowsky paper should be added to the famous site that compiles “A complete list of things caused by global warming:” http://www.numberwatch.co.uk/warmlist.htm
Maybe it could go in the letter R (rejection of science) right between the links for “reindeer larger” and “release of ancient frozen virus”.