Guest post by A. Scott
There has been considerable discussion about the methodology and data regarding the recent paper “Lewandowsky, S., Oberauer, K., & Gignac, C. E. (in press). NASA faked the moon landing – therefore (climate) science is a hoax: An anatomy of the motivated rejection of science” (copy here)
This allegedly peer reviewed paper claims their survey data show climate skeptics are supporters of wild conspiracy theories, such as “NASA faked the moon landing.” The author admits, however, no climate skeptic sites were involved in the survey, that essentially all survey results were obtained thru posting the survey on pro-global warming sites.
Due to the serious and legitimate questions raised, I have recreated the Lewendowsky Survey in an attempt to replicate and create a more robust set of replies, including from skeptic users.
Please click on the Lewandosky Survey Page above and you’ll be presented the survey. This survey replicates the questions, both the paper, and several sites have indicated were in the original survey, including those questions deleted from the survey results.
The only change was to use a 1 to 5 ranking vs. Lewandowsky’s 1 to 4, which several people with experience have noted should improve the overall responses.
Each visit to the survey is tracked. Access is password protected for an additional layer of tracking.
THE PASSWORD FOR THE SURVEY IS “REPLICATE” (case sensitive)
Please only complete and submit once. Also, please respond to each question with the answer that best reflects your position, even though the question may not be perfectly worded.
This survey is built on the Google Doc’s open access platform. Results are collected automatically. As no significant randomization or counterbalancing was performed on the original survey none is applied here. Data collected will be provided upon request.
A. Scott
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
When behind the big firewall no address with the word wordpress or blogspot or any other common blog server is reachable.
This way the Chinese government made easy for themselves to remove all free opinions from the chinese people and visitors.
I’m waiting for the DownFall version.
JF
I emailed Lewandosky and pointed out to him that people who had actually BEEN to the moon were skeptics.
Oddly, he didn’t reply…
This poll told me more about Lewandowsky than myself, his left wing eco loony bias really shines through…
Not academic at all (more like the work of a Year 1 grad student) and as an academic I’d like to know how the hell his work got through peer review? I suspect pal review had a lot to do with this and folk should be asking his university to investigate the whole process as it must be calling into question that institute’s integrity
No mention of Kurt Cobain or Courtney Love? Forget it.
Seriously… a person who would create that original survey is mentally unbalanced and probably should not be allowed out on their own. Is it possible to actually believe that someone who disagrees with the discredited hypothesis of catastrophic anthropogenic global warming must, by definition, be an idiot?
Ironic… the very people who think skeptics are part of a conspiracy are crying about conspiracy theories. Projection, anyone?
Rolf says:
September 9, 2012 at 1:52 am
“When behind the big firewall no address with the word wordpress or blogspot or any other common blog server is reachable. ”
Try to find out the IP address of the server you want to contact and replace the server name with the IP adress. This way you bypass the DNS system and get to the exact same server.
There could be a genuine reason why the original questionnaire used four answers rather than five and to change this here invalidates any comparison that is intended.
For example: “There is no clear consensus on the number of options that should be given in an closed format question. Obviously, there needs to be sufficient choices to fully cover the range of answers but not so many that the distinction between them becomes blurred. Usually this translates into five to ten possible answers per questions. For questions that measure a single variable or opinion, such as ease of use or liability, over a complete range (easy to difficult, like to dislike), conventional wisdom says that there should be an odd number of alternatives. This allows a neutral or no opinion response. Other schools of thought contend that an even number of choices is best because it forces the respondent to get off the fence. This may induce the some inaccuracies for often the respondent may actually have no opinion. However, it is equally arguable that the neutral answer is over utilized, especially by bored questionnaire takers. For larger questionnaires that test opinions on a very large number of items, such as a music test, it may be best to use an even number of choices to prevent large numbers of no-thought neutral answers”
http://csdl.ics.hawaii.edu/techreports/05-06/doc/Stasko.html
There is clearly debate as to whether odd or even numbers are best, but once one has been selected, it should be stuck with.
Good idea.
More to the point though, surely Lewandosky should be doing exactly the same thing and throwing the old data out, if he is genuinely interested in the truth.
Thank you all for continuing to take this survey. The response, in numbers and apparent quality, has been gratifying, with folks from across the globe already participating in this project in th few short hours it’s been up.
Additionally, a large share of those responding have taken the time to leave comments. I have read each one, and tried to correct the things you pointed out that I could. As we progress, we’ll try to compile these comments and present as constructive input to the discussion. Rest assured we will not publicly associate comments with posters.
Perhaps more surprisingly, the majority of respondents have voluntarily left contact information including emails, which I think pays strong testament to the validity and quality of the responses.
So far the trusty Google Doc’s “Form” platform also seems to be performing flawlessly, as it appears so too does the visitor tracking.
To those who have pointed out the difficulty in answering some of the questions, but have indicated they persevered and gave the best answers they could – a big thank you. I have a feeling that effort will pay off well, as more responses helps in effect “smooth” the data – helping show a more accurate overall response to those hard questions.
Some of questions are sure funny.
“The claim that sky is pink with green dots is a hoax perpetrated by long-nosed yiddish octopi from Mars”
Sometimes, I’m not even sure what “true” and “false” means in the context of the question.
I found that quite curious, as have many respondents.
Considering the original author, Mr. Lewandowsky is not, to my knowledge, from the US – the US centricity of the “conspiracy” questions was quite interesting.In a roundabout way the questions included – IMO from looking at the original survey data – have actually created a good “control” for the validity of the data.
James – an excellent point, and a clearly identified issue with the original survey. I had the same concerns and gave a lot of thought.
While it is impossible to eliminate these kind of childish games you can make effort to address. Tracking and cross-checks help some. A few “controls” help some more.
But the biggest “hammer” in the toolbox is number of responses IMO.
The more responses the better “baseline” or profile can be created. Won’t share details, but answers can be checked against the majority of responses for similar respondents, and true manipulative responses can be I think pretty clearly identified.
More importantly however, with a large number of legitimate responses, the small number of these attempts to manipulate will have little to no effect as they’ll be so diluted as to not really “move the needle.”
Based on the response in the few short hours the survey has been up – looking at indicators of quality, and number of responses – it appears we’ll have a very strong baseline set … a good “control” to work from.
What a blunder L. made not to run the survey on Above Top Secret, etc. That way he wouldn’t have tipped his hand, and there’d have been less gaming of answers–at least less in a way that would affect climate-related matters, because it wouldn’t have been apparent that that was his focus.
Nick in vancouver says:
September 8, 2012 at 7:41 pm
The original survey is bogus and a stooge, do not complete this version as it gives credibility to the unmitigated bias of first laughable piece of junk.
Here’s a better survey question.
What qualifies one to be an Australian Professor of Psychology?
We can have more fun and learn more by
coming up with 4 alternative answers.
**********************************************************************************
Answer: Only one possible answer.
Not being clever enough to do anything worthwhile, meaningful or valuable for the human race.
Steve T
Lazy Teenager, in Australia, states: “Many of you guys visit pro warming sites so presumably you entered the survey. Especially since you aren’t shy about gaming online surveys.”
The ‘Lewandowsky word’ is “presumably”. It means in this context, I don’t know but I’m going to insult you anyway. His comment is interesting because in his mind the reverse should also be “presumably” what he believes. Therefore the only conclusion Lazy Teenager could reach is that the results of the survey are meaningless.
However why did Lewandowsky fail to include another major conspiracy theory in his survey? For instance on January 23rd this year Cook and Lewandowsky wrote in relation to a book they had had published:
“A Muslim forum speculated that it “Should be useful when engaging people who believe lies about Islam”.
http://www.shapingtomorrowsworld.org/Debunking-Handbook-update-feedback.html
One can only “speculate” why Lewandowsky chose not to ask a conspiracy theory question about Islam.
The Dan Rather School of Accuracy has more members than one might think. Lewandowsky evidently skipped the entire freshman class including the basic intro class “151 – Found It In The Mail” and “182 How To Fake A Survey Using Math” and went directly to postgrad “502 Headlines For Fake but Accurate”,
The conflations show a constant pattern. Lewandowsky is so completely locked into fashionable thinking that he’s unable to write an open question. He can’t break out of his idiot assumptions about “sustainability”.
The choice of conspiracy theories also shows the same bubble. Most of them are the theories of left-wingers. Example: the question about McVeigh assumes that the conspiracists would associate McVeigh with neoNazis. Right-wing conspiracists would associate McVeigh with Arabs and FBI. In fact both associations are true.
BTW, someone should check out the positions taken by and affiliations of Dr. L’s co-authors, Oberauer, K., & Gignac, C. E.
No, no, Anthony. Don’t be contaminated by bad methodology. Say what you will about Australian sociology research then drop it.
Oh come on. Are we seriously going to promote this ‘study’ to the rank of science by trying to replicate it?
Didn’t the title “NASA faked the moon landing – therefore (climate) science is a hoax: An anatomy of the motivated rejection of science” give the game away.
For anyone who has filled in the ‘survey’ – where is your self respect?
Free market is not answer to all issues, In India there is free market, do you want that US would be the same? Still AGW is HOAX ! Where poor people need police or Army? Wealthy peoples need them to protect their welthiness so it’s right that they need to pay more taxes than poor people. It was those free market’s wich brought us this economic crisis. You should try to understand what Johnn Rawls really said in Theory Of Justice.
“This allegedly peer reviewed paper claims their survey data show climate skeptics are supporters of wild conspiracy theories, such as “NASA faked the moon landing.” ”
When the statement recruiting people to answer a survey tells them what hypothesis the author wants to disprove, the resulting data is going to have zero validity whether ten or ten thousand people answer.
I wonder how many other similarly goofy articles Psychological Science has published–especially regarding climate science. Worth researching.
We have a new word for junk scientific papers, Lewpaper.
A new Lewpaper… A Lewpaper using models…
Strange as I get older I am finding that the difference between intelligence and stupidity is wafer thin.
I took the survey after putting on my tin-foil hat so the aliens couldn’t manipulate my thoughts! 🙂