Guest post by A. Scott
There has been considerable discussion about the methodology and data regarding the recent paper “Lewandowsky, S., Oberauer, K., & Gignac, C. E. (in press). NASA faked the moon landing – therefore (climate) science is a hoax: An anatomy of the motivated rejection of science” (copy here)
This allegedly peer reviewed paper claims their survey data show climate skeptics are supporters of wild conspiracy theories, such as “NASA faked the moon landing.” The author admits, however, no climate skeptic sites were involved in the survey, that essentially all survey results were obtained thru posting the survey on pro-global warming sites.
Due to the serious and legitimate questions raised, I have recreated the Lewendowsky Survey in an attempt to replicate and create a more robust set of replies, including from skeptic users.
Please click on the Lewandosky Survey Page above and you’ll be presented the survey. This survey replicates the questions, both the paper, and several sites have indicated were in the original survey, including those questions deleted from the survey results.
The only change was to use a 1 to 5 ranking vs. Lewandowsky’s 1 to 4, which several people with experience have noted should improve the overall responses.
Each visit to the survey is tracked. Access is password protected for an additional layer of tracking.
THE PASSWORD FOR THE SURVEY IS “REPLICATE” (case sensitive)
Please only complete and submit once. Also, please respond to each question with the answer that best reflects your position, even though the question may not be perfectly worded.
This survey is built on the Google Doc’s open access platform. Results are collected automatically. As no significant randomization or counterbalancing was performed on the original survey none is applied here. Data collected will be provided upon request.
A. Scott
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
When the phone rings at dinner time, more likely than not the party at the other end wants to survey me for something or another. I regarded this intrusion as an affront to my privacy. Over the years I have learned to regard such calls as entertainment; pure and simple. Issues that may matter to them, just don’t interest me. So, I am free to provide “off the cuff” responses; frequently with a glint in my eye, to deceive, mislead.
This survey is of the same character, and I regard it as same; treat it as same; fluff.
Next question.
There should always be an “I don’t know” choice.
Frankly, that is the point when it comes to climatology.
We don’t know enough to justify any action.
Same thing happened to me until I re-read the instructions. Password is CASE SENSITIVE and the password is REPLICATE. GK
Make no mistake. This will be used against you. Have you learned nothing? Sorry Anthony.
I wonder if Chris Mooney will refer to this survey in some later updated edition of “The Republican Brain”
David Ball … first, I’d note this is my project – Anthony was simply kind enough to allow me to get the word out. And for what its worth the reach of WUWT has been proven again to be worldwide and huge. The response from that single post has been phenomenal.
2nd, plenty of interesting and useful things going on with this project – more than I envisioned when I began it.
It started as a response to those typical taunts we all hear – ‘if you don’t like it, the questions are available, go do it yourself.’ This time I said – ok … I will ….
I hadn’t fully thought about much other than that I wanted to re-create the survey and build a more robust, transparent data set which attempted to avoid or minimize the data collection issues raised regarding the original survey. I wanted to try and make sure as broad a base as possible was aware of, and had opportunity, to respond.
Wanted to duplicate as closely as possible the original, so the data collected would be a meaningful comparison. As noted I did struggle over decision re: the single change – to use a 5 point vs 4 point rating. Pretty much all the pro’s I looked at or spoke with said that was significant improvement. That it improved quality a lot without changing the issues regarding question structure, phrasing and the like. Respondents would have to make the same effort to understand poorly constructed questions, we just gave a scale that allowed a more nuanced answer.
I pretty much understood up front we would not see a vast improvement or benefit – as the survey’s problems still existed. What I thought we could do was try to at least help generate broader based and higher quality data, and allow ALL the opportunity to participate. If we stopped right now that goal has been met and exceeded in my opinion.
Along the way though it’s become apparent this project is generating other significant benefits. IIt has spurred review and discussion about the many issues – about quality, confirmation bias, standards and practices, communication practices of the pro-AGW side and much more. It has – again IMO – helped bring the peer review, and publication process directly under the bright lights of public scrutiny.
It also has the potential for a lot more. Several other interesting and potentially valuable experiments and projects have come up along the way – both using the data being collected, and regarding future projects as well. It has also been an education about the potential of crowd-sourcing, showing that simple tools, when coupled with a good plan, can be as, or likely more, effective than high dollar institutional work.
As this project proceeds, and runs its course, I believe that finding will become reinforced and apparent to all.
Then again, maybe the data will show skeptics are full on conspiracy theorists, as Mr. Lewandosky claimed – that they do believe the Moon landing was faked. If that IS the conclusion though, and I wouldn’t hold your breath , it will be based on a robust, broad based, transparent data set, with input from all.
You may worry that is a risk. I have less concern. If that were to happen, as this process and the large base of comments is showing, it would be largely an artifact of the survey and question failings and bias, not a reflection of the real world..
Time will tell. Until then the more responses the better the data. Yes it will still have the flaws as a result of survey problems, but it will also improve the quality of the overall data. And there are many uses for that data, unrelated to coming up with conclusions of the original authors.
That is my, now educated, layman oriented opinion … and I’m sticking with it.
Once again – a big thanks to the large number of participants – while we may not be doing brain surgery with this, we are accomplishing something positive regardless, now – and I think – potentially in the future.
Again – all your efforts and comments are much appreciated.
…It seems clear this “paper” was intended to serve one purpose – to provide the headline and basis for ongoing attack, denigration and derision of those skeptical of AGW – in order to promote the authors well documented agenda.
This is already occurring right on cue – see: How do people reject climate science?
… please check out the author. Big surprise there right? .
I have already seen this “paper” referenced by AGW alarmists in a number of places:
This is EXACTLY the purpose of their flawed work. It was not to do actual research, but rather to create this exact pre-planned talking point. Which will now serve as the “support” and basis for these attacks to continue regardless of the controversy over the flawed results.
In my belief, one of biggest positives of the the whole re-created survey project is that it is continuing to generate awareness, visibility and more important, fueling wide discussion of the serious flaws of Lewandowsky’s work.
It doesn’t simply tell people about the issues it has allowed them to directly EXPERIENCE them first hand. And as the comments show – once you have that direct experience the serious failings becomes extremely obvious.
The authors I’m sure fully expected this issue disappear and fade away, and they’d be able to continue the attack and derision of skeptics long after …
Not gonna happen – not this time. That train has left the station.
So by all means take the re-created survey – see firsthand the quality of the original authors work. Its really even different than reading thru the questions – having to decide on an actual answer is truly enlightening.
.
stupidest survey EVAH
As an old friend used to say to fools, “you think you’re a wag but in fact you’re a %&^%$”. Such is Lewandowsky and his survey. The laughable questions are framed as, “Mr Bloggs is a good man who likes taking things, agree or disagree”. Absolutely hopeless. Are we sure that Lewandowsky isn’t the janitor at UWA?
For supporters of an unregulated free market, regulating polluting industries to reduce global warming is so unpalatable that they are far more likely to reject that climate change is happening.“
What utter nonsense. Governments have been regulating polluting industries for many decades, and its the political issue that probably the broadest public support. Hardly anyone objects to bringing in measures that reduce pollution.
I guess the survey is down. The link in the post above leads only to a page asking a PW input. But as I have no password, it is impossible to continue. No link leads directly to any survey.
Tom Vonk says
Please reread A. Scott’s post carefully once more. You will see that he tells you the password you need. If you cannot do this, the relevant extract is
When filling out the survey just don’t mention that you have a roo culling licence or you will not get any jelly at the lunch that is supplied.
/Sarc
I think that the CAGW debate on the wrmistas side is lost and people like Lew are just outliers that can be ignored. The hard work will be getting government departments like the EPA to start to refocus on true EPA problems.
Given that we know what the survey is asking for, and we know how AGWwrs think, answer the questionnnaire as an AGW believer who also believes in the moon landing hoax, the Kennedy assassination conspiracy, the Princess Di conspiracy etc- thus “proving” that AGW believers are
kook conspiracy nuts- the same as the AGWers did when answering the questionnaire on THEIR blogs- only in reverse
Seems this poll is also a test of following instructions…
Hope there’s an analysis of this poll. S McIntyre’s analysis of the original poll is interesting:
http://climateaudit.org/2012/09/08/lewandowsky-scam/
@ur momisugly 3×2 says:
September 9, 2012 at 1:25 pm
What a bizarre video you linked to! I honestly wondered if it were a parody of the actual person. Amazing, but not in a good way. 🙁
A.Scott
I do not know who you are. I have never seen your name before and A.Scott does not tell me much. Neither does your blog. I have never been a sheep to mindlessly follow others. The alarmist have proven over and over again to be deceptive and use tactics just like this. Even after reading your response to me, something is still not kosher here. Sorry.
I did try to complete it and the ruddy thing disappeared as I came near to the end. I’m not going through that again. I wasn’t very impressed by some of the questions; ticking boxes never allows one the chance to give what one’s real position is anyway. Some of the questions were way-out loopy too; one wonders what sort of mind thinks up some ideas.
I’ll stick to mainline WUWT and JoNova I think; thanks all the same!
“When did you stop beating your wife?”
Puh-leeeze…
Thanks for your help davidmhoffer and G. Karst. It wasn’t a character-case or scrolling issue. I tried a different browser (something I should done in the first place as issues like this is the reason I have different browsers on my hard drive). I typically use Firefox but succeeded in entering the survey with Internet Explorer. My first alternative choice, Chrome, also failed -curious, as the survey is powered by Google Docs. Go figure.
David Ball wrote:
And
I share some of your concerns, David, but A.Scott is a regular contributer to WUWT so you can gauge some of his views/integrity/motivation by reading them.
http://www.google.co.uk/search?hl=en&as_q=&as_epq=a.scott&as_oq=&as_eq=&as_nlo=&as_nhi=&lr=&cr=&as_qdr=all&as_sitesearch=http%3A%2F%2Fwattsupwiththat.com%2F&as_occt=any&safe=off&tbs=&as_filetype=&as_rights=
He has expended some effort on the issue of integrity in science.
The Fakegate Timeline
http://wattsupwiththat.com/2012/02/28/the-fakegate-timeline-from-soup-to-nuts/
I have already expressed my doubts about the value of replicating a fundamentally flawed survey. But, WUWT is, above all else, about airing opposing points of view. People I respect have asked for help, so I gave it. They may see some worth in this endeavour that I do not.
Note: You do not have to provide an email address to participate in the survey. I didn’t.
Incidentally, why the password? To control participation, identify which announcement is used, or keep out spambots?
Mr Scott, can you give us a clue how many responses you have received so far?
Where and when can we see the results?
In order to replicate the previous experiment, should you release preliminary results when there are 1101 answers?
Steve McIntyre has reported on a 3rd and 4th site finding evidence of being contacted – Pilke Jr. and Morano
Coincidentally he also shares email Pileke Jr. rcvd from Lewandowsky indicating his ethics committee says he can release the names. No word on whether they inquired about the well documented quality and issues with the research itself 😉
We also need to see copies of each of the 4 different surveys distributed.