I’m a bit of a latecomer to this affair, as Lucia and Jo Nova took an early lead on pointing out the many problems with the survey methodology (or lack thereof) with the paper:
Lewandowsky, S., Oberauer, K., & Gignac, C. E. (in press). NASA faked the moon landing – therefore (climate) science is a hoax: An anatomy of the motivated rejection of science. Psychological Science.
“Motivated” is the key word here, as it appears there were hidden motivations for this paper. It seems though, once you scratch the surface of Lewandowsky’s paper, that it is nothing more than a journal sanctioned smear of climate skeptics based on not only faulty data, but data gathered with a built in bias.
Besides what we already know about the flawed sampling method, the lack of follow up with skeptic blogs to make sure they got communications inviting them to post the survey, and the early release of results before the survey was complete, the most troubling new revelation appears to be that some climate skeptic blogs got different questionaires than their counterpart AGW advocate blogs. If true (and it appears to be based on the survey numbering system) this negates the study on the basis of inconsistent sampling, and I think it is time to ask Psychological Science editor Robert V. Kail to investigate this paper, and if he finds what the skeptics have, start a retraction. I’ve sent him a courtesy note advising him of this issues with this paper.
Here’s a summary of what has been going on the last couple of days.
Jo Nova has a great summary here, and writes about one Australian investigator who was invited to take the survey questions two years ago, kept screen shots of it, and did an analysis. She wrote:
Graham from OnlineOpinion was so struck by the study he’s written a post titled: Fish rot from the head Part 1.
Read it to get familiar with the survey questions.
Next there’s the who got what version of what survey problem, Jo notes this:
Leopard on the Bishop Hill thread has noted that Steve McIntyre is asking Lewandowsky why there are two or even three different forms of the survey? Why indeed?
Paul follows them up:
The Deltoid, Tamino, Mandia and Hot-Topic blogs were sent the survey number surveyID=HKMKNF_991e2415 on about August 29th. That survey is on the archive, and starts with 6 questions about free markets.
Bickmore and Few Things had the survey number surveyID=HKMKNG_ee191483 also about Aug 29, but this one doesn’t seem to be on the archive.
Steve Mc was sent survey number surveyID=HKMKNI_9a13984 on Sept 6th. This survey is on the archive, and it starts with 5 completely different questions! About how happy you are with life.
This right here should be enough for a retraction from the Journal. If different surveys were sent to different bloggers, and no mention of it was made in the paper or justified in the methodology, then this amounts to purposely biased data from the beginning. UWA may also find grounds for academic misconduct if Lewandowsky purposefully sent different sets of questions based on the type of blog he was inviting.
And then we have the fact that Lewandowsky was discussing preliminary results at a seminar, while the surveys were still open and he had not heard back from the skeptic blogs yet, such as the follow up invitation to Steve McIntyre. Having an open discussion of the survey is highly irregular, because attendees/viewers are free to take the survey, possibly biasing the results.
On the 23rd of September, 2010, Dr. Lewandowsky gave a presentation at Monash university which included the following slide:
Lewandowsky & Gignac (forthcoming)
•Internet survey (N=1100)
•Endorsement of climate conspiracy (“hoax by scientists to get grants”) linked to endorsement of other conspiracies (“NASA faked moon landing”)
•Conspiracy factor without climate item predicts rejection of climate science
So three days after (unsuccessfully) asking for cooperation in fieldwork, Lewandowsky is publicly announcing the preliminary results while the surveys are still open, and he hasn’t heard back from invited distributors. Steve McIntyre of Climate Audit notes that he received a follow up invitation on around Sept 20th. (which he didn’t notice until this story broke). Note the N=1100 value in the preliminary slide. The final paper cites an N<1200 value.
And it seems they are still at it, here’s a recent WUWT comment:
Anthony, there was recently another survey (longer, and with a 1-5 scale) put out by Lewandowsky’s research assistant, Charles Hanich, on June 4, 2012. It seems that the link for this survey was only posted on two blogs: Watching the Deniers and Skeptical Science. Charles Hanich was also responsible for creating Lewandowsky’s 2010 survey, as mentioned in the comments here.
Unfortunately, the link to the June 2012 survey is also unavailable. However, a skeptic called the “Manic Bean Counter” captured all the survey questions and dissected them on his/her blog, here. The following is Manic Bean Counter’s breakdown of the types of questions asked in the survey:
1. Climate Change – 5 questions
2. Genetically Modified Foods – 5 questions
3. Vaccines – Benefits and harms – 5 questions
4. Position of the Conservative / Liberal perspective (US definitions) – 7 questions
5. Select neutral (check of the software, or check for spam?) – 1 questions
6. Free market system v social justice / environment / sustainability – 5 questions
7. Conspiracy theories (political) – 6 questions
8. Conspiracy theories (scientific) – 6 questions
9. Personal Spirituality & Religion – 8 questions
10. Evolution – views upon – 7 questions
11. Corporations – 13 questions
12. Personal emotional outlook – 6 questions
The striking thing is that we have John Cook’s Skeptical Science blog listed as presenting both the original as well as the most recent survey. It as been discovered that Cook is a co-author with Levandowsky on a similar paper:
Lewandowsky, S., Ecker, U. K. H., Seifert, C., Schwarz, N. & Cook, J. (in press). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest.
http://websites.psychology.uwa.edu.au/labs/cogscience/documents/LewandowskyEcker.IP2012.PSPI.pdf
One wonders how much Cook contributed to the questions, based on his understanding of his readers likely responses. It is strange irony indeed that the paper discusses “debiasing”, when so many potential biases in Lewandowsky’s methods are clearly obvious to even the casual reader. Wikipedia even cites them for this paper in a section on “debunker”
Australian Professorial Fellow Stephan Lewandowsky[5] and John Cook, Climate Communication Fellow for the Global Change Institute at the University of Queensland (and author at SkepticalScience.com)[6] both warn about “backfire effects” in their Debunking Handbook.[2] Backfire effects occur when science communicators accidentally reinforce false beliefs by trying to correct them. For instance, a speaker about global warming may end up reinforcing the crowd’s beliefs that global warming is not happening.
Backfire indeed, this Lewandowsky “moon landing” paper is a full force backfire now.
Based on what I’ve seen so far, it is my opinion that Lewandowsky set out to create the survey data he wanted by manipulation of the survey system through multiple undocumented surveys, incomplete and non-representative sampling, biased survey questions, and essentially no quality control. There weren’t even significant safeguards in place to prevent individuals from taking the survey multiple times, appearing as other identities. There are so many things wrong with this paper that I can’t see it surviving intact.
I think what we have witenessed here is yet another example of noble cause corruption, where the end justifies the means in the minds of the players.
In reviewing Lewandowsky’s writings (here at The Conversation) over the past couple of years, it because painfully obvious that he sees climate skeptics as a scourge to be dealt with and that even crime can be justified:
Revealing to the public the active, vicious, and well-funded campaign of denial that seeks to delay action against climate change likely constitutes a classic public good.
It is a matter of personal moral judgment whether that public good justifies Gleick’s sting operation to obtain those revelations.
I believe that Dr. Lewandowsky set out to show the world that through a faulty, perhaps even fraudulent, smear campaign disguised as peer reviewed science, that climate skeptics were, as Jo Nova puts it, “nutters”. Worse, peer review failed to catch any of the problems now in the open thanks to the work of climate skeptics.
My best advice to Dr. Lewandowsky right now is: withdraw the paper. It has become a lighting rod for everything that is wrong with team climate science today, and multiple lines of investigation are now in progress including FOI requests and demands for academic misconduct reviews at your University of Western Australia.
I can’t see any of it ending well for you given your reticence to offer supporting data or explanations.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Paul H:
At September 6, 2012 at 11:53 pm you say to D. J. Hawkins,
No your claims of “irony” and “conspiracy theory” are mistaken: there is only your display of refusal to face reality which is typical of warmists.
It is normal for web surveys to be ‘gamed’ so competently conducted surveys use methods to inhibit the ‘gaming’. This survey failed to take such precautions.
Richard
In response:
1. Who knows?
2. Who knows?
3. Who knows?
The fact that it is impossible to determine how much gaming may have occured due to premature disclosure is one of several fatal flaws in the study. It isn’t up to me to prove it was gamed, it’s up to the survey designer to show what he did to prevent it. In this case, precious little.
“You seem certain of the answers to these questions.” No, I am not. Nor did I ever say so. Nor did I ever even infer that was the case.
“And the irony here is that you respond with conspiracy theory…” You need to improve your reading comprehension skills. A LOT. No where did I use the word conspiracy. And gaming wouldn’t need to be an organized, collective choice. Individuals of similar political inclinations could easily decide on their own to “help” the cause.
You think simply throwing out the word “conspiracy” is enough to embarrass people so they won’t look deeper into the facts, otherwise they’ll be considered just another nutter. You’re pathetic.
Apart from the ridiculousness of the survey, there is another issue here to do with comparison of numbers of different magnitudes. The number of people who believe the moon landings were faked is tiny. If one looks at climate alarmists you will find moon landing conspiracists make up a very small part of that group. And if you look at climate sceptics you will also find that moon landing conspiracists make up a very small part of that group. Looking at the proportions of moon landing conspiracists in each group actually tells us almost nothing about either climate sceptics or climate believers.
I have no idea what moon landing conspiracists think about climate change. What moon landing conspiracists think about anything is of very little interest to me. These people have logical powers of deduction so impaired that even if they reach conclusions I might agree with, it is almost certainly by accident and via a logical reasoning process that I would find incomprehensible.