I’m a bit of a latecomer to this affair, as Lucia and Jo Nova took an early lead on pointing out the many problems with the survey methodology (or lack thereof) with the paper:
Lewandowsky, S., Oberauer, K., & Gignac, C. E. (in press). NASA faked the moon landing – therefore (climate) science is a hoax: An anatomy of the motivated rejection of science. Psychological Science.
“Motivated” is the key word here, as it appears there were hidden motivations for this paper. It seems though, once you scratch the surface of Lewandowsky’s paper, that it is nothing more than a journal sanctioned smear of climate skeptics based on not only faulty data, but data gathered with a built in bias.
Besides what we already know about the flawed sampling method, the lack of follow up with skeptic blogs to make sure they got communications inviting them to post the survey, and the early release of results before the survey was complete, the most troubling new revelation appears to be that some climate skeptic blogs got different questionaires than their counterpart AGW advocate blogs. If true (and it appears to be based on the survey numbering system) this negates the study on the basis of inconsistent sampling, and I think it is time to ask Psychological Science editor Robert V. Kail to investigate this paper, and if he finds what the skeptics have, start a retraction. I’ve sent him a courtesy note advising him of this issues with this paper.
Here’s a summary of what has been going on the last couple of days.
Jo Nova has a great summary here, and writes about one Australian investigator who was invited to take the survey questions two years ago, kept screen shots of it, and did an analysis. She wrote:
Graham from OnlineOpinion was so struck by the study he’s written a post titled: Fish rot from the head Part 1.
Read it to get familiar with the survey questions.
Next there’s the who got what version of what survey problem, Jo notes this:
Leopard on the Bishop Hill thread has noted that Steve McIntyre is asking Lewandowsky why there are two or even three different forms of the survey? Why indeed?
Paul follows them up:
The Deltoid, Tamino, Mandia and Hot-Topic blogs were sent the survey number surveyID=HKMKNF_991e2415 on about August 29th. That survey is on the archive, and starts with 6 questions about free markets.
Bickmore and Few Things had the survey number surveyID=HKMKNG_ee191483 also about Aug 29, but this one doesn’t seem to be on the archive.
Steve Mc was sent survey number surveyID=HKMKNI_9a13984 on Sept 6th. This survey is on the archive, and it starts with 5 completely different questions! About how happy you are with life.
This right here should be enough for a retraction from the Journal. If different surveys were sent to different bloggers, and no mention of it was made in the paper or justified in the methodology, then this amounts to purposely biased data from the beginning. UWA may also find grounds for academic misconduct if Lewandowsky purposefully sent different sets of questions based on the type of blog he was inviting.
And then we have the fact that Lewandowsky was discussing preliminary results at a seminar, while the surveys were still open and he had not heard back from the skeptic blogs yet, such as the follow up invitation to Steve McIntyre. Having an open discussion of the survey is highly irregular, because attendees/viewers are free to take the survey, possibly biasing the results.
On the 23rd of September, 2010, Dr. Lewandowsky gave a presentation at Monash university which included the following slide:
Lewandowsky & Gignac (forthcoming)
•Internet survey (N=1100)
•Endorsement of climate conspiracy (“hoax by scientists to get grants”) linked to endorsement of other conspiracies (“NASA faked moon landing”)
•Conspiracy factor without climate item predicts rejection of climate science
So three days after (unsuccessfully) asking for cooperation in fieldwork, Lewandowsky is publicly announcing the preliminary results while the surveys are still open, and he hasn’t heard back from invited distributors. Steve McIntyre of Climate Audit notes that he received a follow up invitation on around Sept 20th. (which he didn’t notice until this story broke). Note the N=1100 value in the preliminary slide. The final paper cites an N<1200 value.
And it seems they are still at it, here’s a recent WUWT comment:
Anthony, there was recently another survey (longer, and with a 1-5 scale) put out by Lewandowsky’s research assistant, Charles Hanich, on June 4, 2012. It seems that the link for this survey was only posted on two blogs: Watching the Deniers and Skeptical Science. Charles Hanich was also responsible for creating Lewandowsky’s 2010 survey, as mentioned in the comments here.
Unfortunately, the link to the June 2012 survey is also unavailable. However, a skeptic called the “Manic Bean Counter” captured all the survey questions and dissected them on his/her blog, here. The following is Manic Bean Counter’s breakdown of the types of questions asked in the survey:
1. Climate Change – 5 questions
2. Genetically Modified Foods – 5 questions
3. Vaccines – Benefits and harms – 5 questions
4. Position of the Conservative / Liberal perspective (US definitions) – 7 questions
5. Select neutral (check of the software, or check for spam?) – 1 questions
6. Free market system v social justice / environment / sustainability – 5 questions
7. Conspiracy theories (political) – 6 questions
8. Conspiracy theories (scientific) – 6 questions
9. Personal Spirituality & Religion – 8 questions
10. Evolution – views upon – 7 questions
11. Corporations – 13 questions
12. Personal emotional outlook – 6 questions
The striking thing is that we have John Cook’s Skeptical Science blog listed as presenting both the original as well as the most recent survey. It as been discovered that Cook is a co-author with Levandowsky on a similar paper:
Lewandowsky, S., Ecker, U. K. H., Seifert, C., Schwarz, N. & Cook, J. (in press). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest.
http://websites.psychology.uwa.edu.au/labs/cogscience/documents/LewandowskyEcker.IP2012.PSPI.pdf
One wonders how much Cook contributed to the questions, based on his understanding of his readers likely responses. It is strange irony indeed that the paper discusses “debiasing”, when so many potential biases in Lewandowsky’s methods are clearly obvious to even the casual reader. Wikipedia even cites them for this paper in a section on “debunker”
Australian Professorial Fellow Stephan Lewandowsky[5] and John Cook, Climate Communication Fellow for the Global Change Institute at the University of Queensland (and author at SkepticalScience.com)[6] both warn about “backfire effects” in their Debunking Handbook.[2] Backfire effects occur when science communicators accidentally reinforce false beliefs by trying to correct them. For instance, a speaker about global warming may end up reinforcing the crowd’s beliefs that global warming is not happening.
Backfire indeed, this Lewandowsky “moon landing” paper is a full force backfire now.
Based on what I’ve seen so far, it is my opinion that Lewandowsky set out to create the survey data he wanted by manipulation of the survey system through multiple undocumented surveys, incomplete and non-representative sampling, biased survey questions, and essentially no quality control. There weren’t even significant safeguards in place to prevent individuals from taking the survey multiple times, appearing as other identities. There are so many things wrong with this paper that I can’t see it surviving intact.
I think what we have witenessed here is yet another example of noble cause corruption, where the end justifies the means in the minds of the players.
In reviewing Lewandowsky’s writings (here at The Conversation) over the past couple of years, it because painfully obvious that he sees climate skeptics as a scourge to be dealt with and that even crime can be justified:
Revealing to the public the active, vicious, and well-funded campaign of denial that seeks to delay action against climate change likely constitutes a classic public good.
It is a matter of personal moral judgment whether that public good justifies Gleick’s sting operation to obtain those revelations.
I believe that Dr. Lewandowsky set out to show the world that through a faulty, perhaps even fraudulent, smear campaign disguised as peer reviewed science, that climate skeptics were, as Jo Nova puts it, “nutters”. Worse, peer review failed to catch any of the problems now in the open thanks to the work of climate skeptics.
My best advice to Dr. Lewandowsky right now is: withdraw the paper. It has become a lighting rod for everything that is wrong with team climate science today, and multiple lines of investigation are now in progress including FOI requests and demands for academic misconduct reviews at your University of Western Australia.
I can’t see any of it ending well for you given your reticence to offer supporting data or explanations.
Skiphil says: “ ‘…Psychological Science is the highest ranked empirical journal in psychology.’ ”
It is certainly rank, if the Lew Paper is any indication of their typical quality. Perhaps they should change the name to ‘Psycho Science.’
Maus:
With Pr Lewandowsky we can dispense with the science when it comes to climate (no, I don’t imply any fraudulant element at all).
If a field calls itself science then the key participants ought behave as scientists and rigorously question the methodology rather than scratch around and then advocate, politicise and foster with religious zeal.
I am familiar with Prof Lewandowsky’s writing on the subject of CAGW and this paper reflects the same lack of scientific thought and pall of confirmation bias and advocacy.
His motives may be simply altruistic but I have little time for scientists who wish to be excused on the basis of wanting to save the planet. The likes of Hansen admitting to exaggeration on the grounds of altruism simply means that the process is politicised and accordingly science is shunted out the window.
If he is unaware of his predilection then he makes for being a poor scientist and is a poor advertisement for UWA which historically has had a good standing.
Perhaps it calls for a psych analysis – of the guys who wrote the paper.
Maus says: “…I can hardly sort out how to make things obviously absurd unless I resort to typing with my forehead. However, this seems rather inconvenient and I suspect the output would be inscrutable.”
Though much improved.
Son, since I don’t see a /sarc tag lying about, I mean this sincerely: Have you always been this thick or did someone hit you in the head recently? Do you seriously mean to suggest that the unsurveyed stations would become aware of the preliminary results and conciously or unconciously adjust their historic readings or metadata? Because that’s the only way you could possibly draw a parallel to the current discussion.
Kyle says:
September 6, 2012 at 9:26 am
“I have a Ph.D. in Genetics and yours is the type of attitude that resulted in tremendous lost time not studying so-called “junk” DNA, now shown by the ENCODE project to be highly functional and deeply involved in the regulation of gene transcription.”
That was pretty obvious – when I was a kid I started to learn programming and at the same time read about genetics. I read that a gene was something that encoded a protein, and that the rest was junk DNA. I asked myself, so where’s the control logic.Well, can only be in the stuff that doesn’t produce genes.
I told my biology teacher that we have enzymes that repair genetic defects. He didn’t believe me and gave me a bad grade. Biology was not his strong point. I learned to have more patience with my teachers.
John Brookes says:
September 6, 2012 at 5:50 am
“All these poor persecuted free thinkers. Does it never occur to them that they aren’t being persecuted for their radical thoughts? They are being persecuted for being stupid, or ratbags, or both.”
You justify persecuting stupid people? Do you persecute stupid people yourself? Do you think that persecution for the stupid is right? Are you a warmist? Are all warmists of such a vile bent?
Kyle,
Could you elaborate a bit on the inhibiting effect of designating some DNA “junk?” Is what these guys just did what would have been done sooner, or is it different? Where did the demarcation between useful and junk DNA lie? How was junk DNA distinguished from the good stuff? Was it just that no-one had gotten to it yet?
On a more climate related issue, could it be that disposing of all the not-understood DNA as junk might be similar to dismissing the question of whether there has been an increase in “Total Energy” on Earth because it’s too difficult to arrive at that figure and “seemingly” much easier to derive a temperature – which seems a pretty sorry substitute to me, at least.
I read that a gene was something that encoded a protein, and that the rest was junk DNA.
That is one, narrow definition of a gene; another includes the upstream and downstream untranslated regions within the mRNA; another includes upstream and downstream promoter elements beyond the transcribed regions. You could also define a gene based upon heritable elements. Promoter elements can be far away from the gene, such as enhancer elements; and these and other “control” elements have been known and studied for decades.
I learned to have more patience with my teachers.
And the reverse, perhaps.
What controls were in place to prevent warmists from feigning skepticism and answering the survey dishonestly?
“What controls were in place to prevent warmists from feigning skepticism and answering the survey dishonestly?”
We have our conspiracy, we have a winner, no more calls please.
@j ferguson
Don’t want to get too far OT, but the designation “junk DNA” was first coined in 1972. Basically the consensus notion that most of the genome was nonfunctional prevented funding for that research for about 20-25 years. The prevailing theory of the time was that the 2% of protein encoding DNA was the only functional part and the rest of the 98% was mostly non-functional. Leslie Orgel and Francis Crick argued in a 1980 article in Nature that “much DNA in higher organisms is little better than junk…[it] can be compared to the spread of a not-too-harmful parasite within its host…[it has] little specificity and conveys little or no selective advantage to the organism.”* Same issue, Doolittle and Sapienza: “Natural selection operating within genomes will inevitably result in the appearance of DNAs with no phenotypic expression whose only ‘function’ is survival within genomes. Prokaryotic transposable elements and eukaryotic middle-repetitive sequences can be seen as such DNAs, and thus no phenotypic or evolutionary function need be assigned to them.”
Can’t do research if you can’t get money! To be fair, much of our recent progress is due to improved methods and the human genome map. However, the attitude of “we can’t figure out what this DNA does therefore it’s useless” was an argumentum ad ignorantiam that has since been proven false. This article from 2007 does a decent job of summing up the field:
http://www.scientificamerican.com/article.cfm?id=what-is-junk-dna-and-what
My views on what constitutes genes, junk DNA, and the like are far different from the mainstream so a WUWT comment isn’t really the best place to flesh out the idea. I cross-trained in computer science so I view the genome as a fluid multi-dimensional parallel processing unit. That’s not widely accepted although several prominent genetic researchers have privately expressed their agreement.
Back on topic, the Lewandowsky paper should be reported to the Australian equivalent of the Institutional Review Board for UWA. It’s their job to protect the institution from researchers improperly recording data with human subjects. Believe me, they will not hesitate to shut him down. Failure to do so could result in an audit of all IRB protocols and a loss of all funding to the institution.
—————————
*Orgel LE, Crick FH (April 1980). Selfish DNA: the ultimate parasite. Nature 284 (5757): 604–7
Doolittle WF, Sapienza C (April 1980). Selfish genes, the phenotype paradigm and genome evolution. Nature 284 (5757): 601-3
DirkH~ you are SO lucky brooksie isn’t honest. Otherwise he’d be nodding ‘Yes’ so hard, his head would snap clean off.
The level of debate of the Lewandowsky paper (and sadly, some of the posts) is about as good as ‘skeptics have more nutters than warmists – so we are right – nah nah nah nah nah’!
Unfortunately it sums up much of todays climate ‘science’ with ‘research’ aimed at how they can prove their warmist political and religious mantra. Junk science is just that, whatever side of the argument you are on as WUWT contributors are only too ready to point out.
September 6, 2012 at 2:15 pm | Otter says:
DirkH~ you are SO lucky brooksie isn’t honest. Otherwise he’d be nodding ‘Yes’ so hard, his head would snap clean off.
————————————
Otter, given Brooksie’s predilection to fantasy as supported by his posting and the pasting that he gets at JoNova as well, I can only conclude that he has a particular fetish for ‘S&M’ culture … what other explanation could there possibly be for his perpetual invitation of such regular scorn and ridicule in blogs ? Clearly, the bloggers at JoNova are beneath him in the quality of the scorn and abuse that he considers that he deserves and this leads him here to challenge the eminent WUWT bloggers to do their worst.
Re psychology: Jung once speculated that because UFO sightings usually cited saucer shapes or cylindical objects that this might be down to sexual projection.
Does the fascination of modern climate scientists with hockey stick shapes demonstrate a similar phenomenon?
Worth a paper or two in a journal, imo. /sarc
Maus says:
September 6, 2012 at 8:41 am
jorgekafkazar: “Maus has a history of making insulting comments.”
I can hardly sort out how to make things obviously absurd unless I resort to typing with my forehead. However, this seems rather inconvenient and I suspect the output would be inscrutable.
It appeared that is what you were doing!
TonyM: “If a field calls itself science then the key participants ought behave as scientists and rigorously question the methodology rather than scratch around and then advocate, politicise and foster with religious zeal.”
I don’t disagree with you. The problem is that the mythological definition of science is ‘the scientific method’, while the practical definition is ‘what people with university degrees do’. To move from the latter to the former requires asking a great deal of people. That they observe science and see if there’s any science in it.
RokShox: “What controls were in place to prevent warmists from feigning skepticism and answering the survey dishonestly?”
None of the standard tools to detect the most obvious spoilers and deceivers were used. Multiple instances of the same question were not asked and interviews were not performed. Nor of course was it ‘survey’ as there were multiple different surveys that were given out.
Of course, the data that was received was contrary to presentation of the conclusions regardless. Errors of this magnitude require a purposeful approach or a complete and utter ignorance of the most basic research practices. The only answer here is that it was fraud. Either on the part of Lewandowsky, the editor, and the reviewers. Or that they were all victims defrauded by whichever institution claimed that they have learned their craft.
Komrade Kuma says: September 6, 2012 at 3:47 am
Thanks for the videos … I had never seen those before. They truly speak for themselves. I couldn’t believe they weren’t a spoof, but I checked his photos at UWA. /boggle
Incidently I notice Desmog now has a piece on Lewandowsky and the reactions at:
http://www.desmogblog.com/2012/09/05/research-links-climate-science-denial-conspiracy-theories-skeptics-smell-conspiracy
I find this an interesting stance when Desmog’s primary funder is Mr John Lefebvre, who on Wiki has the following:
As an environmental advocate, Lefebvre is a chief benefactor of DeSmogBlog.com,[1] a whistleblower blog run by Vancouver public relations specialist James Hoggan. The site’s focus is on exposing those who deny the effects of fossil fuel upon the world’s climate while covertly working on behalf of fossil fuel producers.
Source: http://en.wikipedia.org/wiki/John_Lefebvre
So their mind bubble that “denialists” are all funded by Big Fossil Fuel is legit, but the suggestion that climate scientists might be influenced by the billions in known government funding for climate science is conspiracy?
Two things:
1) I am confused.
2) where’s my BFF cheque?*
Cheers,
Bulldust
will accept wire transfer as well.
John Brookes – We don’t like strangers in in this town…. particularly those of the warmist variety. (sarc)
Seriously John, I quite look forward to your comments at Jo Nova’s blog. Although I don’t always agree with you, I find you come across as quite civil. You also seem be shown respect there, perhaps tolerated may be a better word :). It’s a shame many of the commenters on other blogs are so partisan and intolerant of other views.
John Brookes:@September 6, 2012 at 5:50 am
I read Windshuttle because he focused on research theory and scientific method. He came from the old UNSW Marxist school (pre post modern), that insisted that conclusions could only be made if supported by a testable hypothesis and verifiable evidence. One of his old fashioned rules was that you should read a book before you make claims about the author, the content or the conclusions. Another rule was that ad hominem attacks do not substitute for rational argument. I doubt you have read Windshuttle, and that deficiency informs your opinion of his work.
In academia I have met many people with the same superior attitude that you have, and generally they are happy to support conclusions without supporting evidence, so long as it suits their broader political beliefs and their access to grant funding or tenure. Like Lewandowsky, they are politically astute but intelectually flaccid.
Are you aware of the Stalinist authoritarianism that lies just under the surface of your soft leftism? You seem to think its justifiable to persecute somebody who you think is stupid, or you think is a ratbag. Be careful, you will be judged and condemned by your erstwhile friends, using the same cruel standards that you apply to others.
The design of Lewandowsky et al.’s survey is a mess. “Climate change” is supposed to be at the core of it, yet there are many more questions about a wide range of conspiracy theories than about CAGW. And, as many have noted, there is no item about the notion that opposition to CAGW doctrines is all the product of a conspiracy by oil companies. The “other science questions” on the survey number exactly two: one about smoking and lung cancer and the other about HIV and AIDS.
All of this is assuming that there was just one version of the survey, instead of multiple versions whose variations were not mentioned in the report, as may in fact may have been the case.
The manner in which participants were recruited was grossly inept.
Only correlations and the results of structural equation modeling were reported—no means, standard deviations, or frequencies for any of the survey items.
The framing provided for the research, in the introduction and in the discussion, is completely prejudicial and includes many citations of dubious sources.
The title doesn’t fit the results of the study, as belief in a free market appears to account for a lot more of the CAGW skepticism in the sample than “conspiracist ideation” does.
Political psychology is already one of the weakest specialties within the discipline, but an article like this will reduce those who are trying to bring some rigor to banging their heads agains the nearest wall.
I teach survey research methods to undergraduates as part of an experimental psychology course. I have to wonder whether Lewandowsky or his coauthors ever passed such a course. A student project as poorly executed as theirs appears to have been would earn an F.
If something like this had happened back in Mark Twain’s time, the good professor would be a candidate for tar and feathers and being run out of town on a rail (like Huck’s Duke and Dauphin).
Times have changed and not for the better.
RobJM hit the nail on the head: psychology is nowhere near being a science, in the proper sense of the last five hundred years, since Galileo. A good Mexican friend of mine refers to the practitioners of this ‘art’ as pepsichologists (it works better in Spanish, because of the spelling conventions). But then, there are really weird things in Spanish: ‘ciencia’, ‘science’ includes areas of ‘knowledge’ like theology, aptly described by Ronald de Sousa (I found the quote in Daniel Dennet, Darwin’s Dangerous Idea) as ‘intellectual tennis without the net’! Light relief, I recognise.
D. J. Hawkins,
“Do you seriously mean to suggest that the unsurveyed stations would become aware of the preliminary results and conciously or unconciously adjust their historic readings or metadata?”
How many people actually saw Lewandowsky’s presentation prior to filling in the survey? How many of those were then motivated to fill in his presentation? How many of those had mischievous intent? You seem certain of the answers to these questions. And the irony here is that you respond with conspiracy theory: evil warmists saw Lewandowsky’s presentation and became motivated to game the survey.
Fitting Lewandowsky et al into a wider context …
http://thepointman.wordpress.com/2012/09/07/an-assessment-of-current-alarmist-propaganda/
Pointman