Guest post by A. Scott
There has been considerable discussion about the methodology and data regarding the recent paper “Lewandowsky, S., Oberauer, K., & Gignac, C. E. (in press). NASA faked the moon landing – therefore (climate) science is a hoax: An anatomy of the motivated rejection of science” (copy here)
This allegedly peer reviewed paper claims their survey data show climate skeptics are supporters of wild conspiracy theories, such as “NASA faked the moon landing.” The author admits, however, no climate skeptic sites were involved in the survey, that essentially all survey results were obtained thru posting the survey on pro-global warming sites.
Due to the serious and legitimate questions raised, I have recreated the Lewendowsky Survey in an attempt to replicate and create a more robust set of replies, including from skeptic users.
Please click on the Lewandosky Survey Page above and you’ll be presented the survey. This survey replicates the questions, both the paper, and several sites have indicated were in the original survey, including those questions deleted from the survey results.
The only change was to use a 1 to 5 ranking vs. Lewandowsky’s 1 to 4, which several people with experience have noted should improve the overall responses.
Each visit to the survey is tracked. Access is password protected for an additional layer of tracking.
THE PASSWORD FOR THE SURVEY IS “REPLICATE” (case sensitive)
Please only complete and submit once. Also, please respond to each question with the answer that best reflects your position, even though the question may not be perfectly worded.
This survey is built on the Google Doc’s open access platform. Results are collected automatically. As no significant randomization or counterbalancing was performed on the original survey none is applied here. Data collected will be provided upon request.
A. Scott
* SPOILER ALERT *
Seriously, Lewandowsky presented this survey and its brethren “counter-balancing” surveys as a valid, statistically significant, meaningful survey? Just reading through the survey many issues jumped out at me. I would expect that even an incompetent psychologist could find hundreds of issues with these questions and a psychologist or sociologist who specialized in group profile analysis could many more.
My take on the questions is below. As I’m not a psychologist, sociologist, or statistician my wording might not be technically correct for the point I’m trying to make. (i.e. I used logically in places where I perhaps should have said contextually or statistically).
* SPOILERS BELOW *
Question 3: What is ‘social justice’? This term is relative culture, ethos, and mores. In a non-anthropologic or sociologic survey, with such a complete lack of context or definition, this term is functionally meaningless.
Question 4: The word localized is not defined and thus contextually void. Localized here could mean at a city, county, province, national, continental, global, or stellar scale. Without defining the bounds there is no ability to ascertain the level of ‘acceptable impact’ the survey participants feel is allowable to a free market economy.
Question 5: Again, sustainable development is vague and contextually void. Sustainable development is a term that is dependant on local and regional economy, culture, and historic population factors which are not explored by this survey. Question is meaningless.
Question 6: “Unsustainable consumption”? See comments re: Q5
Question 7: Allows one of two choices thus automatically filtering out a valid third choice that WMD were ONE of the considerations among others leading to the invasion.
Question 8: Given the apparent intent of the survey author, including the phrase “known as the New World Order” applies an automatic filter to a certain number of participants. Question would have greater validity if the phrase was omitted.
Question 9: Leaves out the option that is was produced in a lab under research conditions and released accidentally which is a valid choice. Question is predetermined to show either a ‘conspiracy’ or ‘mindlessly accept the press-release’ mindset.
Question 10: Assumes that foreknowledge (if any) was specifically for the purpose of allowing the attack to occur, not poor military planning or failure to comprehend the realistic possibility of an attack. Question is biased.
Question 11: Many flaws. Leaves out a valid choice of “or other agencies”. Also falls subject to the same criticism of question 9.
Question 13: Question is logically flawed. Properly asked should have omitted “in a Hollywood studio”.
Question 14: Could be argued that this question is logically flawed as well. Given that the question appears intended to uncover a ‘secret alien contact’, ‘secret alien influence’, or ‘conspiracy’ mentality, the question probably should have been written as follows: “One or more government or international agencies have, or had in the past, secret bases that contain hidden alien spacecraft, alien bodies, or alien technology.”
Question 16: Also logically flawed and poorly written unless this question was worded to filter out everything but a very narrow group respondants. Everything following ‘attacks to take place’ could have been omitted. Alternatively everything between “achieve” and “goals” could have been omitted. The wording of the question excludes an entire group of potential respondants.
Question 18: Narrow filter (again). Excludes assassination attempt theory by groups other than the royal family. Possibly invalid question again as it is very narrow. Would need to see the underlying methodology and psychology that went into the question design.
Question 19: Again and oddly placed, narrow filter. Why “Neo Nazi” ? There are many other conspiracy theories here as well: Government (foreign or domestic); Shadow government; false flag; etc.
Question 20: False choice leaving some respondant no alternative but to answer falsely. What about the group of respondants who do not feel it is a hoax perpetrated by scientists but by government agencies or corporations with vested financial interest in ‘green’ technology? Or that don’t feel it is a hoax, merely poor science? Or that it isn’t a hoax, but simply a runaway political process that has taken on a life of its own?
Question 24: Once again, requires a selection between only two choices where there are multiple alternative answers. Question is predisposed to display only one of two responses and thus is not valid for true picture of the respondant group beliefs.
Question 25: Predisposes that the changes are “serious”. Built in selection bias.
Question 26: Assumes that all respondants believe that CFC’s WERE a serious threat originally. Question should have a predecessor or descendant question to filter out or identify the bias inherent in the question wording.
Question 27: See comments re: Q 26
Question 35: Possible inherent bias. By not quantifying the intensity (somelots) and direction (positivenegative), nor including any clarifying questions in the survey, this question classifies all respondants into one of two buckets. This may be valid or may not depending on the intent of the survey.
Question 38: Identical criticism as in Q 35.
==================
Question 3: What is ‘social justice’? This term is relative culture, ethos, and mores. In a non-anthropologic or sociologic survey, with such a complete lack of context or definition, this term is functionally meaningless.
Question 4: The word localized is not defined and thus contextually void. Localized here could mean at a city, county, province, national, continental, global, or stellar scale. Without defining the bounds there is no ability to ascertain the level of ‘acceptable impact’ the survey participants feel is allowable to a free market economy.
Question 5: Again, sustainable development is vague and contextually void. Sustainable development is a term that is dependant on local and regional economy, culture, and historic population factors which are not explored by this survey. Question is meaningless.
Question 6: “Unsustainable consumption”? See comments re: Q5
Question 7: Allows one of two choices thus automatically filtering out a valid third choice that WMD were ONE of the considerations among others leading to the invasion.
Question 8: Given the apparent intent of the survey author, including the phrase “known as the New World Order” applies an automatic filter to a certain number of participants. Question would have greater validity if the phrase was omitted.
Question 9: Leaves out the option that is was produced in a lab under research conditions and released accidentally which is a valid choice. Question is predetermined to show either a ‘conspiracy’ or ‘mindlessly accept the press-release’ mindset.
Question 10: Assumes that foreknowledge (if any) was specifically for the purpose of allowing the attack to occur, not poor military planning or failure to comprehend the realistic possibility of an attack. Question is biased.
Question 11: Many flaws. Leaves out a valid choice of “or other agencies”. Also falls subject to the same criticism of question 9.
Question 13: Question is logically flawed. Properly asked should have omitted “in a Hollywood studio”.
Question 14: Could be argued that this question is logically flawed as well. Given that the question appears intended to uncover a ‘secret alien contact’, ‘secret alien influence’, or ‘conspiracy’ mentality, the question probably should have been written as follows: “One or more government or international agencies have, or had in the past, secret bases that contain hidden alien spacecraft, alien bodies, or alien technology.”
Question 16: Also logically flawed and poorly written unless this question was worded to filter out everything but a very narrow group respondants. Everything following ‘attacks to take place’ could have been omitted. Alternatively everything between “achieve” and “goals” could have been omitted. The wording of the question excludes an entire group of potential respondants.
Question 18: Narrow filter (again). Excludes assassination attempt theory by groups other than the royal family. Possibly invalid question again as it is very narrow. Would need to see the underlying methodology and psychology that went into the question design.
Question 19: Again and oddly placed, narrow filter. Why “Neo Nazi” ? There are many other conspiracy theories here as well: Government (foreign or domestic); Shadow government; false flag; etc.
Question 20: False choice leaving some respondant no alternative but to answer falsely. What about the group of respondants who do not feel it is a hoax perpetrated by scientists but by government agencies or corporations with vested financial interest in ‘green’ technology? Or that don’t feel it is a hoax, merely poor science? Or that it isn’t a hoax, but simply a runaway political process that has taken on a life of its own?
Question 24: Once again, requires a selection between only two choices where there are multiple alternative answers. Question is predisposed to display only one of two responses and thus is not valid for true picture of the respondant group beliefs.
Question 25: Predisposes that the changes are “serious”. Built in selection bias.
Question 26: Assumes that all respondants believe that CFC’s WERE a serious threat originally. Question should have a predecessor or descendant question to filter out or identify the bias inherent in the question wording.
Question 27: See comments re: Q 26
Question 35: Possible inherent bias. By not quantifying the intensity (somelots) and direction (positivenegative), nor including any clarifying questions in the survey, this question classifies all respondants into one of two buckets. This may be valid or may not depending on the intent of the survey.
Question 38: Identical criticism as in Q 35.
This survey adds more evidence to prove that there is no such thing as an impartial position. Scientists always start out with an hypothesis and then proceed to prove it, however, the scientist is only a man/woman influenced by their preconceptions. For example, Global Warming is happening so when I adjust the temperature figures the adjustments always go up. This is why any scientific work that requires interpretation, adjustment or averages creates doubt. And if this paper was peer reviewed my doubts are increasing by the minute. I did the survey; love surveys; can we do another one?
Steve McIntyre at Climate Audit pulls no punches in his analysis of the Lewandowsky imbroglio:
“Anatomy of the Lewandowsky Scam”
http://climateaudit.org/2012/09/08/lewandowsky-scam/
LazyTeenager says:
September 8, 2012 at 10:36 pm
“Many of you guys visit pro warming sites so presumably you entered the survey. Especially since you aren’t shy about gaming online surveys.”
a) Where is your evidence that we did? Remember, Lewandowsky promotes his drivel under the guise of science. Science is evidence based. Your presumption needs a proof.
b) I do occasionally visit grist when I need a laugh. But I wouldn’t stand visiting them systematically. Too hard on the mind.
I don’t visit SkS as enough warmists come over to notrickszone, paste huge amounts of SkS drivel and let me have the fun of shooting them down. Unfortunately, none of them brought the Lewandowsky survey.
So what you’re trying to do there, some after the fact justification of a flawed methodology? Why is it that you defend each and every warmist idiot researcher? Because there are no smart ones?
Mike McMillan said:
“No, the moon landing wasn’t faked in Hollywood.
Mike, in Houston, who knows.”
Where was it faked then?
/tinfoilhat
I’m pretty sure I got two wrong. The one about Area 51 causing global warming for sure. May have been a trick question.
Let’s clear up the Area 51 question once and for all. Anyone who has worked in classified areas should appreciate this:
Area 51 Commander: Washington, we had a classified aircraft go down near Roswell NM.
Washington: Does anyone not cleared know about this?
Area 51 Commander: Yes, locals have found the wreckage.
Washington: Put out a cover story and refer all subsequent inquires to us.
Area 51 Commander: Roger.
Washington (hours later): What did you use as a cover?
Area 51 Commander: We planted an alien spacecraft story.
Washington: YOU DID WHAT????? Oh my God, you’ve got to be kidding me! Now there will be no end to the nut jobs who will say this is proof of aliens. Well, maybe on second thought, we could make good use of this in the future.
Well,it was pretty awful, but I completed it as well and left a comment that it is impossible to answer many of the questions accurately where they include words like `cause’,, `significant,, or` serious’ without defining rather more what is meant – e.g. smoking has been shown to increase the risk of contracting lung cancer considerably, but it doesn’t necessarily `cause’ lung cancer. That’s not a particularly good example, but the same sort of objection could be made – more strongly – to many of the questions concerning climate change (and indeed the free market).
How many medical students believe smoking causes lung cancer? Well, did he missed out the word only. What kind of question is that?
How many climate scientists believe CO2 …etc. Well, how many climate scientists want a paycheck?
There was a conspiracy to kill John Kennedy … and it as successful.
There is a conspiracy to form a world government … it’s called the United Nations.
That was as hilariously bad a survey as I’ve ever seen. That the idiot left wing ideologue who thought that has a doctorate shows the level of quality the educational system has sunk to.
Quite astonishing that such a shoddy set of questions should be the basis for a “study”, and then accepted by a Journal.
I filled it in – holding my nose; I have had a lot of practice at filling in daft questionnaires after a lifetime in industry and commerce – but don’t know if any good will come of it.
John Brookes says:
September 8, 2012 at 10:10 pm
But are people being honest in answering the questions? I can see a lot of “skeptics” gritting their teeth and saying they don’t believe in Roswell, or 9/11 truth, or all those other silly conspiracy theories, because they know it would make them look bad if they did admit to belief.
On the other hand, many “skeptics” are probably proud of their conspiracy beliefs…
=====================================================================
Then you have just admitted having the exact same bias against people who are skeptics of CAGW as the author of the study. I don’t believe in ANY of that conspiracy horsebleep. Not one of them. But just the wording of the questions makes it obvious that the author thinks I, and other skeptics do believe them. Not just do believe, but MUST believe. Projection.
==================================================================
==================================================================
LazyTeenager says:
September 8, 2012 at 10:36 pm
The author admits, however, no climate skeptic sites were involved in the survey, that essentially all survey results were obtained thru posting the survey on pro-global warming sites.
—————-
Why would this matter?
Many of you guys visit pro warming sites so presumably you entered the survey. Especially since you aren’t shy about gaming online surveys.
So how relevant this all is depends on the research question. Obviously questions like how many people believe global warming is not a valid research question. But relationships between beliefs would be a valid research question assuming the courage to go and read what other people saying does not bias the results.
=====================================================================
LT,
“Why would this matter?”
For the reason stated above to John. They will project what they believe “skeptics” think about all of these nutty conspiracies.
I don’t believe climate scientist/researchers are reaching the conclusions they are reaching “to get more grant money”, reseachers were trying to get grants long before global warming became the issue “du jour”, and they will be doing it long after people stop talking about global warming.
I think they are doing it as they believe they are doing something for the general good. I am skeptical because when I hear about scientists that wont release their background data and methodology my personal bullsh*t meter starts going off. I think they are cherry picking the data to support their beliefs. Rant over.
If this (original survey) was done as an April Fools joke, it might be funny. That the original survey might be sold as serious “science” in any way shape or manner is tragic.
I like this approach to shooting down a botched survey –
http://motls.blogspot.com/2012/08/where-can-you-find-out-whether-climate.html
Lubos does a great job of showing how polls taken on one side of the argument are going to be heavily skewed.
Also, I forgot to mention this in my comments on the actual survey, but I think 1-3 questions gauging education background (and maybe area of expertise) would be useful. First, it would give you more data to work with. Previous studies have shown that the average skeptic is more educated than the average CAGW believer. Second, it’d make the survey harder for the CAGWers to game by two mechanisms. First, their vanity might get in the way because they love to show off their letters/degrees. Secondly, if they did try to pretend to be a nutso skeptic, they’d have a hard time forcing themselves to pretend to be an educated conspiracy theorist. Thus, the result would likely be show pretty accurate results at the education levels above the lowest category.
-Scott
In principle, any voluntary online poll is biased and not worth the paper it’s printed on. Those working for a valid poll will be trying hard to get responses from a representative sample of the populace. I look on ANY online self selecting voluntary poll as a push-pull poll with political motives- this is reinforced by all the online crappy leading polls we get here in the US during election season.
Had I been in Anthony Watts’ position, and had I received such a request to post this poll, I would have justifiably consigned that request to my spam file, along with all the other polls asking leading questions in an effort to get cash support for a specific proposition or candidate.
In quantum theory, there’s a problem of the observer influencing the results- I’m thinking of the “double slit” experiment. That same problem occurs in the macro field of poll questions. When those questioned are aware that they’re being tested to determine whether THEY or their OPPONENTS are kook conspiracy nuts, that’s going to bias the answers.
Jumping between quantum theory, politics, and economics, I see a similar problem with Keynes’ theory, According to the theory, government spending will help boost the economy when it’s in a slump. Keynes was implicitly assuming that workers would be UNAWARE of this government stimulus, and would accept constant wages in the face of inflation. When workers are AWARE of government stimulus efforts- again we have the observer influencing the results.
A quick question if I may… If I use my real name will it be publicized? Will my online moniker be sufficient?
Such a transparently biased survey – what a joke. Anyone who can’t see what results they were trying to get out of this shouldn’t be allowed to respond due to low IQ.
IT IS AXIOMATIC….only bogus evidence can support a bogus hypothesis ! ! !
BTW….i am firmly in the “conspiracy camp” as there is overwhelming evidence that all of the many frauds perpertrated on humanity stem from our defective banking system. This is explainded in “Fractional Reserve Banking Begat Faux Reality”. A few of the many elitisted directed, government funded science frauds are explored in “Becoming a TOTAL Earth Science Skeptic”. When the monarch/monopolists publish their neo-feudalist game plan, then direct their puppet governments and lap dog media to enforce this new-think mind warp, you’ve gone beyond conspiracy theory to CONSPIRACY FACT. It is time for a new Magna Carta and to free ALL of humanity from this suffocating corruption.
I’ll answer those questions right after I stop beating my wife.
I want those five minutes back.
Anthony,
Please ask this post’s author, A. Scott, to include a prominent note in the explanation of the survey he provides here that, at the very end of the survey, one is asked for both a name and an email address.
I wasted 10 minutes filling it out and then didn’t submit it because of the quite personal nature of the last five to ten questions about quality of life, income, etc., followed by the request for my name and email address.
People should be informed going in that they will need to supply that information, or if the field is optional, that should be specified in Mr. Scott’s post. (In fact, it might be optional, since I don’t recall it having a red asterisk by it, but I gave up at that point, so I don’t know.)
In the former british empire is Lewpaper the same as Loo paper? Inquiring minds want to know.
I filled in the survey for a bit of fun – but entirely honestly. I liked the questions at the end, for example, about how much you earn relative to your neighbours and are you happy with your life. Are we supposed to feel guilty or something?
I have worked hard all my life to get where I am today and I did it all by myself. I was born into a poor family and everything I have now I earned by my own hard work and I do not feel guilty in the slightest.
Now for another ice cold carbonated beer.
Cheers and all the best!
I know three conspiracy theoreticians. They are all rabid CAGW believers too.
I did not ‘submit’ the survey on A. Scott’s page. I also left the following as a post there and at the good Bishops site.
I found the survey as posted by “A. Scott” impossible to answer.
First: I refuse to answer personal questions about sex, age or financial status.
Second:
I am convinced that whoever (lewandosky or ?) prepared this list of questions knows much less than the average citizen and has many assumptions of causes without evidence or proof. Their level of understanding is not that of a scientist, but seems to be that of a religious zealot.
e.g. Question 34: Smoking causes cancer. Is this a generic or specific question? Does smoking always cause cancer or is it that smoking tobacco may cause cancer. As stated the question is absolute but in the most general way, therefore I cannot agree nor disagree to any degree as I do not wish to further belief in general absolutes.
Much of this survey is framed in variations of this general absolutes form. Did a true Doctor of Psychiatry truly vet these questions?
And where did they get their inadequate level of scholarship on historical events? The absolute simplistic questions on some very complex historical events are just plain absurd, incompetent or outrageously fraudulent.
I pity anyone who actually believes useful knowledge can be derived from completing this survey. Surely such people are activists or worse, zeolots.