Readers may recall this survey: A poll to test the Lewandowsky methodology The results are in, which is why we can’t say global warming proponents support pedophilia.
Guest essay by Brandon Schollenberger
They don’t. The fact there is a correlation (0.14) between believing global warming is a serious threat and saying pedophilia is good is meaningless. The fact this correlation is “statistically significant” (at the 99.99% level) is meaningless. Anyone who looks at the data can immediately see the results are bogus (a small jitter value was added to allow us to see the density of responses):
There are only 20 or so respondents (out of over 5,000) who claim to believe global warming is a threat and claim to support pedophilia. It’s likely those responses were false. Nobody can seriously claim that proves global warming proponents are pedophiles.
The issue of false responses received a lot of attention with Stephan Lewandowsky’s paper, “NASA faked the moon landing—Therefore (Climate) Science is a Hoax. In that paper, similarly false responses created spurious correlations. Unfortunately, the focus on false responses meant a more fundamental issue got missed. Namely, the entire idea behind this approach is nonsensical. The approach is like taking the data displayed above and drawing this line:
The line fits okay in the bottom left corner where most of the data lies. That means there is positive correlation between the two data sets. However, that corner clearly shows a correlation between thinking pedophilia is bad and being a skeptic. It tells us nothing about global warming proponents or pedophiles. Similarly, when I said global warming proponents support genocide, I was doing this:
If we put this in words, the argument is:
Skeptics believe genocide and pedophilia are bad. Global warming proponents are the opposite of skeptics so they must believe genocide and pedophilia are good.
Change a few words, and you have Lewandowsky’s argument for why we should believe skeptics are conspiracy theorists:
Global warming proponents believe the moon landing was real. Skeptics are the opposite of them so they must believe the moon landing was faked.
With this corresponding image:
All of these results are “statistically significant.” However, all of these results assume skeptics must hold the opposite view of global warming proponents on all things. Assuming that guarantees the results. We can do that to criticize any group we want. Just follow these simple steps:
1) Ask group X if they think the moon landing was real. They’ll say yes.
2) Assume group Y would answer the opposite way.
3) Conclude group Y believes the moon landing was faked.
You can replace “the moon landing was real” with anything you want. I showed this by doing it with genocide and pedophilia. Had Lewandowsky asked about those, he could have concluded skeptics are pedophiles. He could have probably got it published too. After all, he didn’t do this just once. He published a second paper using the same approach (with a slightly less skewed sample).
And he’s not the only one who uses it. Lewandowsky’s recent paper cites the paper, Dead and alive: Beliefs in contradictory conspiracy theories. That paper argues conspiracy theorists are so loony they’ll accept multiple, contradictory conspiracies. It’s namesake comes from the “statistically significant” correlation between believing Princess Diana and/or Osama Bin Laden was killed in a conspiracy and believing he/she is still alive. The image for this claim would be:
The scale in that image is correct. Let it sink in.
There is no justification for this methodology. Even so, three scientific journals have approved of it. Dozens of scientific articles approvingly cite its results. Half a dozen people have been paid papers using it. It has been promoted hundreds of times in the media. It is widely accepted in the global warming debate. It is complete and utter nonsense, but people like the results so they don’t mind.
And if my suspicions are correct, it’s probably been used in many other papers.
They’re Mod Scientists!
So pedophilia is wicked? Come on, people, what’s so awful about a harmless foot fetish? It may not be exactly a standard deviation, but in Australia “footie” is enjoyed by hundreds of thousand of people.
Maybe millions..
See here:
Chris H says:
September 10, 2011 at 1:11 pm
“To a limited degree Dessler is right in saying that opposition to big government and climate scepticism go together. However, his implication that the one determines the other is incorrect. As Melanie Phillips points out in her book “The World Turned Upside Down”, the liberal left mindset predisposes to a set of values that is in favour of AGW, “green” issues and big government….
“In contrast, those on the right tend to be more pragmatic and look at what works and consider the evidence. As a consequence, AGW scepticism and opposition to the current US government … will go together without one “causing” the other.”
==============
Roger Knights:
If Global Warming Didn’t Exist, It Would Be Necessary to Invent It:
… To lift climatology out of its backwater status …
… To increase research funding for Academia …
… To justify the de facto political empowerment of a sector of the scientific / academic elite, setting a precedent for the subsequent empowerment of other sectors of that elite.*
… To refresh the raison d’être of the EPA & UN …
… To move environmentalism from the fringes to the center of social concern …
… To justify increased media coverage of environmental issues …
… To give enviro-groups a powerful fund-raising and consciousness-raising tool …
… and allow them access to the levers of national and international power …
… To give activist & green parties a vote-getting wedge issue …
… and a case-study justification for their habitual “hammer” (increased regulation and taxation) …
… To provide at-a-loss “engagé/enragé” types with a new stick with which to bash the beastly bourgeoise…
… To transfer wealth from the West to the South …
… To fund alternative energy developers and researchers …
(* See Pareto on “the circulation of the elites.”)
So why not “warm” to global warming, if you’re:
… a climatologist?
… a bigshot in a boffins’ brigade?
… a university administrator?
… an environmentalist?
… an environmental reporter?
… an official of an environmental organization?
… a UN official?
… a socialist?
… a natural-born “true believer”?
… a country in the global South?
… a worker or investor in an alternative energy company?
For such as those, what’s not to like about “climatism”? It’s all upside—a gravy train that’s glory-bound. It would be tempting to get aboard, wouldn’t it? (Especially after your peers did so, threatening to leave you alone on The Wrong Side.)
Told you I was smelling a rat.
Thanks for the analysis.
Now Landowski has been sent into the woods we have this to attend to: http://stevengoddard.wordpress.com/2014/01/24/solzhenitsyn-criticized-stalin-then-sent-to-prison/
Or are we taking a poll on this one too?
I vividly remember my very first stats lesson at college (more years ago than I care to remember) where the lecturer asked us to work through the amount of manure farmers produce compared to the number of cars sold. He then asked if we thought there was any meaningful correlation (of course, there was, but it wasn’t meaningful).
It was a good lesson to learn; it stood me in good stead in later years when I read the Lew papers and realised my lesson in manure stats was not wasted.
rogerknights says “So why not “warm” to global warming, if you’re ….a climatologist? … a bigshot in a boffins’ brigade? … a university administrator? (et al).
Thanks, but I think the “warming” process you described is more of a reasoned political calculation than what I had in mind.
I’m looking for people’s out of-the-box instinctive reaction when they first heard the suggestion that increasing levels of CO2 causes GW. My immediate reaction was along the lines “that can’t be right – the planet’s too large, the human species too small and the system (orbiting our massive local star and with the planet’s molten core beneath us) has run in stable cycles whatever might have happened within a mile or so of the surface”. But a great mate of mine (a PhD and uni lecturer) says (I paraphrase) “it’s happening, it’s going to get much worse..” – the difference being our political views, and I’ve often noticed that first reactions seem to correlate with political views.
Global warming proponents believe the moon landing was real. Skeptics are the opposite of them so they must believe the moon landing was faked.
=============
“Skeptics are the opposite” is of course false. Skeptics believe the opposite on some issues, not all issues.
This logical argument confuses people because the structure of argument itself is logically valid. But the conclusion is false because one of the premises is false.
pat says: @ur momisugly January 23, 2014 at 6:24 pm
it’s all down to the Finance Ministers now!
24 Jan: Guardian Economics Blog: Lord Stern: I should have been fiercer in climate change review…
http://www.theguardian.com/business/economics-blog/2014/jan/23/lord-stern-climate-change-review-davos
>>>>>>>>>>>>>>>>>>>>>>>>>
At the top of the present comment list is this nasty bit:
This is the problem with the crap that people like Lewandowsky put out. Skeptics are not seen as scientists with opposing views but as lunatics and criminals. They are deliberately inciting the mob to riot and kill.
Gail Combs, I posted this reply on the Guardian:
___
So freedom of speech is to be abandoned then?
But then who will tell you when you are wrong? No-one is infallible.
How can we gain an understanding of why the climate models have failed if we don’t allow anyone to ask the question?
___
Interesting to see if it is censored or not in the context.
Stephen Rasey, I see. I think I misunderstood because I did do what you seem to be describing. My initial post discussing my results was this one (republished on a couple sites, including this one). It claimed global warming proponents support genocide (and think they’ve never been wrong).
I then wrote another post which was primarily focused on making replication of my results convenient, but concluded global warming proponents believe they’ve been abducted by aliens. That led to my favorite post, which began:
The point of my series of posts was to begin with a remotely plausible correlation then move on to less and less believable results, eventually culminating in results nobody would believe. Unless I’m misunderstanding you, that’s largely what you had in mind.
Mac the Knife:
It’s possible I will. There are a lot of issues that go into publication choices. One of the largest issues is the problem with Lewandowsky’s work is he applied statistical methodlogies to data that didn’t fit the the requirements of those methodologies. It’s an incredibly basic mistake.
Those can be the hardest things to publish about. When a mistake is that simple, it’s hard to show a discussion of it is worthwhile. The initial reaction is, “Yeah, but that’s such a stupid thing to do, nobody needs to be told not to do it.”
Anyway, I don’t know. I’ll have to look into publication options some. I’m not even sure what (if any?) fees I’d have to pay with that journal. I can just think of things I’d much rather do with the money!
Mindert Eiting, there’s actually no inherent reason to believe asking a question multiple times produces results which more accurately or precisely represent the respondent’s view. Reliability ratings barely help since they only tell you how consistent a response is.
I’ve discussed this issue a bit in relation to Lewandowsky’s latest paper. How should one feel if 20% or more of the data for a survey is inherently contradictory? As in, if one in five people give responses than cannot logically be true, can we really say the additional questions gave us more information? How do we know they didn’t mostly just introduce noise (especially by reducing data quality)? How do we know their combination reflects the view we’re interested in?
Using multiple questions can certainly help. However, it is not inherently better. For example, the 200+ question survey I took in high school would have gotten much more accurate results from me and my friends if it had only been 50 questions long. Instead, it got a bunch of joke responses from us. There’s no telling how our answers would impact any attempt to combine questions.
As with most things, an approach being more sophisticated does not mean it is better. It may be, but one should demonstrate it is before adding the complexity. At the very least, one needs to test that the combination of results gives a coherent picture.
I’m mostly pointing this out because Lewandowsky’s second paper abused the use of multiple questions. It did stupid stuff. One of the weirder examples is it had a “balanced” question set by having reverse scored questions, but not an even amount. If one believes there is a bias, using reverse scored questions is silly if the number of them is different (and you don’t do some other test to check their effect). The paper also used an outdated question set and arbitrarily modified one it took from one of its sources. They didn’t even test to see if any questions were outliers (one was).
There’s no particular reason to believe that would produce more meaningful results than my simple approach.
BoyfromTottenham says: @ur momisugly January 23, 2014 at 6:53 pm
……Given the number of steps taken to achieve the result, surely this has to be the work of one or more persons acting with a purpose (e.g. to create a powerful piece of “disinformation”), rather than honest, scientific research?
>>>>>>>>>>>>>>>>>>>>
“Polls” are used as political weapons all the time.
After the 2010 elections in the USA a poll was specifically designed to defame the Tea Party (I am not a member BTW) This poll was then broadcast far and wide by the Mass Media to show the Tea Party was a party of racist whites. It is still used today and has pretty much killed the Tea Party as a threat to the two party system. This one poll with the willing aid of journalists was a very effective weapon against the emergence of a third party in the USA.
The Tea Party is for reducing the size of the federal government. That is one of the main reasons they came into being. The other is they are opposed to raising taxes. The name means:
Tax
Enough
Already
Party
A set of questions in a poll was asked:
The answers are divided into, Tea Party Whites = TP and Non-Tea Party Whites = OW
The ‘Non-Tea Party Whites’ would include whites who were democrats. Rassmusen finds that 41% of Likely U.S. Voters would vote for the Democrat and thirty-five percent (35%) would choose the Republican in the Congressional Ballot in November. Democrats are for big government in general so you are comparing a population that is around 40% or more for big government to a population that is close to 100% against big government and against more taxes. In that light the findings are rather surprising.
Even though Tea Party members are against big government and against more taxes only 30.7% disagree with the statement “Our society should do whatever is necessary to make sure that everyone has an equal opportunity to succeed.” The answer to that question nullifies the title of the report.
The other trick in this poll is they report STRONGLY disapprove and disapprove combined sometimes and at others only STRONGLY disapprove. If you look at Rasmussen polls you see many people just do not care.
Here is an example of Rasmussen reporting
So most voters do not have strong feeling either way. If you wanted to tease ‘Racist” from these bias questions you would have to look at only the STRONGLY disapprove responses. Even then you would still have confounding with STRONGLY disapprove of big government and STRONGLY disapprove of more taxes.
Maxwell’s Shifting Support for Democratic Party among American Elderly shows she is anything but neutral in her political leanings.
Maxwell is all about Democratic campaign strategy not truth. Lewandowsky is all about the CAGW campaign strategy not truth. The only difference between the two is Maxwell is a real pro and Lewandowsky is a bumbling amateur. Unfortunately the Journalist will still feed the results to the general populus where it will be gobbled up as ‘Truth handed down from on high’
This type of data mining/statistical gibberish is common in the social sciences where a correlation of 0.15 is taken to be meaningful if p<0.01, even though you can get a correlation of 0.30 with random data. It is also common in the "bran muffins cure cancer….no wait, they don't" weekly news item. People (even scientists) want novel results (aka "news") not hard analysis.
Thanks, Brandon, I can follow your argument. As a final point, take the scatter of genocide against threat. In the genocide responses the number in each category drops with about one tenth of the former (I cannot find your article of last Monday in which you gave the frequencies). In the most extreme category (totally agree) we would expect zero subjects but there are over twenty. These divide in two groups with extreme responses on the threat item. I would suggest that these respondents are jokers or people who give blindly extreme responses. Whatever kind of correlation you use, it may express aspects, like response bias, which have nothing to do with opinions. Of course, I am only saying this for sharpening the criticism of Lew’s research.
Let me chime in by complimenting Brandon on a vivid demonstration. By coincidence, both of us had independently obtained the Wood data within a few days of one another. Wood’s Diana claims (cited by Lewandowsky) are a particularly bizarre example.
A number of readers have attempted to assimilate this phenomenon to other issues. The phenomenon is not the same thing as “some loonies are skeptics/warmists”. And while it is a spurious correlation, it is a very specialized form of spurious correlation that warrants sunshine.
Brandon, I experimented with a different technique of showing the response size. I did a scatter plot but used a solid circle for the point (in R, pch=19) and then defined the radius of the circle to be proportional to the count (in R, cex= sqrt(count)). In a black-on-white plot, the small counts were invisible. However, with a sort of “starlight” plot, yellow-on-black, the faint counts were discernible. I don’t remember whether I did a post showing this, but I’ll either do one, or send you an example.
Cheers,
Steve Mc
David, UK says:
January 23, 2014 at 2:48 pm
Hey, maybe I am paranoid but I sense that a goodly portion of the warmist camp thinks that paedophilia is the lesser evil.
Osama Bin Laden believed in CAGW therefore……………
The moon landings were so faked that a group of those astronauts who did not land on the moon are sceptical of CAGW. Makes complete sense to me. By the way the Moon landings happened, it’s fact.
Thanks Brandon
i think all that want to understand get the point now. I thought it was cherrypicking and data manipulation but it is just a poor grasp of statistics. Amazed it passed peer review but not all publications can have decent statisticians at the gates.
Steve McIntyre, thanks. You’re right about this being a specialized form of spurious correlation. I’m still trying to work out how to write it formally. I feel like I should have focused on math instead of computers in school. Intuition is hard to put on paper.
By the way, you did do a post using the technique you describe. I thought it looked better than this approach. I’m just not very familiar with R’s plotting options. I knew something like your cex=count (scaled however you want) would work, but I didn’t know the command/syntax. I’ll have to remember it.
Mindert Eiting, I agree those responses are almost certaintly false. They’re not too important though. Every correlation I’ve discussed in regards to my survey would be “statistically significant” even if you removed all the responses claiming to support/believe crazy things.
That’s the most remarkable thing about this methodology. You could publish a paper saying skeptics are racists without having anyone respond claiming to be a skeptic or a racist. You literally do not need any data for any group you’re drawing conclusions about.
That said, outliers like the ones you highlight do influence the results. It’s important to check how much weight they have on the results. I made sure to do so. I just didn’t post the calculations since I’m trying to keep things simple enough anyone can immediately spot the problems. There are lots of details like that I can post if they become relevant.
By the way, I didn’t post the frequencies for the data, only the correlations. The code I shared would make it easy to get the counts, but it might be better for you to look at this post. manicbeancounter posted some useful frequency tables.
“And if my suspicions are correct, it’s probably been used in many other papers.”
And if my suspicions are correct, it can now be used as toilet paper.
New Journal: Pattern Recognition in Psychology
Peer-Review rules: All papers to be reviewed by persons of the editors’ absolute and final discretion.
Contents: (so far)
Papers’ promoting climate hysteria published with obvious and substantial errors.
Papers’ skeptical of climate hysteria subject to unprecedented publishing obstacles.
Correlation between climate hysteria and disposable income.
MishaBurnett said @ur momisugly January 23, 2014 at 3:03 pm
No he didn’t. Relativity was “Jewish Science”, not “German Science”.