Lewandowsky – call your office: Correlation is Meaningless

Readers may recall this survey: A poll to test the Lewandowsky methodology The results are in, which is why we can’t say global warming proponents support pedophilia.

Guest essay by Brandon Schollenberger

They don’t. The fact there is a correlation (0.14) between believing global warming is a serious threat and saying pedophilia is good is meaningless. The fact this correlation is “statistically significant” (at the 99.99% level) is meaningless. Anyone who looks at the data can immediately see the results are bogus (a small jitter value was added to allow us to see the density of responses):

Fig1

There are only 20 or so respondents (out of over 5,000) who claim to believe global warming is a threat and claim to support pedophilia. It’s likely those responses were false. Nobody can seriously claim that proves global warming proponents are pedophiles.

The issue of false responses received a lot of attention with Stephan Lewandowsky’s paper, “NASA faked the moon landing—Therefore (Climate) Science is a Hoax. In that paper, similarly false responses created spurious correlations. Unfortunately, the focus on false responses meant a more fundamental issue got missed. Namely, the entire idea behind this approach is nonsensical. The approach is like taking the data displayed above and drawing this line:

Fig2

The line fits okay in the bottom left corner where most of the data lies. That means there is positive correlation between the two data sets. However, that corner clearly shows a correlation between thinking pedophilia is bad and being a skeptic. It tells us nothing about global warming proponents or pedophiles. Similarly, when I said global warming proponents support genocide, I was doing this:

Fig3

If we put this in words, the argument is:

Skeptics believe genocide and pedophilia are bad. Global warming proponents are the opposite of skeptics so they must believe genocide and pedophilia are good.

Change a few words, and you have Lewandowsky’s argument for why we should believe skeptics are conspiracy theorists:

Global warming proponents believe the moon landing was real. Skeptics are the opposite of them so they must believe the moon landing was faked.

With this corresponding image:

Fig4

All of these results are “statistically significant.” However, all of these results assume skeptics must hold the opposite view of global warming proponents on all things. Assuming that guarantees the results. We can do that to criticize any group we want. Just follow these simple steps:

1) Ask group X if they think the moon landing was real. They’ll say yes.

2) Assume group Y would answer the opposite way.

3) Conclude group Y believes the moon landing was faked.

You can replace “the moon landing was real” with anything you want. I showed this by doing it with genocide and pedophilia. Had Lewandowsky asked about those, he could have concluded skeptics are pedophiles. He could have probably got it published too. After all, he didn’t do this just once. He published a second paper using the same approach (with a slightly less skewed sample).

And he’s not the only one who uses it. Lewandowsky’s recent paper cites the paper, Dead and alive: Beliefs in contradictory conspiracy theories. That paper argues conspiracy theorists are so loony they’ll accept multiple, contradictory conspiracies. It’s namesake comes from the “statistically significant” correlation between believing Princess Diana and/or Osama Bin Laden was killed in a conspiracy and believing he/she is still alive. The image for this claim would be:

Fig5

The scale in that image is correct. Let it sink in.


There is no justification for this methodology. Even so, three scientific journals have approved of it. Dozens of scientific articles approvingly cite its results. Half a dozen people have been paid papers using it. It has been promoted hundreds of times in the media. It is widely accepted in the global warming debate. It is complete and utter nonsense, but people like the results so they don’t mind.

And if my suspicions are correct, it’s probably been used in many other papers.

Advertisements

  Subscribe  
newest oldest most voted
Notify of

Thanks, Brandon.

Dave

Point understood, but pedophilia?
Really?

milodonharlani

The BBC supports CACA & harbors pedophiles, so not such a statistical stretch.
Ditto PSU.

Jim Clarke

Is Lewandowsky so stupid that he actually believes that his papers are meaningful, or is he so smart in believing that there are many who are stupid enough to accept his work? I think it is a fundamental tenant of the anointed academics that their personal brilliance is so great, that it is morally acceptable to lie to the ‘ignorant masses’ at will, as long as it is for a ‘good cause’.
Ironically, the ignorant masses see through this in a heart beat. It is his fellow academics who appear to be persuaded by this garbage.

Henry Bowman

Wow, it’s amazing that someone has the audacity to a publish such obvious dreck. Even if the methodology was correct (which it is not), a correlation of 0.14 is as close to meaningless as anything I’ve ever seen. Personally, I never believe correlations less than about 0.90. These folks seem deranged.

Bob Tisdale, it was my pleasure! Except for the rewriting part. This is actually the fourth version of the post. I kept trying to find ways to make the mathematical underpinnings of the post interesting for lay readers. That was painful. I’m glad I decided to take a simpler approach.
Dave, yup. I included genocide in the survey so I’d have something bad to “pin on” warmists, but I always knew I’d use pedophilia as the transition into showing why this approach is despicable. I wanted to show this approach lets you “scientifically prove” the people you dislike are the most vile things you can imagine.
That is, if we accept this methodology, anyone can be labeled a pedophile via “science.”

Is it possible Lewandowski is so stupid he actually believes in the validity of his technique?

David, UK

Dave says:
January 23, 2014 at 2:14 pm
Point understood, but pedophilia?
Really?

Did not get the point? I’ll spell it out: It’s easy to demonise a sector of society with an abuse of statistics.
(That sector of society is sceptics, not paedophiles – thought I’d better spell that out for you.)

Mindert Eiting

It’s quite easy to get some subjects from the dense block at the lower left to the block at the upper right. You have always some people who don’t understand the instruction and mark ‘agree very much’ when they mean ‘disagree very much’. Sometimes they are called outliers. BTW, are the significances based on the assumption of normally distributed variables?

Hitler believed in the theory of gravity! That proves that gravity isn’t real.
I mean, seriously, it seems that warmists will try anything to avoid any discussion of what the actual facts are and what conclusions, if any, can be drawn from those facts. The personality of a believer is not a test for truth. Even if it were true that every single person who denies AGW believed that the moon landing was faked it would have no relevance to whether or not the AWG theory is true.

Mindert Eiting, yup. The fact the data isn’t remotely close to having a normal distribution is actually why one can get “statistically significant” results that are obviously nonsensical. One could use transformations to make the data have a normal distribution then repeat the calculations. The results be wildly different.
Earlier versions of this post discussed that, but they dragged on too much. I couldn’t find a way to keep people from falling asleep while I explained, mathematically, what the problem was, how one could address it and how doing so would change the results.

flyingtigercomics

Socrates is a fish.
All men are mortal.
Therefore Socrates is mortal.
Because Obama.

F.A.H.

It also speaks to the “peer review” process. Clearly the reviewers of such papers either do not possess or choose not to exercise the same level of critical thinking as evident in this straightforward but insightful illustration.

jorgekafkazar

“However, all of these results assume skeptics must hold the opposite view of global warming proponents on all things.”
Really? It’s worse than I thought, then.

What is the meaning of “paid papers ” ?

F.A.H., that reminds me of an important point. I discussed the point I made in this post with Michael Wood, author of the last paper I cited in this post. I posted about our exchange here. I challenge anyone to read his responses to me and not be amazed. The part I find most mind-boggling part is when he said:

This is because we don’t have any reason to believe – either in the case of Diana or this hypothetical abortion study – that there is some discontinuity at the midpoint of the scale that changes the form of the relationship between the two variables (there’s no shortage of examples of people holding mutually contradictory beliefs at the same time), and in fact if we had a larger data set we would by default test for a linear effect anyway.

He openly acknowledged his results came from one end of the scale but argued that’s okay because we don’t know there’s a discontinuity that changes the relationship between the variables. That’s like someone using the tactic I described in this post then saying, “Well, we don’t know skeptics aren’t pedophiles so it’s okay to use this methodology.”
He basically admitted the problem then said it isn’t a problem because I can’t prove his baseless assumption false.

garymount, a mistake when editing. I must have accidentally erased “to write” from that sentence.

Joe

jorgekafkazar says:
January 23, 2014 at 3:15 pm
“However, all of these results assume skeptics must hold the opposite view of global warming proponents on all things.”
Really? It’s worse than I thought, then.
———————————————————————————————————–
Which proves that you’re a warmist, cos that’s what they keep saying.

manicbeancounter

Brandon gave a more theoretical explanation of the dot analysis here.
Although the survey was a bit of fun, to demonstrate that Lewandowsky’s correlations were meaningless, the first three questions on opinions on global warming were:-

1.       Do you believe global warming is a real?
2.       Do you believe humans are responsible for most global warming?
3.       Do you believe global warming poses a serious threat?

The “Moon Landings” paper had referred to the Doran and Zimmerman 2009 opinion survey to show that skeptics were at odds with the consensus of climate scientists. This was the first paper that claimed the magical 97% figure. The two questions asked were:-

1. When compared with pre-1800s levels, do you think that mean global temperatures have generally risen, fallen, or remained relatively constant?
2. Do you think human activity is a significant contributing factor in changing mean global temperatures?

Compared with Doran and Zimmerman, Schollenberger
– Asked about belief in the predicted future climate catastrophe.
– Allowed for a much larger range of positions, not just agree or disagree.
In particular from the results of the Schollenberger survey, it is possible to separate the catastrophists from those who believe humans are warming the planet, but see this as a trivial issue.

clipe
richardscourtney

Brandon Shollenberger:
Thankyou, that is a clear, insightful and cogent explanation of the flaws in the analytical method used by Lewandowsky.
Please submit a paper explaining those flaws to the journal ‘Psychological Science’ which published the Lew. paper.
Richard

manicbeancounter

Jim Clarke says: January 23, 2014 at 2:34 pm

Is Lewandowsky so stupid that he actually believes that his papers are meaningful, or is he so smart in believing that there are many who are stupid enough to accept his work?

I think you should read Steve McIntyre’s post More False Claims from Lewandowsky to answer the question for yourself.

Dan in Nevada

So some warmists aren’t genocidal pedophiles?

richardscourtney, I’ve considered that, but I believe I’d have to pay a fee if Psychological Science was publishing it. I don’t like the idea of spending my money that way. Part of it is simple finances. I’m not getting paid to do this. Paying to publish would mean I’m losing not just time, but money.
But the biggest part is they screwed up, badly. They shouldn’t get to make money off forcing people to do work to correct their screwups. The only way I could see wanting to publish with them is if they gave a waiver of any and all fees. Otherwise, I’d want to publish somewhere else.

Gail Combs

Henry Bowman says: @ January 23, 2014 at 2:36 pm
Wow, it’s amazing that someone has the audacity to a publish such obvious dreck….
>>>>>>>>>>>>>>>>
It is even more amazing that Stephan Lewandowsky not only got his papers using this method peer-reviewed but the mass media then used the papers to paint skeptics as lunatic conspiracy nuts.
Brandon Schollenberger does a beautiful job of satirizing Lewandowsky’s method and results by turning the tables on him and the Warmists.

Kpar

Brandon, this was both the most outrageous (in the raging use of the term) and hysterically funny article I have read in a looong while. Have you really found that many “scientific” journals who support this nonsense?
Many Thanks.

Txomin

Stay away from pay-to-publish journals.

richardscourtney

Brandon Shollenberger:
Thankyou for the reply to me which you provide to me at January 23, 2014 at 4:05 pm.
OK. I understand that and it makes sense. Please let me know if I can help with publication elsewhere.
Richard

R. de Haan

Garbage in, garbage out

“Skeptics believe genocide and pedophilia are bad. Global warming proponents are the opposite of skeptics so they must believe genocide and pedophilia are good.”
Obviously you can’t assume the two groups have no overlapping beliefs and just ‘invert’ beliefs like that. I’m pretty sure that’s not what Lewandowsky does though. It’s hard to tell because I found his paper to somewhat hard to follow (what is his ‘Loading’ variable for example. Weighting? Normalization factor?)
I downloaded and looked at the data myself in a way I can understand and there does seem to be obvious group correlations. This shouldn’t really be surprising or alarming though and I see no point in attacking it. It’s not imaginary, it’s not a figment of data processing, the relationship is there, but so what? For example, few would deny the relationship between CAGW belief and belief in the value of economic freedom (or freedom in general). It’s a pretty obvious and strong correlation. Every group has it’s fringe. Some on the left probably are more tolerant of pedophilia as was evidenced by the defense of Roman Polanski a couple of years ago. Those who suspect deception in every government statement and action (conspiracy theorists) are more likely to think global warming is a nefarious plot as well. That’s hardly surprising. These belief clusters do exist. It’s real. Again though, so what?
Lewandowsky and many others in the climate community are being childish by even going down this route. It’s an attempt to stereotype everyone in a group as being fringe by identifying that a group does have some fringe. It’s equivalent to saying “crime is highly correlated with black people, therefore blacks are criminals.” Statically, blacks do commit more crimes per individual, but to make such a blanket statement about ALL blacks is clearly illogical and racist. The same goes for labeling ALL skeptics. It’s a game unworthy of playing.

Peter Miller

i do not know if this is relevant, but in the world of natural resources, all observations in one particular deposit should fit a log normal distribution. if they don’t, then the data is almost certain to be fraudulent.
Is that relevant in any way in regards to Lewandowsky’s ‘research’ findings?

Latitude

I think everyone puts way too much faith in peer review..

george e. smith

Well, as I have pointed out before, statistical mathematics is a rigorous, and exact mathematical discipline, just like trigonometry, or Euclidean plane geometry . Text books define and explain the various and sundry concepts of statistics, from means and quartiles, to standard deviations, and confidence levels.
These are all defined properties of ANY data set of already exactly known elements or numbers; they are tabulated in the data set.
The numbers in the data set, could be calculated rigorously from some closed form algebraic equation, or they can be numbers, no two of which are related to each other in any way. They could be the weights of pieces of concrete being carted away from a recently imploded building site.
The results of any statistical analysis, of any of these completely known data sets, are equally valid, if the algorithms were applied correctly in all cases. They are an intrinsic property of that data set.
And, they can tell you exactly nothing, about anything else not in that data set; nor can they predict ANYTHING, that is not already known.
Statistics, is just about as enlightening, as contemplating the lint in your navel ! But it is still a rigorous branch of mathematics.

Ian Schumacher, why are you “pretty sure that’s not what Lewandowsky does”? I demonstrated that’s exactly what his methodology does. He may not intentionally assume that, but his methodology does. You can do that by calculating the source of the correlations. I’ve posted an example of that (with Wood’s paper instead of Lewandowsky’s). You can do the same with any of the data sets.
Peter Miller, that is relevant. Simple correlation tests like those being discussed here assume a normal (multivariate) distribution. It doesn’t indicate fraud when the data isn’t, but it does indicate the tests are inappropriate. One would normally try to transform the data to fit that distribution before applying such tests.

FrankK

The biggest load of Dog pooh I have ever seen. It doesn’t warrant any further comment.

Colorado Wellington

David, UK says:
January 23, 2014 at 2:48 pm
… I’ll spell it out: It’s easy to demonise a sector of society with an abuse of statistics.
(That sector of society is sceptics, not paedophiles – thought I’d better spell that out for you.)

Heh.
I’m glad you cleared it up, David. I know people who would not want to see any poor misguided pedophiles demonized by association with vile CAGW skeptics like me.

Mike Maguire

I often refer to our politicians as “elected magicians” because of the way they can put a spin, twist or manipulate situations to make it sound like they and their positions are the right ones for you to support.
This is how the world works. Billions are spent effectively advertising products to consumers that pays off handsomely because consumers……….and voters can be tricked into believing something that sounds convincing.
Most people don’t have the actual data needed or analytical skills to interpret it in fields that they are not experts on………….so, they trust the experts that sell the best sounding/most convincing and appealing sounding position, slogan or products.
But very often, this can only work for so long. If you pay good money for a piece of shiiit, after it conks out, you realize it was a piece of shiiit.
If you elect sometbody that makes a bunch of promises and………….never mind on that one, somehow politicians aren’t held to very high standards.
However, regarding the global warming/climate change scare that was based on numerous very convincing sounding statements, positions, predictions and proposed costly solutions(some already in effect) time has already run out. Even faithful, brainwashed followers of global warming religion will be/are now having a hard time not seeing they bought a big piece of climate shiiiit.

Brian H

Edit: have been paid papers have been paid [for] papers
Sceptics prefer oranges. Therefore Believers prefer to suck lemons. Or is it vice versa?

Colorado Wellington

Dave says:
January 23, 2014 at 2:14 pm
Point understood, but pedophilia?
Really?

Yes. The principle is sometimes called “punch back twice as hard” .
h/t Glenn Reynolds
http://pjmedia.com/instapundit/86990

pat

it’s all down to the Finance Ministers now!
24 Jan: Guardian Economics Blog: Lord Stern: I should have been fiercer in climate change review
Global temperatures are set to be 4-5C higher in the next century and governments are fooling themselves if they think this will only have a modest impact on their economies, says Stern
Posted by Larry Elliott in Davos
Stern is in Davos beating the drum for an organisation called the New Climate Economy, headed by the former president of Mexico, Felipe Calderón. Its aim is persuade finance ministers rather than simply environment ministers that tackling climate change should be a top priority.
This makes sense. Finance ministers hold the purse strings and dictate economic policy. They are much higher up the political food chain than environment ministers…
Secondly, he says that someone is going to win the green race and at the moment the likeliest candidate is China…
Finally, Stern says things have moved on in the eight years since his review. “I would have been much fiercer”, he says.
***“Emissions have gone up faster than I thought and some of the effects of global warming are coming through more quickly, such as melting of the glaciers and the polar ice caps. But technical change has been faster too.”…
http://www.theguardian.com/business/economics-blog/2014/jan/23/lord-stern-climate-change-review-davos

What are we really sceptic about?
The basis for the CAGW idea is the political decided UNFCCC.
Would it not be smarter to label ourself as UNFCCC sceptics?

pat

READ ALL: 23 Jan: World Bank: World Bank Group President: This Is the Year of Climate Action
Story Highlights:
At the World Economic Forum in Davos, President Jim Yong Kim called for a price on carbon, requiring companies to disclose their climate risk exposure, and greater investment in green bonds in the fight against climate change…
Kim also called for doubling the market for green bonds, which support climate adaptation and mitigation projects such as renewable energy, energy efficiency, and carbon reduction, to $20 billion this year and $50 billion by the time a new international climate agreement is reached in Paris in 2015. He urged institutional investors to commit to green bonds targets in their portfolios…
http://www.worldbank.org/en/news/feature/2014/01/23/davos-world-bank-president-carbon-pricing
23 Jan: Blue&GreenTomorrow: ‘This is the year to fight climate change’, World Bank president tells finance world at Davos
However, Kim stressed that financial leaders must also lead sustainability efforts.
“The so called long-term investors must recognise their fiduciary responsibility to future pension holders who will be affected by decisions made today”, he said.
“Corporate leaders should not wait to act until market signals are right and national investment policies are in place. Be the first mover. Use smart due diligence. Rethink what fiduciary responsibility means, in this changing world. It’s simple self interest.”
He added, “Every company, investor and bank that screens new and existing investments for climate risks is simply being pragmatic.”
Kim also called on financial regulators to enforce disclosure of climate risks, requiring companies and financial institutions to assess their exposure to climate related impacts…
The World Bank president called for the use of green bonds, as described in the Green Bond Principles released this month, to “expand the universe of investors who are investing in green assets”, and called on institutional investors to “commit to purchasing specific significant amounts of green bonds for their portfolios.” …
http://blueandgreentomorrow.com/2014/01/23/this-is-the-year-to-fight-climate-change-world-bank-president-tells-finance-world-at-davos/

I sent an e-mail to both Stephan Lewandowsky and Michael Wood to notify them of this post. Based on my previous exchanges with them, I don’t expect a response, but it’ll be interesting to see if they say or do anything.

TimTheToolMan

Lewandowsky is the Paris Hilton of the “science” world. Not really good at anything except getting his name in the spotlight.

Brandon Shollenberger,
I’ll take your word for it. Unfortunately for me, Lewandowsky’s paper is a bit like jumping into the middle of a conversation and in some real sense, because he relies on a lot of previous work, I guess it is. I’m not familiar with structural-equation modeling (SEM) at all and since this paper relies heavily on it, I’m kind of lost.
I do believe though that there are ‘clusters’, which I think are the best way to look at it (although maybe that’s just because I understand that approach) 😉
Either way, one is likely to find real correlation. I guess that’s my concern. We may be attacking Lewandowsky’s method when even a proper approach would still identify that, yes indeed, people that believe in conspiracy theories are also more likely to believe CAGW is a hoax. That is just a statistical reality. Both sides have their fringe members to be sure (there are more than enough in the world to go around). The fallacy is in then implying ALL skeptics should be treated as though they are fringe members. I would hope that fallacy is pretty obvious to all and so doesn’t really warrant refutation. I doubt even those that use this argument believe it themselves. They just want to annoy their ideological opponents 😉

Merovign

I think I’ve permanently sworn off answering any poll that does not include a “your question is based on a flawed premise” response.

ossqss

Amazing stuff Brandon. Thanks.
I had the opportunity to watch what real science and research can do [tonight]. I watched the atlas [rocket] launch at 9:33 eastern time. Amazing stuff, and confirmed by observation through technical scruitiny. Unlike Mr. LEW’S products of science by proxy.

BoyfromTottenham

Eric Worral said “Is it possible Lewandowski is so stupid he actually believes in the validity of his technique?”. I say “given what we know about how the “survey” was constructed, undertaken, and “analysed”, how the heck can anyone answer “Yes” to Eric’s question? Given the number of steps taken to achieve the result, surely this has to be the work of one or more persons acting with a purpose (e.g. to create a powerful piece of “disinformation”), rather than honest, scientific research?

Merovign

Also, as a side note, a lot of responses assume people’s positions are thought out on a rational basis, even if the rationale ifs flawed.
A surprising number of people live on the “my team vs your team” basis, or “tribalism” to put it another way.
Roman Polanski was a member of the Hollywood Tribe, I don’t think most of his supporters spent much more time than that thinking about it. Some surely did, some surely didn’t, but my limited expose indicates the latter is more common.

u.k.(us)

I stopped reading after the second “pedophilia” reference.
Still stopped.