I and (according to Cook) 50 other blogs (with a supposed 50/50 skeptic to advocate split) have received this invitation:
Hi Anthony
As one of the more highly trafficked climate blogs on the web, I’m seeking your assistance in conducting a crowd-sourced online survey of peer-reviewed climate research. I have compiled a database of around 12,000 papers listed in the ‘Web Of Science’ between 1991 to 2011 matching the topic ‘global warming’ or ‘global climate change’. I am now inviting readers from a diverse range of climate blogs to peruse the abstracts of these climate papers with the purpose of estimating the level of consensus in the literature regarding the proposition that humans are causing global warming. If you’re interested in having your readers participate in this survey, please post the following link to the survey:
[redacted for the moment]
The survey involves rating 10 randomly selected abstracts and is expected to take 15 minutes. Participants may sign up to receive the final results of the survey (de-individuated so no individual’s data will be published). No other personal information is required (and email is optional). Participants may elect to discontinue the survey at any point and results are only recorded if the survey is completed. Participant ratings are confidential and all data will be de-individuated in the final results so no individual ratings will be published.
The analysis is being conducted by the University of Queensland in collaboration with contributing authors of the website Skeptical Science. The research project is headed by John Cook, research fellow in climate communication for the Global Change Institute at the University of Queensland.
This study adheres to the Guidelines of the ethical review process of The University of Queensland. Whilst you are free to discuss your participation in this study with project staff (contactable on +61 7 3365 3553 or j.cook3@uq.edu.au), if you would like to speak to an officer of the University not involved in the study, you may contact the Ethics Officer on +61 7 3365 3924.
If you have any questions about the survey or encounter any technical problems, you can contact me at j.cook3@uq.edu.au
Regards,
John Cook
University of Queensland/Skeptical Science
I asked Cook a series of questions about it, because given his behavior with Lewandowsky, I have serious doubts about the veracity of this survey. I asked to see the ethics approval application and approval from the University, and he declined to do so, saying that it it would compromise the survey by revealing the internal workings. I also asked why each of the 50 emails sent out had a different tracking code on it, and he also declined to explain that for the same reason. I asked to see the list of 12,000 papers, so that I could see if the database had a true representation of the peer reviewed landscape, and he also declined, but said the list would be posted “very soon”.
I had concerns about the tracking codes that were on each email sent out, and I ran some tests on it. I also tested to see if they survey could be run without tracking codes, it cannot and I asked him if he would simply provide a single code for all participants so that there can be no chance of any binning data by skeptic/non skeptic blogs or any preselection of the papers presented based on the code. I said this would truly ensure a double blind. He also declined that request.
He stated that he had an expectation (based on past experience) that no skeptic bloggers would post the survey anyway. So why send it then?
Meanwhile many other bloggers shared their concerns with me. Lucia posted a large list of questions about Cook’s survey methodology here:
http://rankexploits.com/musings/2013/dear-john-i-have-questions/
It is a good list, and Lucia’s concerns are valid.
Brandon Schollenberger writes at Lucia’s in comments about some tests he did:
========================================================
Brandon Shollenberger (Comment #112328)
May 3rd, 2013 at 12:48 am
For those following at home, the issue I wanted to talk to Lucia about is the non-randomness of this survey. I was curious when two people at SkS said they got an abstract which said (in part):
Agaves can benefit from the increases in temperature and atmospheric CO2 levels accompanying global climate change
I got the exact same abstract when I clicked on the link at SkS. I wondered if that meant there were only 10 abstracts being used at all. I then had a disturbing thought. The earlier Lewandowsky survey had different versions sent to different people for publishing. What if they had done that here? What if each site was sent a link to 10 different abstracts?
To test this, I contacted lucia to get the link she was sent. I then was able to find a site which had already posted the survey, and I got a different link from it. It turned out all of them resulted in me getting the same survey. I concluded everyone was simply getting the exact same 10 abstracts.
I was going to post a comment to that effect when lucia told me she did not get the Agave abstract I referred to. That made me take a closer look. What I found is by using proxies, I was able to get a number of different surveys. Moreover, some proxies got the same surveys as others. That suggests the randomization is not actual randomization, but instead, different samples are given based on one’s IP address.
Unfortunately, that’s not the end of the story. I’ve followed the links with my original IP address again, and I now get a different sample. However, each time I follow the link with the same IP address now, I get the same sample. That suggests I was right about IP addresses determining which sample you get, but there’s an additional factor. My first guess would be time, but if that’s the case, it’s a strange implementation of it. It would have to be something like an hourly (or even daily) randomization or some sort of caching, neither of which makes any sense to me.
Anyway, my head hurts from trying to figure out what screwy “randomization” John Cook is using. I know it’s nothing normal, and it certainly isn’t appropriate, but trying to figure out what sort of crazy thing he might have done is… difficult. I have no idea why he wouldn’t just use a standard approach like having time in seconds be a seed value for an RNG that picks 10 unique values each time someone requests a survey from the server.
=============================================================
So it appears non random after all and has what I (and others) consider fatal sampling issues.
If you want to look at the survey, you can go to Cook’s website and take it there, because until there are some answers forthcoming, like Lucia, I won’t be posting the coded link for this blog.
See Cook’s survey link: Participate in a survey measuring consensus in climate research
It seems like a set up. They choose the material, and invite you to step in it.
Steve Goddard posted his earlier this morning. However, given the shenanigans of the Lewandosky survey, I prefer not to participate in another Hoax.
The problem with Tim Cook’s survey may or may not be tracking codes. But it is definitely one of integrity. Or more precisely, the lack thereof. The unanswered questions are evidence of that as well.
John Cook prefers pdeudoscience to real climate science. I tried to educate him but he censored and then completely silenced and blackballed me from his web site.
I wouldn’t touch a survey run by John Cook if they paid me. What part of “WE DON’T TRUST YOU ONE BIT” escaped his notice last time?
w.
This is in no way research. When you pull the crap this guy and his associates pull, nothing they do has any credibility. They are activists pure and simple. If that makes me a conspiracy theorist, so be it.
Engaging these people has become too draining. They arise, zombielike with their old-hat guesswork, regurgitated myths and claims of consensus, all the while throwing insults at any who disagree with their smug surety.
Their lack of honesty has been exposed, multiple times. Their activism is all they have – they project and dissemble to maintain their world view and nothing will alter their mindset.
I have given up commenting on their blogs and articles for a couple of reasons additional to the above. The Hydra was easier to kill than these bandits’ ideations. I no longer feed their egos, their fears or their trolling
The results of this little poll have already been determined and if you refuse to join in it is because you are a fake skeptic, in The Nile, suffering cognitive dissonance or Dunning-Kruger, afraid to face a future without your fossil fuel powered toys, paid by a fictional entity, old, not-a-scientist, a right-wing extremist, in the employ of a corporation, weak minded, fearful of the results, biased against the Team, not in full command of your faculties, not in full command of the science, …..
Even if Cook didn’t “Cook” the books, this study amounts to little more than the logical fallacy of appealing to popularity. Of course we know that Cook has constructed the study, so as to prove we skeptics are raving lunatics. Perhaps if he was interested in real science, his readers would be better served by an online library of all pro and skeptical AGW papers.
It’s a trap.
Cook’s methodology may not be quite as bad as suspected.
But the Cook-Lewandowsky combine has destroyed its credibility.
After Lewandowsky, Oberauer, and Gignac (Psychological Science, 2013), and Lewandowsky, Cook, Oberauer, and Hubble-Marriott (Frontiers in Psychology, in limbo), no one should trust them to design, conduct, or analyze survey studies of minimum acceptable quality.
And apparently no one will.
Cook is wasting enough time of my life just by occupying blog space reporting on his shenanigans. I’m not inclined to throw away another 15 minutes to further them.
Just took a look at the survey and the comments.
Apparently the survey returns a value for your evaluation based on the following point system:
Endorsement (of AGW theory) – 1 pt
Implicit endorsement – 2 pts
Explicit endorsement – 3 pts
Neutral – 4 pts
Explicit rejection – 5 pts.
Implicit rejection – 6 pts
Rejection (of AGW theory) – 7 pts.
Your score is then compared with the authors own evaluation.
One example I saw was 8 neutral + 2 explicit endorement / 10 = 3,8 pts.
Authors own evaluation 2,8 pts.
“I am now inviting readers from a diverse range of climate blogs to peruse the abstracts of these climate papers with the purpose of estimating the level of consensus in the literature regarding the proposition that humans are causing global warming. ”
His premise is to determine “level of consensus”, as if consensus in itself is proof of anything. Just ask Giordano Bruno who was persecuted by the Church for his heretical views, including his belief that there are countless inhabited worlds around distant stars. He also advocated the Copernican system and when he wouldn’t submit to the consensus of the time, he ended up being burned at the stake.
Remember when dealing with Cook and his cohorts: fool me once shame on you, fool me twice shame on me.
Beware of a lefty extending an olive branch or an offer to be “fair.” The other hand wields a knife. (If only the GOP would learn this.)
A suggestion (which I would do myself if I had anything close to time or experience in the matter):
Why not do a ‘study’ ourselves? But do it open. I mean, follow the same methodology, but keep it all out and accessible. My kids make FB surveys all the time, so there is no rocket science to this, mostly it would be setting up some questions, and we could even have a 5 point scale for how much people agree or disagree with statements.
Internet surveys are very easy to do. The data collection is automatic. Setting it up in the open, and then using a known randomizer program would get past any of these un-answered questions.
There might be some argument on the method, but the result should be as good or better.
This, and any survey like it, is completely useless. Regardless of any attempt to create a random distribution of the survey, you will result in a voluntary response. It is basic, elementary, statistics that a sampling reliant on voluntary responses is biased. The clear result of this particular survey will demonstrate that X% of respondents agree with John Cook’s opinion that humans are causing global warming – In other words, he is conducting a survey to find out how many people agree with him. This is exactly the same biased method that obtained the 97% consensus in the first place.
Academic papers might be a population but it is not a homogeneous population. 10 randomly selected papers from that population will tell you nothing definitive about population characteristics except that they are 10 randomly selected papers that successfully got through peer review. Population characteristics might be as large as the population of papers. If that is the case then no sample can be representative of the population. Of course, this says nothing about peer review and how peer review might have been captured by a popular delusion.
Willis got the sensible answer: not with a 10ft pole.
Apparently in an abstract you can say anything you want, even if your research doesn’t support your assertions. So the whole exercise is ludicrous. I suspect Cook has already done a survey of the papers. There’s a website somewhere that points out the discrepancies between what the abstract states (global warming topics) and what the paper actually proposes. I believe that website might have been featured here. I’ll try to find and post later.
I smell a set up, Anthony….
“NavarreAggie” at Goddard’s site reports that the survey did not accurately record his assessments.
How about a survey of climate govt grant awardees on whether their grant proposals have a consensus about AGW? The consensus is these folks do whatever it takes to get the grant money, and the last 15 years has been paying for carbon tax support.
There is no need to be worried about ;saving’ each initiative by someone as academically compromised as this Mr Cook. The value of his work is informed by the value of his earlier works which were manipulated and badly flawed as was evident to anyone who took the time to investigate. As usual, the assessments will have a Nixonian level of ‘plausible deniabilty’ turned on its head to show an AGW consensus is ‘plausibly undeniable’. Excuse me while I barf.
Like nearly all AGW promotion, it already smells of bias and cherry picking with a foregone conclusion pre-written and waiting for CTRL+v to apply it to the result.
A credible survey has to start with credible ethics, credible methods and a credible corresponding author. Call me where there is one.
Why does the science of Antropogenic Global Warming need surveys to bolster its credibility? I thought it was all first principles?
Lew’s writing a conspiracy paper about your comments already…
Speech Recognition Problems
“I have series doubts…” Is that a time series? 🙂 When speech recognition understands context I will know we’ve made progress.