John Cook's 97% consensus claim is about to go 'pear-shaped'

Analysis of raters in Cook’s 97% paper by Shollenberger

pear shaped (slang)

A British expression used to indicate that something has gone horribly wrong with a person’s plans, most commonly in the phrase “It’s all gone pear shaped.”The OED cites its origin as within the Royal Air Force; as of 2003 the earliest citation there is a quote in the 1983 book Air War South Atlantic. Others date it to the RAF in the 1940s, from pilots attempting to perform aerial manoeuvres such as loops. These are difficult to form perfectly, and are usually noticeably distorted—i.e., pear-shaped.

Dr. Richard Tol writes about a new revelation coming from an analysis of Cook’s climate publications volunteer raters, conducted by Brandon Shollenberger:

My comment on Cook’s consensus paper has at last been accepted. It was rejected by three journals — twice by Environmental Research Letters and once by two other journals for being out of scope. Fifth time lucky.

As these things go, my comment is out of date before it is published.

One of my main concerns was the partial release of data. The data that was available suggests that all sorts of weird things were going on, but without the full data it was hard to pinpoint what went on. Cook’s resistance to release the data, abetted by the editor, the publisher and the University of Queensland, suggested that he may have something to hide.

Brandon Shollenberger has now found part of the missing data.

Unfortunately, time stamps are still missing. These would allow us to check whether fatigue may have affected the raters, and whether all raters were indeed human.

Rater IDs are available now. I hope Shollenberger will release the data in good time. For now, we have to do with his tests and graphs.

His comment of May 10, 1:16 am shows that individual raters systematically differed in their assessment of the literature. This is illustrated by this figure; the circles are aligned if the raters are the same.

This undermines Cook’s paper. Theirs was not a survey of the literature. Rather, it was a survey of the raters.

Source: http://richardtol.blogspot.co.uk/2014/05/the-97-consensus.html

Of note is the comment “Brandon Shollenberger has now found part of the missing data.”. While I don’t know for sure, it seems that the SkS kidz have left another gaping security hole wide open which allowed Shollenberger (and likely anyone, as we’ve seen before with their forum fiascos) to have a look at that rater’s data. Cook has been resisting requests to provide it.

Shollenberger writes in comments at his blog:

I’ve sent John Cook an e-mail alerting him to what material I have, offering him an opportunity to give me reasons I should refrain from releasing it or particular parts of it. I figure a day or two to address any potential privacy concerns should be enough.

His response will determine how much information I provide. No obligations were placed upon me regarding any of the material I have, but I don’t see any compelling reason to provide information about how I got it either. I’d need a better reason than just satisfying people’s curiosity.

But we’ll see what (if anything) Cook says. I said I’d give him the weekend. If I don’t hear anything tonight, I’ll try contacting him via Twitter/Skeptical Science. I may try having someone else from SkS get his attention for me. I don’t want him to simply overlook the e-mail I sent.

By the way, there is some value in associating ids and names. We have comments from many of the people who participated in the study. It could be useful to try to match up biases in the ratings with people’s stated views.

Tick Tock.

 

 

Get notified when a new post is published.
Subscribe today!
0 0 votes
Article Rating
181 Comments
Inline Feedbacks
View all comments
May 10, 2014 11:13 am

{all bold emphasis mine – JW}

Shollenberger writes in comments at his blog:,
His [Cook’s] response will determine how much information I provide. No obligations were placed upon me regarding any of the material I have, but I don’t see any compelling reason to provide information about how I got it either. I’d need a better reason than just satisfying people’s curiosity

– – – – – – – –
Brandon Shollenberger,
That turn of phrasing implies fairly reasonably that you got from a person(s) the “part of the missing data [from Cook’s consensus paper]”. It implies you didn’t just find the data.
After you duly consider any potential harm to the raters by making their names and IDs public, I do think it would be valuable in assessing bias if the names and IDs of the raters in the data you have were made public.
John

Evan Jones
Editor
May 10, 2014 11:16 am

I don’t mean to start a pond war, but why do so many Americans think everything was started there? It reminds me of a conversation I heard a few years back.
It was all really inwented in Russia.
An American woman was talking to an English woman, and remarked on the Peter Rabbit books. “Do you have Beatrix Potter in England?” asked the American lady. The English woman just groaned.
Uhggh. Todd rocks!

John F. Hultquist
May 10, 2014 11:18 am

The 97% story just keeps going on and on and ….
… and speaking of rabbits, Ghost 10:17 asks why so many Americans think everything started there.
Many groups of people that become organized (a tribe?) and name themselves use a word or phrase that translates as “the people” and their beginning or origin story starts the history or timeline of what they know. For example, when Gouverneur Morris wrote the words “We the People … do ordain … the United States of America” – history began. It is that simple.

Morph
May 10, 2014 11:24 am

What shape does a pear go when it is going right ?

Mike Bryant
May 10, 2014 11:27 am

My lovely wife, who is absolutely British, believes that the American penchant for claiming all things British as our own is quite endearing. She says that Americans have grown up with so much British culture, including ancestry in many cases, that our cultures have melded into one. I agree.

Twobob
May 10, 2014 11:29 am

Tits up, is better than being over a barrel.

The definition Guy
May 10, 2014 11:29 am

First it was global warming, then it became climate change. Now it has been downgraded to climate disruption.
I propose we adopt a new, more accurate name, “Climate Intransigence.” Because the climate steadfastly refuses to do what it’s been told to do.
It just might catch on!

May 10, 2014 11:35 am

The term ‘pear shaped’, from reading mythology from Frazer’s ‘Golden Bough’ and Joseph Campbell’s ‘The Hero’s Journey’ (monomyth), has meant to me the mature fertile female form in mythology. It is a wonderful symbol.
When the term is applied to a male, it is pejorative.
John

Trond A
May 10, 2014 11:40 am
Gary Hladik
May 10, 2014 12:08 pm

Mike Maguire says (May 10, 2014 at 11:01 am): “In a world that gives Al Gore a Nobel Peace Prize and an Emmy for his movie…”
Two Oscars: one for “Best Feature Documentary” and one for “Best Original Song” (!?).
http://www.imdb.com/title/tt0497116/awards

Keith
May 10, 2014 12:15 pm

Climate intransigence is cool I like it

Admad
May 10, 2014 12:33 pm

renminbi
May 10, 2014 12:44 pm

Toes up- a corpse on an autopsy table with the little ID tag on the toe. Maybe tits up is the euphemism for that.

gbaikie
May 10, 2014 12:57 pm

He’s dead, Jim

Jake J
May 10, 2014 12:57 pm

When all of this comes out, I really hope that Dr. Tol has written it in a way understandable to an intelligent non-specialist. Thus far, when I followed the links all I could find were arcane puns that only a statistician might decipher. Very frustrating.

maccassar
May 10, 2014 1:02 pm

Mike Bryant-
I agree with you and your wife. My lovely wife has gone back through my family tree and found my ancestors lived in England in the 1500s. If I were to criticize the Brits it would be with a knowing look and a smile and a wink. But I guess they have their intra-mural disputes. While on a Scottish golf trip, I asked a young Scottish Lad how far we were from the English border. He quickly replied “Not far enough”

Another Geologist's Take
May 10, 2014 1:06 pm

I’ll go with milodon…as teens, we used tits up (as in belly up) in 1960s LA.

Spillinger
May 10, 2014 1:18 pm

Language, like the climate changes all the time. ‘It’s all gone Pete Tong’ amounts to the same as pear shaped, tits/belly up. (Cockney rhyming slang for wrong, using a radio 1 DJ’s name).

May 10, 2014 1:22 pm

Mosher
That is indeed the accepted standard in journals that publish survey-based research (although some argue that you should save key strokes as well).
Environmental Research Letters, however, does not usually publish this kind of work and the editor is bit out of his depth; as is the Institute of Physics, the publisher.

EternalOptimist
May 10, 2014 1:25 pm

talking of the Brits, I guess Monty Python would be making a sketch about Cookies stuff being ‘Pear reviewed’
I guess you have to say that in an english accent

May 10, 2014 1:26 pm

Bran Don Scholl N’ Berger
Brandyn Sheolanbergyr
Brandan Showlenbirgir
I done gradiated wif fonix! 🙂 You’re a trooper Brandywine Shelfandbooger. That you live rent free in John Cook’s head is just a bonus. (And GrumpieOldeManook always, always makes my day!!! 🙂

Matthew R Marler
May 10, 2014 1:42 pm

My last name was spelled three different ways in this post.
I added a “c”; sorry.

May 10, 2014 1:45 pm

Pear shaped predates the 1980s in this Limey’s recall.
Great work Brandon. I’ve given up rebutting the 97% consensus claim, as even when it is deconstructed for true believers, such is their belief, they still believe it. Whatever gets you through the night, I guess.

Michael Bentley
May 10, 2014 1:45 pm

With no intent to offend anyone —
This Thread is a Hoot! I’ve never laughed so hard in all my life. Knowing some here have more brains in their little finger than I have in my whole….well….head – the humor (humour) expressed is refreshing and wonderful. THANKS TO ALL (raised voice not shouting!)
Mike Bentley

Larry Ledwick
May 10, 2014 1:47 pm

I would not be surprised if the origin of pear shaped as a reference to things going bad, dates back much further then the references given above. When a machinist is boring a hole (as in boring out a cannon), when things go wrong the bore takes on an egg or oval shape, as the boring bar gets in resonance with the changing load.The boring bit gouges deeply into the metal then unloads and briefly makes a lighter cut. I have no reference to that in literature but when working as a machinist I many times heard frustrated comments about items that were squeezed too hard and distorted or the boring bar started vibrating and distorted the bore. Sometimes you did not realize the hole bored was really oval or pear shaped until you took it out of the 4 jaw chuck of the lathe and released all pressure on it. It was a round and true bore while clamped in the chuck, but unknown to you had been clamped with much more force across two opposing jaws than the other pair of jaws. As soon as you released it from the machine your nice round bore would go pear or oval shaped.
Likewise a carpenter boring a hole in wood will also end up creating a pear shaped or oval hole if the brace and bit is not held perpendicular to the work during each turn of the brace. I would look in old references from the mechanical trades for the earliest usage of the term if I was an etymologist.