John Cook's 97% consensus claim is about to go 'pear-shaped'

Analysis of raters in Cook’s 97% paper by Shollenberger

pear shaped (slang)

A British expression used to indicate that something has gone horribly wrong with a person’s plans, most commonly in the phrase “It’s all gone pear shaped.”The OED cites its origin as within the Royal Air Force; as of 2003 the earliest citation there is a quote in the 1983 book Air War South Atlantic. Others date it to the RAF in the 1940s, from pilots attempting to perform aerial manoeuvres such as loops. These are difficult to form perfectly, and are usually noticeably distorted—i.e., pear-shaped.

Dr. Richard Tol writes about a new revelation coming from an analysis of Cook’s climate publications volunteer raters, conducted by Brandon Shollenberger:

My comment on Cook’s consensus paper has at last been accepted. It was rejected by three journals — twice by Environmental Research Letters and once by two other journals for being out of scope. Fifth time lucky.

As these things go, my comment is out of date before it is published.

One of my main concerns was the partial release of data. The data that was available suggests that all sorts of weird things were going on, but without the full data it was hard to pinpoint what went on. Cook’s resistance to release the data, abetted by the editor, the publisher and the University of Queensland, suggested that he may have something to hide.

Brandon Shollenberger has now found part of the missing data.

Unfortunately, time stamps are still missing. These would allow us to check whether fatigue may have affected the raters, and whether all raters were indeed human.

Rater IDs are available now. I hope Shollenberger will release the data in good time. For now, we have to do with his tests and graphs.

His comment of May 10, 1:16 am shows that individual raters systematically differed in their assessment of the literature. This is illustrated by this figure; the circles are aligned if the raters are the same.

This undermines Cook’s paper. Theirs was not a survey of the literature. Rather, it was a survey of the raters.

Source: http://richardtol.blogspot.co.uk/2014/05/the-97-consensus.html

Of note is the comment “Brandon Shollenberger has now found part of the missing data.”. While I don’t know for sure, it seems that the SkS kidz have left another gaping security hole wide open which allowed Shollenberger (and likely anyone, as we’ve seen before with their forum fiascos) to have a look at that rater’s data. Cook has been resisting requests to provide it.

Shollenberger writes in comments at his blog:

I’ve sent John Cook an e-mail alerting him to what material I have, offering him an opportunity to give me reasons I should refrain from releasing it or particular parts of it. I figure a day or two to address any potential privacy concerns should be enough.

His response will determine how much information I provide. No obligations were placed upon me regarding any of the material I have, but I don’t see any compelling reason to provide information about how I got it either. I’d need a better reason than just satisfying people’s curiosity.

But we’ll see what (if anything) Cook says. I said I’d give him the weekend. If I don’t hear anything tonight, I’ll try contacting him via Twitter/Skeptical Science. I may try having someone else from SkS get his attention for me. I don’t want him to simply overlook the e-mail I sent.

By the way, there is some value in associating ids and names. We have comments from many of the people who participated in the study. It could be useful to try to match up biases in the ratings with people’s stated views.

Tick Tock.

 

 

0 0 votes
Article Rating

Discover more from Watts Up With That?

Subscribe to get the latest posts sent to your email.

181 Comments
Inline Feedbacks
View all comments
May 11, 2014 4:54 pm

The abstracts were rated by Skeptical Science zealots, who doesn’t accept that there was bias? Their methodology was so flawed that they rated skeptic papers as endorsing the consensus,
http://www.populartechnology.net/2013/05/97-study-falsely-classifies-scientists.html

Don B
May 11, 2014 5:38 pm

Since the authors of the belatedly rejected paper which names names of people with alleged psychiatric problems did not hesitate to name names, Schollunburger (sic) should not hesitate to name names, and spell them correctly. Those people who aim to justify “97%” are crazy.
(I am originally from Central Oregon, not Eastern Oregon, but I have relatives there.)

bushbunny
May 11, 2014 9:40 pm

That border map went along very quickly, but when the French ruled England it wasn’t noted.
That was the Plantagenets. Henry the II and Richard the lst were basically French. King Stephen and Matilda had a real civil war. She lost (being a woman in a warrior king being preferred, didn’t help) Patrick have you ever heard of ‘strine’ very funny. When I arrived in Sydney, 1965, some Aussies were a bit hard to understand, and I had to have an interpretater. Not just words, but expressions such as ‘Bit lacking in the top paddock’ (dumb) Adams ale (water) So low he could crawl under a rattlesnake. (awful person).etc., etc. Should I throw me hat in before I enter? (will I be welcomed). I love the Aussie venacular I put a glossary in the back of my book for ‘foreigners’ to understand.

Mike T
Reply to  bushbunny
May 11, 2014 10:35 pm

BB, the map was wrong in some respects, as it showed invasions and not changes in borders. So Germany in WWII borders shouldn’t have changed apart from the places they formally annexed (Sudatenland, Austria etc). They didn’t annex Alsace-Lorraine despite several wars having been fought over the region, and the local dialect was, and is, Germanic, not French. It was interesting to see all the little places that made up the Holy Roman Empire before German unification in 19th century although the details were very hard to see.

May 11, 2014 9:50 pm

Don, I posted a list of the top raters above. That is not the final numbers so the order may have changed but it is close enough. The only one I am not that confident on is Bernard Walsh since his name was not mentioned in the acknowledgements but that could have been at his request or it was a pseudonym.

bushbunny
May 11, 2014 9:50 pm

Talking about language, if anything the English language has also adopted a lot of American
One of the things that I still get confused by is the use of initials. Even on this blog. In essays at our University, if we were to refer to the full name or title, such as Bureau of Meteorology (BOM)
We had to print the full name and put in brackets afterwards (later referred to as BOM) I know some journalist noted this and wrote a piece using all initials in the text, it was hilarious. Anyway, it’s fun eh?

Mike T
Reply to  bushbunny
May 11, 2014 10:30 pm

BB, that’s normal for university essays even today, refer to the org. in full with acronym in parentheses after it then any further reference to the org. can be as the acronym.

bushbunny
May 11, 2014 10:35 pm

Yes, but if you don’t know what the organisation is then we do have problems. Oh, I only got my BA and GCA in 2005 and 2013. LOL. Just in case you thought it was in the last century, LOL

Pethefin
May 11, 2014 11:29 pm

Ari Jokimäki has his own blog::
http://agwobserver.wordpress.com/

Skiphil
May 11, 2014 11:41 pm

I haven’t followed the details on Cook’s paper. I strongly applaud those who work on this, it’s just that I cannot devote the time currently and have to restrict what I look at in blogworld.
That said, the following comment strikes me as pointing to an important issue, don’t know if it has been followed up much or not. Brandon linked above to this comment of his — if one of the leading SkS types (Tom Curtis) dropped out of the rating process because he did not like having to rate so many abstracts “neutral” then what might that suggest about the attitudes and biases of the SkS crowd doing these ratings?? How many did not have Tom Curtis scruples about needing to be objective in rating papers “neutral” etc.?? (apologies if this has already been thrashed out somewhere):
[Brandon]:

“Heck, one participant (Tom Curtis) quit because he couldn’t stomach how many papers he had to rate neutrally, and yet, he must be one of the 24 “independent” raters.”

http://rankexploits.com/musings/2013/u-of-queensland-application-for-ethical-clearance/#comment-112937

Skiphil
May 11, 2014 11:54 pm

Sad to see Robert Way’s name on the list of co-authors.
I’d hoped on the basis of some of his candid, critical comments in the secret SkS treehut files that he was the only one there with a brain and some hints of intellectual integrity.
It seems not….

Pethefin
May 12, 2014 1:50 am

Anthony Watts says:
May 11, 2014 at 11:37 pm
The fact that Jokimäki uses the childish SkS-inspired “debunking”-meme is all we need to know about his understanding of science and his motives.

E.M.Smith
Editor
May 12, 2014 7:27 am

Bryant
Then there’s folks like me. Mum was from England so I drink tea in the morning and sometimes type colour and behaviour and… as she was my first teacher. Also raised me with a few book she brought from England. Dad was an American Mix, but 1/2 of first generation Irish American, so some of that slang was around the house. As he had spend some key years of his youth in England (WWII army) there was ample opportunity for language mixing both ways.
At about 8 years old I met my first Texan (he came into the family restaurant in California). Took me nearly a day and a half to ‘latch on’ to what he was saying. British English was nearly natural for me, but Texan! Like Australian, full of “colourful sayings”…
@all:
The present day folks in Scotland are a mix of a couple of origins. One large group came from Ireland (who in turn came from the Celtic part of Spain, so all you Irish are really Hispanic 😉 while there was an indigenous group already there. There’s reasonable evidence for a Scotts dialect of an English like language pre-dating the arrival of either Gaelic or actual English. So southern Scotts may be closer to the really old forms of English than even some English… That, as pointed out, has been through the ringers of French, Roman, etc. etc.
Oh, and on the genetic thread:
There are lots of decent maps of haplogroup and Y or X chromosome migrations. Anyone who thinks that there is a “pure” type anywhere in Europe, the Americas, Asia, etc. is sorely mistaken. It’s more like stew with some bits you can point at and claim a bit of what they might be… Yet you can still see the pattern of migration in the genetic smears. Celts, for example, were in places like Turkey and even down into Egypt (as a mercenary army to the Pharaoh) way back. North Italy was Celt, as was Austria and parts of Bavaria, through Gaul and on. Generally moving west all the time. Eventually crossing into Ireland (and from there to N. America / Australia ….)
Folks forget that the Germans wandered down south. Spain even. Oh, and that France is named for the Germanic tribe that had the throwing ax used to drive out invaders – the Franks.
So everywhere is a stew. Yet with traces of the past. (And don’t even get me started on the Swedes and Goths… starting down near Bulgaria / Thrace, going up north, then down through Spain to North Africa… and with some part out toward the steppes of Asia… though it’s a bit messy sorting Swedes from the various Goths from some other groups that went other places…)
Oh, and honorable mention to the Phoenicians / Carthaginians who went throughout the Mediterranean and up the Atlantic coast to the edge of Germany (and who seem to have left traces of their language in the Germanic languages including English). So even some Semitic component blended in. Yes, even in Germans. (BTW, the lore of Irish origin traces back to a Scythian prince, so perhaps north of Turkey back when it was Celt).
Eventually it looks like most Europeans originated from The Levant or just north of it, as the ice melted. About 12,000 years ago, they spread out. (There were some folks already in Southern Europe then. They have a Basque like genetic type and survive in pockets, mostly in Basque lands – though some of them are in North America now…)
Tying it to Climate: The Migration Era Pessimum was a very cold period about the same as The Dark Ages that had a LOT of folks from more north and east running south and west to escape the bitter cold. There is a periodic warm and good time about every 1500 years (though with smaller nodes at 1/2 and 1/4 that) followed by a terrible cold – and migrations. What is not clear is: Was The Little Ice Age the very cold 1500 year event, or the 1/2 or 1/4 cycle event? If it was the full cycle, then we are good for ‘a while’, but if it was a partial, and the full is 1500 years after 535 AD onset of The Dark Ages…. 2035 might not be a very good year… (Also, it is sometimes called a 1470 year cycle, that would put the start of the downturn at 2005 … or about when the warming went away…)
The more things change, the more they stay the same…
As per “pear shaped” and “Tummy Up”… I heard variations on the latter in the ’50s, and that was from old folks so likey much older. Then there is “4 paws to the moon”… Likey all sorts of references to dead things on their back and bloating from thousands of years back, if you look hard enough. Just “in print” more recently…

May 12, 2014 10:40 am

poptech says:
May 11, 2014 at 4:50 pm
– – – – – – – – –
poptech,
Thanks for your reply. Appreciate the background on where raters names where previously published and relative approximate number of papers they rated.
Are you aware of a list of the specific papers each rater rated?
John

Reply to  John Whitman
May 12, 2014 10:52 am

no

3x2
May 12, 2014 11:58 am

Those of you pontificating upon the sources of various terms should accept that most of them originated in England, France or Germany. Those of you of Spanish/Portuguese/African/Chinese/Japanese … origin will have your own.
The (for example) ‘two fingered salute’ (not used so much in The US but probably familiar) came from the French army order (c.1400) that any captured English longbow man should have his two ‘pulling fingers’ cut off. Hence the English gesture utilising two fingers. We still use it today (in England) as a gesture of defiance. Back in 1400 it was a sign to the opposing (French) army that they still hadn’t got your (English) fingers.

milodonharlani
May 12, 2014 12:05 pm

Pamela Gray says:
May 10, 2014 at 3:04 pm
Don B says:
May 11, 2014 at 5:38 pm
My family is (or are, in British English) from all over Oregon, but I was born & raised in Eastern Oregon. Brother lives in Central Oregon. Descended from Oregon Trail immigrants of 1847 & 1852. The latter took up Donation Land Claim in Yamhill County (site of Linfield College) but moved to Eastern Oregon c. 1860, then SE Washington Territory to avoid hay fever. Grandad’s company built the Turn-Around at Seaside, Wolf Creek Highway (26 to the Coast), Santiam Pass Highway (22 to Sisters, et seq), various bridges & Crown Point Observatory in the Gorge.
Great-grandfathers let sheep eat down the native bunchgrass of the Columbia Plateau so that they could plant wheat. Pendleton was once the world’s leading wool railhead, hence the name of the clothing company now HQed in Portland, which in turn helped give rise to Columbia Sportswear, Nike, et al. One GGF became a Progressive Republican senator advocating initiative, referendum & recall (the Oregon System), votes for women & good roads. Also direct election of US senators, which IMO was not such a good idea, with benefit of hindsight.
Typical American mongrel: Amerindian, 17th century English, 18th century German, Scots-Irish & West African, 19th century Scottish & Swiss.
Oregon climatic data, unadjusted, show that it was warmer here in the 1890s & 1930s than during 1977-2006 or now.
As for American & British English, IMO there is now borrowing both ways. When I said “guy” at Oxford in the early ’70s, it was considered outlandish, but now I hear “chap” & “bloke” in Merrie Olde less than “guy”. Seems that “wimpy” has also been taken up by some Britons. OTOH, Americans are using “bits” more & more in lieu of “pieces” & appear to have adopted “cheeky”. Still not so much “maths”, however.
A junior common room guy in my college had never heard the word “innovative”, however. Nor did a middle common room buddy know that the British had burned down Washington, DC in 1814, shortly before their attack on Baltimore, which inspired the Star Spangled Banner, set to the tune of an English drinking song.
Mike T says:
May 10, 2014 at 5:50 pm
IMO the latest thinking is that the indigenous population of Sub-Roman Britain still mainly spoke Brythonic languages akin to Welsh, Cornish & Breton (brought to Brittany by British refugees). Latin of course continued in clerical use. Anglo-Saxon invaders (originally invited in to help ward off Pictish & Irish raiders, if Bede be believed) brought their Germanic language with them, although there probably were some Saxons & Jutes resident before the main Volk wandering & settling. A distantly related Celtic language (Goidelic, not Brythonic) was spoken in Ireland.
The Picts of Caledonia (Highland Scotland) apparently also spoke a Brythonic language, but Irish invaders (Scots) brought Gaelic with them to the west & Islands, as the Northumbrian Angles carried their brand of Old English to the east Lowlands. Later, the Norse colonized the northern Highlands & Islands, colliding with both the Picts & spreading Gaels. Meanwhile, the Strathclyde Britons held out for a while, speaking a British dialect related to that of Cumberland & Wales, hence Clan Wallace.
The west of England also long remained British-speaking, from Cumberland, through Wales to Cornwall. This was more than a Celtic fringe, but fairly continuous. Then Norwegians invaded Cumbria from their base in Dublin, as did of course the Danes had done the east of England, settling in the Danelaw. If not for Alfred & Athelstan’s victories over the Norse (with Welsh support), English might resemble Icelandic.
DNA can’t distinguish between Anglo-Saxon & Norse pedigrees, so the Great Heathen Army & other Viking invasions aren’t detectable genetically. But Old Norse (both Norwegian & Danish dialects) did leave a strong linguistic imprint on English, especially in Scotland & northern & eastern England.
Some scholars also find traces of British in both the vocabulary & grammar of English.

Mike T
Reply to  milodonharlani
May 12, 2014 4:25 pm

Milodonharlani, yes, I was aware of the false origin of the V-salute, and the “blood-groove” (it’s actually to break suction, I believe, for removal). You sound like a “senior language historian”… I’m merely a “junior historian” studying for fun in my dotage. I suppose the question may never be answered fully, but I do find it odd that a relative handful of “Saxons” could totally supplant Celtic. There should be more than mere “traces” of British language in Anglo-Saxon is this small number of invaders/ settlers came amongst much larger numbers of people speaking a quite different language. I recall reading recently of a Roman general, at the time of the invasion of Britain, saying that the Belgae of Gaul spoke a Germanic tongue, and that the Britons spoke a similar language. Having said that, the quite small number of Vikings did have a huge effect on Anglo-Saxon language (the more so perhaps, because their languages were so similar). Apparently, Mandarin came about because the Mongols couldn’t cope with the 8 or 9 tones in say, Cantonese, and it ended being simplified to four tones.
As for Icelandic, it’s a great pity that English lost some very useful letters (like thorn and eth) thanks to the Normans, and the printing press. The “Ye” in “Ye Olde Inne” is not, of course, a “Y” and shouldn’t be pronounced as one.
You’re quite right about Viking and Saxon DNA, I believe Viking ancestry is deduced from parish records and names.

milodonharlani
May 12, 2014 12:08 pm

3×2 says:
May 12, 2014 at 11:58 am
Fun but not factual:
http://bshistorian.wordpress.com/2007/07/02/two-fingers-up-to-english-history/

jorgekafkazar
May 12, 2014 5:57 pm

Wayne Delbeke says: “Tits up” is something I remember hearing from the time I started to understand language 66 years ago – and on a ranch, it referred to dead animal because when they are totally bloated when you find them in the hinterland, cows tend to be “tits up” ….
It’s my understanding that the expression is of agricultural origins, just as you state.

May 12, 2014 10:15 pm

I think the English pointing the finger at the American penchant for claiming all things British as their own is a case of the pot calling the kettle black. The English penchant for claiming anything they like out of Ireland as British never ceases to entertain us. The amusing story often told by the IRISH (no question about it!) actor Richard Harris is a good case in point: having won an award at Cannes and travelling back through London he spotted an Evening Standard billboard which read ‘British Actor wins award’. Travelling onwards to Dublin to continue the celebrations, drink intervened and he ended up in a brawl. An Evening Standard headline greeted him on his return to London: ‘Irish actor gets arrested in drunken brawl’!
This penchant is not a thing of the past. Only a few weeks ago the Daily Telegraph newspaper ran a story on the ’20 best British novels of all time’ including in the long list such well-known ‘British’ writers such as James Joyce, John Banville and Flann O’Brien. Even this last bastion of British imperialism seems to have accepted, finally (sigh-groan), that Ireland is a separate country (nearly 100 years after the fact!) when their readership pointed this out to them they changed the title to read the ’20 best British & Irish novels of all time’.
To the English contributors here giving out about the American unilateral adoption tendency, please, please, people in glasshouses shouldn’t be throwing stones………..

bushbunny
May 12, 2014 10:52 pm

LOL well Irish were termed British like Aussies and as South African non Afrakaans were once. But I agree I am not complaining about Americanese dialects or countries venacular, just initialising certain organisations or the tendency to do this. But I see your point. Well poor man Richard Harris had to give up the grog in his later years. Look Peter O’Toole was Irish born too, and they all loved their Scotch or Irish Whisky. But their colorful personalities like Oliver Reed were really a part of them, and most of us enjoyed their high jinks, so long as it didn’t harm anyone. At least they didn’t mind being labelled piss pots. . I remember one film that I thought was hilarious, was ‘The Best Year of my life’. Peter O’Toole played a drunken star of old Holywood fame, who had never been recorded live on TV. And it was recorded that he was drunk during filming.

milodonharlani
May 13, 2014 4:14 pm

Mike T says:
May 12, 2014 at 4:25 pm
I’m not senior to you in linguistic history, but have little Old English & Greek, less Latin, & almost no Welsh. However, IIRC from Caesar, the Belgae were said to have originated from across the Rhein, so may well have originally spoken a Germanic language. The best evidence though is that by Caesar’s time at least those in Gaul spoke a Celtic language.
Some Belgae had crossed the Channel & settled in what is now southern England, to include the well-known Iceni of East Anglia. But again, by Caesar’s time, they no longer spoke a Goidelic language, but a form of Brythonic, ie related to Welsh rather than Gaulish, whatever their ethnic origin. Their famous 1st Century AD queen Boudica is known in Welsh as Buddug.
The exact number of Frisians, Angles, Saxons & Jutes immigrating to Britain is unknown, but IMO enough to have caused an exodus of Britons to Brittany. It’s common in history for the language of successful invaders to displace that of the conquered or shoved aside. It seems in the case of eastern & southern Britain to have occurred both by displacing the indigenous population & via assimilation.
The Wiki entry on the subject seems to cite valid sources, including on estimating number of immigrants:
http://en.wikipedia.org/wiki/Anglo-Saxon_settlement_of_Britain#Estimating_continental_migrants.27_numbers
Don’t know how historical is the claim that all the Angles upped & moved en masse from the Baltic & North Sea shores of Schleswig-Holstein to eastern Britain, but seems plausible. Climate change in the 5th century might well have played a role.

bushbunny
May 13, 2014 8:06 pm

I can only contest that Roman occupation of England, Wales, and not Scotland, a lot of the soldiers (ranks) were from many parts of the Roman Empire. There was an agreement that occupied territories and farms were allowed to go on unheeded, provided they gave one son to the Roman military. Just a thought, if you have some hairy, armed Roman soldier in you province, one would be learning Latin quickly? The Romans didn’t occupy parts of Europe and Asia minor from being nice to people. “You be nice to us, and we will be nice to you or you die!” When these volunteers (?) served their 15 years conscription they were returned. That’s if you lived that long.

bushbunny
May 13, 2014 8:24 pm

In the 5th Century AD, the Romans had already departed Britain, well most of the legions. And the land bridge between continental Europe and Great Britain, was no longer there! Constantine 1 had settled in Constantinople, and the Roman Empire under his rule allowed Christianity to become legal. (More taxes) But the legend of King Arthur of course lives on, as well as Hereward the Wake, and there were invasions of Saxons, and Danes who settled not without some fights, until the Normans of course in 1066. History is my passion, but sometimes we forget that 100 years spells four generations, and people didn’t live very long then.

1 5 6 7