This new paper may explain the widespread belief in the value of Michael Mann's methods and the 'bet' on the Hockey Stick

Over at Steve McIntyre’s, there’s a fascinating discussion going on about the relevance of the hockey stick in the context of the Myles Allen mis-identification of the temperature record in a 2011 conference on Climategate as being the hockey stick issue rather than the paleo-record, Yamal, and “hide the decline” tricks being the central issue.

Allen is in a furor defending himself and his misstep, even going so far as to suggesting Bishop Hill is picking “the least flattering” photos to put in the blog post when in fact it is nothing more than the default thumbnail from YouTube. Even the Communicate 2011 website featuring Allen’s presentation uses the same thumbnail (scroll down). The FAIL on display here is hilarious.

In the middle of all this there’s a new paper which may explain why so many scientists, the IPCC, NGO’s, and governments bet on the hockey stick as the “hot hand” in the climate science card game. The paper has a prescient title:

Why Do People Pay for Useless Advice? Implications of Gambler’s and Hot-Hand Fallacies in False-Expert Setting

by Nattavudh Powdthavee, Yohanes E. Riyanto (May 2012)

So why would I point out a paper on gambling as being relevant to the hockey stick? Because, the hockey stick was in fact a huge gamble on the part of “The Team”. They knew full well the science in it was shonky, but they hedged their bets with techniques (such as Mike’s Nature Trick) that gave a result that they felt sure would be “bought” by the scientific community at large. It was a good gamble at the time, but as Climategate has shown us, it may have been a winning hand with a one time jackpot, but they are losing the card game as the other players slowly realize they have a cheat in their midst.

At the blog “Stumbling and Mumbling” there’s a review of the paper with the headline:

============================================================

The strong demand for charlatans

In the improbable event of ever being invited to give a commencement address, my advice to graduates wanting a lucrative career would be: become a charlatan. There has always been a strong demand for witchdoctors, seers, quacks, pundits, mediums, tipsters and forecasters. A nice new paper by Nattavudh Powdthavee and Yohanes Riyanto shows how quickly such demand arises.

…

The predictions were organized in such a way that after the first toss half the subjects saw an incorrect prediction and half a correct one, after the second toss a quarter saw two correct predictions, and so on. The set-up is similar to Derren Brown’s The System, which gave people randomly-generated tips on horses, with a few people receiving a series of correct tips.

And here’s the thing. Subjects who saw just two correct predictions were 15 percentage points more likely to buy a prediction for the third toss than subjects who got a right and wrong prediction in the earlier rounds. Subjects who saw four successive correct tips were 28 percentage points more likely to buy the prediction for the fifth round.

This tells us that even intelligent and numerate people are quick to misperceive randomness and to pay for an expertise that doesn’t exist; the subjects included students of sciences, engineering and accounting. The authors say:

Observations of a short streak of successful predictions of a truly random event are sufficient to generate a significant belief in the hot hand.

(h/t to Marc Morano for the link)

==============================================================

To me, this sounds exactly like what happened with the Hockey Stick, as it was that “a-ha” moment for many people. IPCC had a “hot hand” and everybody started betting on it. Matt Ridley elucidates on that very issue in the CA comments

Matt Ridley Posted May 28, 2012 at 6:38 AM | Permalink

Far from being an irrelevancy, for me personally, the MBH hockey stick was absolutely vital in first extinguishing my scepticism then fiercely re-igniting it. When I first saw it, I was blown away by the clear evidence of unprecedented climate change, and I immediately told people I was no longer sceptical about climate change, a subject I had not been paying much attention to or writing about at that point, but had expressed some doubts about in print a few years before. That it had been published in Nature was good enough for me at the time. Aha, I thought, a smoking gun.

Then when I came across Steve’s work and realised how full of holes both the method and the data were, and that the IPCC was not interested in listening the criticisms, it made me doubly sceptical about not only paleo-climate data, but climate change theory generally, Nature magazine’s standards and — following the farcical enquiries — the British scientific establishment’s willingness to be bought. The hockey stick was by no means the only thing that caused me to change my mind twice, but it was the most salient.

=================================================================

This paper would seem to explain why so many bet on the shonky science of the Hockey Stick, and why they keep betting on it even though that “hot hand” has disappeared. I loved this part about ‘“the law of small numbers” – i.e. those who believe that a small sample of signals represents the parent population from which it is drawn‘ because it explains Yamal and the cherry picked ten sample set to a fault:

They write in the paper:

There is little economic theory in this area. Rabin (2002) and Rabin and Vayanos (2010) outline a model in which believers of “the law of small numbers” – i.e. those who believe that a small sample of signals represents the parent population from which it is drawn (Tversky & Kahneman, 1971) – will be willing to pay for services by financial analysts after observing randomly occurring streaks of profitable financial performances predicted by these professionals. This fallacious belief in the hot-hand of a financial expert arises as a consequence of the gambler’s fallacy, which is defined as an individual’s tendency to expect outcomes in random sequences to exhibit systematic reversals.

The authors suggest that an investor who believes that the performance of a mutual fund is a combination of the manager’s ability and luck will, at first, underestimate the likelihood that a manager of average ability will exhibit a streak of above- or below-average performance. Following good or bad streaks, however, the investor will revert to overestimate the likelihood that the manager is above or below average, and so in turn will over-infer that the streak of unusual performance will continue (see also Gilovich et al., 1985). The implication of this is that believers of the law of small number will be happy to pay for real-time price information provided by experts, such as stockbrokers or managers of actively-managed funds, even when it is well-documented that actively-managed funds do not outperform their market benchmark on average (see, e.g., Fama, 1991)

The parallels to the bets made on the Hockey Stick, and the continued faith by many that Mann came by his “hot hand” scientifically and the betting was sound are quite plain. It is another example of confirmation bias.

Here’s the paper and abstract:

Why Do People Pay for Useless Advice? Implications of Gambler’s and Hot-Hand Fallacies in False-Expert Setting

by Nattavudh Powdthavee, Yohanes E. Riyanto

(May 2012)

Abstract:

We investigated experimentally whether people can be induced to believe in a non-existent expert, and subsequently pay for what can only be described as transparently useless advice about future chance events. Consistent with the theoretical predictions made by Rabin (2002) and Rabin and Vayanos (2010), we show empirically that the answer is yes and that the size of the error made systematically by people is large.

Text: See Discussion Paper No. 6557