by John P. Costella | January 18, 2010
From SPPI
INTRODUCTORY ESSAY
Why Climategate is so distressing to scientists
by John P. Costella | December 10, 2009
The most difficult thing for a scientist in the era of Climategate is trying to explain to family and friends why it is so distressing to scientists. Most people don’t know how science really works: there are no popular television shows, movies, or books that really depict the everyday lives of real scientists; it just isn’t exciting enough. I’m not talking here about the major discoveries of science—which are well-described in documentaries, popular science series, and magazines—but rather how the process of science (often called the “scientific method”) actually works.
The best analogy that I have been able to come up with, in recent weeks, is the criminal justice system—which is (rightly or wrongly) abundantly depicted in the popular media. Everyone knows what happens if police obtain evidence by illegal means: the evidence is ruled inadmissible; and, if a case rests on that tainted evidence, it is thrown out of court. The justice system is not saying that the accused is necessarily innocent; rather, that determining the truth is impossible if evidence is not protected from tampering or fabrication.
The same is true in science: scientists assume that the rules of the scientific method have been followed, at least in any discipline that publishes its results for public consumption. It is that trust in the process that allows me, for example, to believe that the human genome has been mapped—despite my knowing nothing about that field of science at all. That same trust has allowed scientists at large to similarly believe in the results of climate science.
Until now.
So what are the “rules” of the scientific method? Actually, they are not all that different from those of the justice system. Just as it is a fundamental right of every affected party to be heard and fairly considered by the court, it is of crucial importance to science that all points of view be given a chance to be heard, and fairly debated. But, of course, it would be impossible to allow an “open slather” type of arrangement, like discussion forums on the Internet; so how do we admit all points of view, without descending into anarchy?
This question touches on something of a dark secret within science one which most scientists, through the need for self-preservation, are scared to admit: most disciplines of science are, to a greater or lesser extent, controlled by fashions, biases, and dogma. Why is this so? Because the mechanism by which scientific debate has been “regulated” to avoid anarchy—at least since the second half of the twentieth century—has been the “peer review” process. The career of any professional scientist lives or dies on their success in achieving publication of their papers in “peer-reviewed” journals. So what, exactly, does “peer-reviewed” mean? Simply that other professional scientists in that discipline must agree that the paper is worthy of publication. And what is the criterion that determines who these “professional scientists” should be? Their success in achieving publication of their papers in peer-reviewed journals! Catch-22.
It may seem, on the surface, that this circular process is fundamentally flawed; but, borrowing the words of Winston Churchill, it is the worst form of government except for all those others that have been tried. Science is not, of course, alone in this respect; for example, in the justice system, judges are generally selected from the ranks of lawyers. So what is it that allows this form of system work, despite its evident circularity?
The justice system again provides a clue: judges are not the ones who ultimately decide what occurs in a courtroom: they simply implement the laws passed or imposed by the government—and politicians are not, in general, selected solely from the ranks of the legal profession. This is the ultimate “reality check” that prevents the legal system from spiraling into navel-gazing irrelevance.
Equivalent “escape valves” for science are not as explicitly obvious, but they exist nonetheless.
Firstly, a scientific discipline can maintain a “closed shop” mentality for a while, but eventually the institutions and funding agencies that provide the lifeblood of their work— the money that pays their wages and funds their research—will begin to question the relevance and usefulness of the discipline, particularly in relation to other disciplines that are competing for the same funds. This will generally be seen by the affected scientists as “political interference”, but it is a reflection of their descent into arrogance and delusions of self-importance for them to believe that only they themselves are worthy of judging their own merits.
Secondly, scientists who are capable and worthy, but unfairly “locked out” of a given discipline, will generally migrate to other disciplines in which the scientific process is working as it should. Dysfunctional disciplines will, in time, atrophy, in favor of those that are healthy and dynamic.
The Climategate emails show that these self-regulating mechanisms simply failed to work in the case of climate science—perhaps because “climate science” is itself an aggregation of many different and disparate scientific disciplines. Those component disciplines are extremely challenging. For example, it would be wonderful if NASA were able to invent a time machine, and go back over the past hundred thousand years and set up temperature and carbon dioxide measurement probes across the breadth of the globe. Unfortunately, we don’t have this. Instead, we need to infer these measurements, by counting tree rings, or digging up tubes of ice. The science of each of these disciplines is well-defined and rigorous, and there are many good scientists working in these fields. But the real difficulty is the “stitching together” of all of these results, in a way that allows answers to the fundamental questions: How much effect has mankind had on the temperature of the planet? And how much difference would it make if we did things differently?
It is at this “stitching together” layer of science—one could call it a “meta-discipline”— that the principles of the scientific method have broken down. Reading through the Climate-gate emails, one can see members of that community usually those with slightly different experience and wisdom than the power-brokers questioning (as they should) this “stitching together” process, particularly with regard to the extremely subtle mathematical methods that need to be used to try to extract answers. Now, these mathematical and statistical methods are completely within my own domain of expertise; and I can testify that the criticisms are sensible, carefully thought-out, and completely valid; these are good scientists, asking the right questions.
So what reception do they get? Instead of embracing this diversity of knowledge— thanking them for their experience (no one knows everything about everything) and using that knowledge to improve their own calculations—these power-brokers of climate science instead ignore, fob off, ridicule, threaten, and ultimately black-ball those who dare to question the methods that they—the power-brokers, the leaders—have used. And do not be confused: I am here talking about those scientists within their own camps, not the “skeptics” which they dismiss out of hand.
This is not “climate science”, it is climate ideology; it is the Church of Climatology.
It is this betrayal of the principles of science—in what is arguably the most important public application of science in our lifetime—that most distresses scientists.
Re: John Hooper (Jan 21 16:09),
Would that be the corpse of AGW?
Just a great informative read by SPPI !
Thank you for posting John Costella here. I think he has done a superb analysis and a great service. It is a long read but worth it because it hangs together: a real Twilight of the Gods, seeing the scientists trap themselves, and as a result move slowly but inexorably towards Climategate. Together with Bishop Hill and Steve Mosher I think we now have a trilogy.
George DeBusk (15:48:01),
Bad explanation on my part. [See, I knew exactly what I meant.] These guys were gaming the system, and they cashed in on their reputations. By doing so they were playing with fire, and now they’re getting badly burned [I tend to disagree with the author’s comments that people will serve jail time; maybe as a prosecutor you could comment on that].
The whole world can now see how inept at science [and statistical analysis] they really were, and how really sloppy their methods and [non-existent] record keeping were, and how they made facts up as they went along, and how devious and self-serving they were. Now their reputations are in tatters. In retrospect, I think every last one of these guys would give back every penny they received, in return for having their pre-climategate halos restored.
John Hooper (16:09:33) :
When you can’t dispute the incriminating emails, make an ad-hom attack on the messenger? Is that it?
Credibility check: We already know that Greenpeace and the WWF take lots and lots of corporate dollars. So please cite in this article where the author is “making up BS to save corporate dollars,” and from which corporation[s]. How many corporate dollars he received would be interesting, too.
Take your time, it’s a long article.
Smokey:
Good point, that slipped past me.
For the record, my analysis was done in my spare time, completely unfunded, and I do not benefit financially from it one cent.
To be slightly critical of this great article, the judiciary analogy doesn’t fully apply. The core of science is that results can be replicated by others. With murder investigations for example, the event in the past being investigated cannot be replicated.
Determining the truth in science is much easier than with legal investigation of past, unrepeatable events
There is an existing, and widely known, example of science which has been contaminated by non-scientific issues: medical research in Nazi concentration camps.
Many medical researchers have had to decide whether to consider information which comes from Nazi experiments, whether the information is valid, and whether it is ethical to use the information. There are also difficulties in trying to duplicate some experiments, both due to missing data and being unable to use the same techniques.
–What I can not “get” is how blinkered, political, and hypocritical the email writers were. They just seem oblivious to their own actions and how political and unscientific they are.
Exactly what they accuse anyone with a different view of being.
It is so obvious, yet they (seemingly genuinely) can not see it, untill very close to the end.–
Yes, that’s exactly what struck me too.
The real gems are when discussing people who want the programs to replicate their work. Their response is “they’re just too lazy to do the work.” Or course, if these people DID generate their own programs based on the principles discussed in a paper and got different results, the answer would never be “Hmmm… we’ll need to double-check our results”, it would instead have been “You did it wrong, idiot. Leave the science to the pros!”
I have to say that George DeBusk’s experience is very far from mine. It may be significant that his field was paleoecology; not at the center but certainly in the suburbs of the AGW debate. It may be that the less physical the field, the more opinion can influence peer review.
That’s not to say that paleoclimatology has no mathematical content. Clearly, it does so, in spades. But there’s no good physical theory for recovering quantitative paleoecological or paleoclimatological information, such as temperature or precipitation or even plant dispersal. I know a paleontologist who just observed that palynology — the study of ancient pollen — is badly impacted by disturbed sedimentary deposits, for example.
Most of the paleo-relevant math is statistical, or numerical analysis, and opinion can creep in as to which methodology is more appropriate. Even more, there is no good measurement theory that allows one apply a particular well-established and standard method, in the manner that spectroscopy serves Chemistry, that, in turn allows disputes to be settled by appeal to very objective data and very well established theoretical meaning. Personality gets short-circuited. I’ve experienced that effect when my one or two controversial papers went through review.
So, one can see how the poisonous disputes that apparently accompanies peer review in anthropology or worse, literary theory, might also invade the climatological fields that are not stabilized by a good foundation in physical theory. We’ve certainly seen that in paleothermometry, which has approximately zero physics-theoretical content.
One has to observe, though, that opinion mongering always arises by choice of the mongers. It’s always possible to have disputes that do not descend into ad hominem disparaging. One might suppose that an unfailing tendency toward this sort of argumentative tactic reflects a kind of personality defect.
Onion (17:04:15) :
“To be slightly critical of this great article, the judiciary analogy doesn’t fully apply. The core of science is that results can be replicated by others. With murder investigations for example, the event in the past being investigated cannot be replicated.
Determining the truth in science is much easier than with legal investigation of past, unrepeatable events”
What you say is true of experimental science, but not so much of observational science. In Paleoecology we deal with “past, unrepeatable events.” Now the way you reconstruct and measure those events must be repeatable – as must crime scene investigation in a murder case. I have actually found the skills from paleoecology to be very useful in handling murder cases precisely because of the past, unrepeatable nature of the events.
The judiciary understands this as well. one of the key factors in the admissibility of scientific evidence is repeatability.
“This question touches on something of a dark secret within science one which most scientists, through the need for self-preservation, are scared to admit: most disciplines of science are, to a greater or lesser extent, controlled by fashions, biases, and dogma.”
As a biomedical scientist, I think you have correctly identified one of the persistent problems in science. In my field (immunology), T suppressor cells became suddenly unpopular in the 1980s (I don’t have space to go into why this occurred, but there was a reason for questioning the T suppressor dogma). There was good evidence from dozens of labs that functional suppressor cells existed, but virtually everyone followed the trend and abandoned this field until recently when the suppressor cell idea was revived (and modified) as T regulatory cells. During 30 years of research I have seen several examples like this. I am not sure if there is a solution, but I like the idea of an open peer review process so the reviewers and the editors as well as the person who submits the paper are accountable. The down side of this is that many scientists would be reluctant to give a negative review to a well known scientist, fearing repercussions. Perhaps open but anonymous reviews?
The climate gate emails indicate clearly that the IPCC reports do not reflect the proper scientific uncertainty recognized by many of the participants.
The response of the Team and its supporters seems to be that even if certainty of AGW has been overstated, we need to decrease use of fossil fuels anyway, so a crash program to do so quickly will not hurt. However, this is not the case. Governments and individuals only have a limited amount of money. The extra amount that would be spent on alternate energy sources would not be available for things that we know will save lives (like anti-smoking or anti-obesity or cancer screening campaigns or cleaning up sites with known toxic pollutants or charitable giving for food, clothing, housing, medical care, etc. for those in poverty). In a very real sense, spending billions on alternate energy in a crash program will cause or allow the avoidable death of many, many people. Thus, the standard of certainty to which the science should be held should be very high. If they reveal nothing else, the Climate gate documents reveal that such a standard has not been met. I agree that moving to alternate energy sources is needed and that it should happen faster than it is occurring presently, but a terribly expensive crash program, whether it involves cap and trade or emission limits, does not seem justified. The risk of acting massively and quickly is virtually certain (lack of money for other known life saving measures), whereas the benefit (stopping AGW) is seeming more and more uncertain.
Could an unknown Swiss patent officer have gotten his theories of special relativity and quantum physics published today? I doubt it.
It was not until 1936 when Eistein had his first run in with the peer review process, and was furious his paper was being challenged before publishing by some anonymous referee. He then published it elsewhere and never submitted another paper to. IIRC it was Physics or Physical Review.
Let all the peers review the papers after publishing. Those who review papers prior to publication are unlikely to approve papers which contradict their own work. It’s a bad system. Perhaps an alternative is to require a scientist to get 3 other scientists who work in the field to attach their name to the paper as reviewers. Einstein used to submit his papers to other scientists he respected for opinions before going to the publishers. Anonymous peer review is a BAD idea. Kind of like Anonymous Judges or Anonymous Presidents.
Also Leif said his publishers require him to pay a page fee, and he paid 11,000 dollars to publish his last paper. This on top of the high subscription fees is not conducive to good science.
Another thing, not covered in this article, is the role of governments in funding science. Eisenhower warned that this may be a problem, where science is controlled by a technocratic elite bent on becoming Philosopher (natural philosphers were the equivalent of scientista back in the day) Kings, and as Plato said, these rulers should be able to tell a Noble Lie for the good of the people. Thats what Climate science has become unfortunately, a Noble (or not so Noble) Lie, paid for by the dupes being lied to.
There are a lot of good scientists out there, all being strangled by the process and the need to publish or perish. Those who speak against the system will find funding hard to come by, so they stay silent.
It has been suggested here that the IPCC should not rely upon the peer review process alone to make their reports and assessments. So true. However, the main point has been missed. They don’t even rely on the so called peers in the peer review process itself. Do I need to remind everyone that in the last IPCC report most of the scientists disagreed with the conclusions, and that the remarks made by those against the AGW thesis were ignored? The IPCC does not follow a true peer review process at all.
pft: “Also Leif said his publishers require him to pay a page fee, and he paid 11,000 dollars to publish his last paper. This on top of the high subscription fees is not conducive to good science.”
Well then if they don’t have 11k lying around what do you do if the university decides not to fund the review, or more likely you’ll be told what research you can and will be funded for review.
He who pays the piper calls the tune. This is about as corrupt a system as you can ever conceive of.
pft: “Let all the peers review the papers after publishing…. Anonymous peer review is a BAD idea. Kind of like Anonymous Judges or Anonymous Presidents.” — Excellent points. And have the reviews printed also so everyone can see if the review has merit or not. You’ll quickly find proper review going on or reviewers will quickly be discredited.
John , so anything over a century mann came up with was pure speculation/manufactured? ( please forgive my ignorance) iam just facinated with this subject and iam just making sure i understand.
Charles. U. Farley (13:25:39) :
The identity of the “leaker” was hinted at at the end of the paper
John Costella (14:08:21) :
Hey John, how nice to “see” you here.
Thankyou very very much for your research into climategate. I read your paper (until 5am) and am going through it a second time.
Is this a one off or are you updating in the future?
Have you seen Mohib Mohammed and Jo Novas 30yr timeline piece?
George DeBusk (11:41:50) :
I guess if you find it “ordinary” that they hid from FOI requests using bogus excuses, and lied and cheated to get certain papers in and keep certain papers out of the IPCC reports, and refused to show their data and methods, and destroyed vital evidence, and “hid the decline”, then you could say there is “nothing out of the ordinary” in the CRU emails.
Me, I find that kind of behaviour extraordinary, but YMMV, I suppose …
Stephen Pruett,you’re presenting the classic Lomborgian false dichotomy: that money cannot be spent on such things as “alternative energy sources” and “things that we know will save lives” simultaneously. Prioritising and allocating spending is the work of every budget ever prepared. We are already spending money on a diverse spread of energy sources,and energy saving measures,and,anyway, whether AGW existed or not,we will be doing ever more of this as fossil fuel resources become scarcer. Your main idea is really that AGW is a crock: you won’t materially prove that by invoking illogical tangents.
Dr Costella,meanwhile, cannot seriously be offering a discussion of scientific ethics and a deciphering of details while identifying members of the group involved in the emails as ‘conspirators’ . This is simply prejudice.
He then raises a false equivalence ( tax minimisation = tax evasion),when he cites his first email.
He cannot be seriously suggesting that using knowledge of lump sum taxation rules on money transfers between sovereignties to minimise taxation implies something sinister. The researchers are simply trying to conserve as much grant money as possible to apply to the project. Every casual international traveller knows the implications of lump sum thresholds,and seeks to inform themselves on how to minimise loss during transfers.
In the second email he visits, he discusses an exchange about the annoyance value posed by Piers Corbyn,who was gaining prominence as an alternative weather forecaster and AGW doubter. From this one exchange, and only the second discussed so far,Dr Costella is confident enough to decide:
‘Already we can start to appreciate that the politics and “spin doctoring” in this field outweighs the scientific issues’
Is such a conclusion possible, from just one email?
Nick (14:46:33) :
It is patently obvious you haven’t read the whole paper. if you had, you wouldn’t have the gumption to comment like you did.
IMHO you are a lemming trying to discredit what is in reality a reprint of the very words out of the mouths of the AGW “team”.
It’s easy to attack the author with ad homs. How about you try dicrediting the content? YOU CAN”T BECAUSE THE CONTENT IS STRAIGHT FROM THE HORSES MOUTH.
I bet you were too gutless to read past the intro. for fear your “belief” will be shattered. Religion indeed, and you are living proof of it right here amongst us
Those who are criticising Johns introductory paragraphs are, with all due respect, missing the real point.
It doesn’t matter whether you agree with the legal system analogy, John even stated it was the best he could come up with for now.
The meat is in the emails themselves. Now, you may have a different interpretation of individual and or chains of emails, thats fine, if you do, maybe you can share your interpretation with us.
John Castello, I love your comments about jail time etc Is there something in the pipeline regards criminal charges that you can share with us?
Baa Humbug (18:59:44):
Thanks for the welcome.
I will certainly be amending the web page for any errors or omissions, but right now I see the analysis page itself as essentially completed.
Absolutely, I have seen the wonderful 30-year timeline — and all the other excellent work; indeed, I and am embarrassed by compliments when I’ve done very little compared to those who have toiled hard on this issue (and given where we are currently talking, I am even more humbled and embarrassed).
At this stage I have not decided what to do next, but I certainly would like to contribute to sorting out this mess in whatever small way I can.
Baa Humbug (19:23:20):
Yes, if you look at the original web page, what SPPI have included as the “introductory essay” is the op-ed piece I wrote on 10 December, with a regular newspaper audience (definitely non-scientists) in mind.
I don’t think there’s much more I can add publicly over what is already public knowledge.
Of course, all opinions are mine and mine alone.
Re: Stephen Pruett (17:45:26) :
“As a biomedical scientist, I think you have correctly identified one of the persistent problems in science.”
I saw the same phenomenon in paleoecology in the early 1990s. The Younger Dryas – a period of return to Glacial Period temperatures that happened at the beginning of the Holocene – was the hot thing at the time. It seemed one merely had to hint that they might find evidence that the YD was global in nature (as opposed to limited to the North Atlantic) to get funded or published. I joked that one French lab could find the YD in the sediment at the bottom of their tea cups. In discussions with other grad students, I found similar fashionable ideas existed in other areas of biology and geology. I called it paradigm chasing. To me it seemed to be a result of centralized government funding. It is very interesting to hear the same phenomenon occurs in biomedical science.
Bias is most problematic in research, where research findings about purported facts can be tweaked, fudged, re-done until right, put in a file drawer, etc. This is where drug companies’ “findings” can seriously mislead, or likewise with “advocacy research” (e.g., about the number of homeless, or the threat to polar bears).
But when only argumentation is involved, as with the material on the SPPI site, it doesn’t matter much about which “wing” of the political spectrum the site’s curators and backers are on. What matters are the arguments. Only a hyper-partisan of one wing would automatically and emphatically spurn anything said by a partisan of the other and/or refuse to hear it.
(But this is the knee-jerk rejectionist reaction that opinion-leading propagandists like Joe Romm routinely and successfully attempt to inculcate in their followers.)
Superficially unbiased organizations like the NSF, the IPCC, and the EPA aren’t ipso facto trustworthy. They may just be wolves in sheep’s clothing, infiltrated or manipulated by partisans.
The arguments have to be judged on their merit.