The following is from the blog Master Resource. Run by Robert Bradley. There are many interesting and well written essays there, and WUWT readers might benefit from a visit. Be sure to support them by checking out their books and leaving comments. – Anthony
Climategate: Is Peer-Review in Need of Change?
by Chip Knappenberger
December 1, 2009
In science, as in most disciplines, the process is as important as the product. The recent email/data release (aka Climategate) has exposed the process of scientific peer-review as failing. If the process is failing, it is reasonable to wonder what this implies about the product.
Several scientists have come forward to express their view on what light Climategate has shed on these issues. Judith Curry has some insightful views here and here, along with associated comments and replies. Roger Pielke Jr. has an opinion, as no doubt do many others.
Certainly a perfect process does not guarantee perfect results, and a flawed process does not guarantee flawed results, but the chances of a good result are much greater with the former than the latter. That’s why the process was developed in the first place.
Briefly, the peer-review process is this; before results are published in the scientific literature and documented for posterity, they are reviewed by one or more scientists who have some working knowledge of the topic but who are not directly associated with the work under consideration. The reviewers are typically anonymous and basically read the paper to determine if it generally seems like a reasonable addition to the scientific knowledge base, and that the results seem reproducible given the described data and methodology.
Generally, reviewers do not “audit” the results—that is, spend a lot of effort untangling the details of the data and or methodologies to see if they are appropriate, or to try to reproduce the results for themselves. How much time and effort is put into a peer review varies greatly from case to case and reviewer to reviewer. On most occasions, the reviewers try to include constructive criticism that will help the authors improve their work—that is, the reviewers serve as another set of eyes and minds to look over and consider the research, eyes that are more removed from the research than the co-authors and can perhaps offer different insights and suggestions.
Science most often moves forwards in small increments (with a few notable exceptions) and the peer-review process is designed to keep it moving efficiently, with as little back-sliding or veering off course as possible.
It is not a perfect system, nor, do I think, was it ever intended to be.
The guys over at RealClimate like to call peer-review a “necessary but not sufficient condition.”
Certainly is it not sufficient. But increasingly, there are indications that its necessity is slipping—and the contents of the released Climategate emails are hastening that slide.
Personally, I am not applauding this decline. I think that the scientific literature (as populated through peer-review) provides an unparalleled documentation of the advance of science and that it should not be abandoned lightly. Thus, I am distressed by the general picture of a broken system that is portrayed in the Climategate emails.
Certainly there are improvements that could make the current peer-review system better, but many of these would be difficult to impose on a purely voluntary system.
Full audits of the research would make for better published results, but such a requirement is too burdensome on the reviewers, who generally are involved in their own research (among other activities) and would frown upon having to spend a lot of time to delve too deeply into the nitty-gritty details of someone else’s research topic.
An easier improvement to implement would be a double-blind review process in which both the reviewers and the authors were unknown to each other. A few journals incorporate this double-blind review process, but the large majority does not. I am not sure why not. Such a process would go at least part of the way to avoiding pre-existing biases against some authors by some reviewers.
Another way around this would be to have a fully open review process, in which the reviewers and author responses were freely available and open for all to see, and perhaps contribute. A few journals in fact have instituted this type of system, but not the majority.
Nature magazine a few years ago hosted a web debate on the state of scientific peer-review and possible ways of improving it. It is worth looking at to see the wide range of views and reviews assembled there.
As it now stands, a bias can exist in the current system. That it does exist is evident in the Climategate emails. By all appearances, it seems that some scientists are interested in keeping certain research (and particular researchers) out of the peer-review literature (and national and international assessments derived there from). While undoubtedly these scientists feel that they are acting in the best interest of science by trying to prevent too much backsliding and thereby keeping things moving forward efficiently, the way that they are apparently going about it is far from acceptable.
Instead of improving the process, it has nearly destroyed it.
If the practitioners of peer-review begin to act like members of an exclusive club controlling who and what gets published, the risk is run that the true course of science gets sidetracked. Even folks with the best intentions can be wrong. Having the process too tightly controlled can end up setting things back much further than a more loosely controlled process which is better at being self-correcting.
Certainly as a scientist, you want to see your particular branch of science move forward as quickly as possible, but pushing it forward, rather than letting it move on its own accord, can oftentimes prove embarrassing.
As it was meant to be, peer-review is a necessary, but not sufficient condition. As it has become, however, the necessity has been eroded. And blogs have arisen to fill this need.
In my opinion, blogs should serve as discussion places where ideas get worked out. The final results of which, should then be submitted to the peer-reviewed literature. To me, blogs are a 21st-century post-seminar beer outing, lunch discussion, or maybe even scientific conference. But they should not be an alternative to the scientific literature—a permanent documentation of the development of scientific ideas.
But, the rise of blogs as repositories of scientific knowledge will continue if the scientific literature becomes guarded and exclusive. I can only anticipate this as throwing the state of science and the quest for scientific understanding into disarray as we struggle to figure out how to incorporate blog content into the tested scientific knowledgebase. This seems a messy endeavor.
Instead, I think that the current peer-review system either needs to be re-established or redefined.
The single-blind review system seems to be an outdated one. With today’s technology, a totally open process seems preferable and superior—as long as it can be constrained within reason. At the very least, double-blind reviews should be the default. Maybe even some type of an audit system could be considered by some journals or some organizations.
Perhaps some good will yet come out of this whole Climategate mess—a fairer system for the consideration of scientific contribution, one that could less easily be manipulated by a small group of influential, but perhaps misguided, individuals.
We can only hope.
===
Please check out Master Resource
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Dave Wendt (11:32:46)
Very well put.
No perfect check may be found for accenting good science. Somewhat similar to regulating the Market – all the regulations will not eliminate all the abuse or excessive exploitation.
Re science perhaps limiting Gov. size would reduce many of the review problems.
Increasing amounts of science/research/tech being done make for ongoing and evolving challenges to quality research. Good luck.
The Fraud Is Everywhere: SUNY Albany and Queens University Belfast Join Climategate (PJM Exclusive)
http://pajamasmedia.com/blog/the-fraud-is-everywhere-suny-albany-and-queens-university-belfast-join-climategate-pjm-exclusive/
Posted By Douglas J. Keenan On December 1, 2009
Some of the emails leaked in Climategate discuss my work. Following is a comment on that, and on something more important.
In 2007, I published a peer-reviewed paper [1] alleging that some important research relied upon by the IPCC (for the treatment of urbanization effects) was fraudulent. The emails show that Tom Wigley — one of the most oft-cited climatologists and an extreme warming advocate — thought my paper was valid [2]. They also show that Phil Jones, the head of the Climatic Research Unit, tried to convince the journal editor not to publish my paper.
After my paper was published, the State University of New York — where the research discussed in my paper was conducted — carried out an investigation. During the investigation, I was not interviewed — contrary to the university’s policies, federal regulations, and natural justice. I was allowed to comment on the report of the investigation, before the report’s release.
But I was not allowed to see the report. Truly Kafkaesque.
The report apparently concluded that there was no fraud. The leaked files contain the defense used against my allegation, a defense obviously and strongly contradicted by the documentary record. It is no surprise then that the university still refuses to release the report. (More details on all of this — including source documents — are on my site [3].)
My paper demonstrates that by 2001, Jones knew there were severe problems with the urbanization research. Yet Jones continued to rely on that research in his work, including in his work for the latest report of the IPCC.
Misconduct at Queens University of Belfast
Arguably, the biggest concern with global warming is that warming itself will cause further warming. For example, the polar ice caps reflect sunlight back into space, thereby cooling Earth. A global warming theory suggests that if the caps shrink due to warming, then they will reflect less sunlight and so Earth will warm even further. It is possible that Earth warms so much that it reaches what is called a “tipping point,” where the global climate system is seriously and permanently disrupted — like when a glass of water has been tipped over and the water cannot realistically be put back into the glass.
No one knows for sure how much Earth would have to warm before it reaches the tipping point — though about a thousand years ago, there was a time known as the Medieval Warm Period when much of Earth appears to have been unusually warm. It is not currently known just how warm the Medieval Warm Period was, but clearly the warmth then was below the tipping point because Earth’s climate continued without problem.
Suppose that during the Medieval Warm Period, Earth was 1°C warmer than today. That would imply that the tipping point is more than 1°C higher than today’s temperature. For Earth’s temperature to increase 1°C might take roughly a century (at the rate of increase believed to be currently underway). So we would not have to be concerned about an imminent disruption of the climate system.
Finding out how warm the Medieval Warm Period is thus of enormous importance for the study of global warming.
It turns out that global (or at least hemispheric) temperatures are reflected by the climate in western Ireland (for a short explanation of that, see my site [4]). Trees grow in western Ireland, of course, and each year those trees grow a ring. Thick rings indicate climate conditions that were good for the trees; thin rings indicate the opposite. If many trees in western Ireland had thick rings in some particular years, then climatic conditions in those years were presumably good. Tree rings have been used in this way to learn about the climate centuries ago.
Queen’s University Belfast has data on tree rings that goes back millennia — and in particular, to the Medieval Warm Period. QUB researchers have not analyzed the data, because they lack the expertise to do so.
They also refuse to release the data. The story is scandalous.
I have been trying to obtain the data [5] via the UK Freedom of Information Act since April 2007.
—————————————-
(Nature.com had a brief piece about my FOIA request [6]. The piece has statements from QUB that are dishonest: see my comment posted there. Statements from QUB therefore should be checked. My site has source documents for its claims.)
For a guide on how to proceed when the peer review process becomes unreliable, look at the testing of medical therapeutics in the US. We have developed a set of rules and regulation (Good Laboratory Practices or GLP) to ensure a high standard of quality in studies involving actions that could endanger the lives of people. Peer reviewing is not a requirement. Rather, what is required is a quality assurance unit that is properly staffed, empowered, and trained. Universities need quality assurance units to protect faculty from themselves. Science will recover when institutions ensure quality.
Yeah, I think this was raised before, but Climategate raises the question “Is peer-review necessary?” The climategate cabal’s insistence of publication in prestigious peer-reviewed journals begs the question since its necessity is assumed in their statements.
It is a bit of a diversion to argue about how to improve peer review.
As people have said, the problem was the taking of “peer review” as some kind of guarantee of correctness—and this article seems to be saying: well if only we could improve it when it fails, then we could take it as even more correct.
The simple truth is that boffins can make blunders.
It should always be acceptable for the people and leaders to say, “our strategy is to do nothing and hope that the scientists got it wrong.”
Whomever was driving this AGW agenda, seems to have known that this is how people ordinarily see things, and decided to counter it by making scientists overstate their claims and their levels of confidence.
“All the world’s top scientists are unanimous”…. yeah well so what? But then they accuse you of being “anti-science” and a “flat earther”.
It is not anti-science to admit that there are limits to what can be known with high confidence. To respect those limits is to respect science, just as a film-maker respects what his lens can shoot, and an athlete respects the limits of their physical endurance.
It is about respecting the craft, not about turning scientists into oracles for our favorite agenda.
At the end of the day, you can listen to science, but that still leaves you with the problem of making a decision about risk. Peer review doesn’t magically disappear the risk. There is always the chance that the scientists are flat wrong, and that whatever you try to do under their advisement turns out to have unintended consequences. It is almost an existential problem—people don’t like feeling that they are flying blind through an uncertain universe. But just wait a bit, wait a bit… look here comes another surprise…
This is a sober and realistic appraisal of the review system as it currently works. But I think double-blind reviewing would be an illusion. It’s usually easy to work out from the text of a paper who wrote it. Many papers emphasise that the work is a continuation of other published work (by the same authors) that is necessarily referred to in the paper.
But I’m surprised this audience doesn’t have more respect for market solutions. There’s no prescribed peer review system at present. There are conventions, but journals operate their own processes, and are free to vary. They thrive or fail on the success of their policies in producing good papers.
You don’t need to have a paper peer reviewed to get it out there and read, as we well know. It’s just that people do trust more papers that appear in reviewed journals. That’s their choice.
Using medical terminology, when a patient presents in the ER with infection, the first thing to do is “debread” the wound (remove all the dead and infected tissue). Unfortunately, AGW is not a patient, it is a rotting corpse. There is nothing to treat.
Alternatively, we could look at the problem as a cancer, which has metastasized. The pathology is everywhere in the body: in addition to the body scientific, it’s all throughout the body politic and the body economic. This is a very sick world, and it is their fault. The epithet “CLIMATE CRIMINAL,” which clowns like Monbiot throw at us, really aptly applies to them.
Climategate begs the question: “is peer review in need of change”?
Short answer – it depends on who you ask.
Here’s my list:
Patrick J Michaels, attempts made to get rid of his PhD thesis
Stephen J Crothers, PhD denied
http://www.worldsci.org/php/index.php?tab0=Scientists&tab1=Display&id=1119
Vincent Courtillot, a French geo-magneticist, director of the Institut de Physique du Globe in Paris studying temp and solar activity (posted several times here) – information denied
http://network.nationalpost.com/np/blogs/fullcomment/archive/2009/11/26/skewed-science.aspx
Dr. Keenan, information denied
posted above by Ed Scott
Great posts here. I like Ike!
Don’t know if anyone reads down this far, but the institutions of science that we’re all familiar with, such as peer review and the less-than-complete descriptions of data and methods, all evolved in an environment where they maximized the progress of science.
In a pure-science world populated with independently funded curiosity- and reputation-motivated researchers and journal editors, each scientist has pretty much the correct incentives under the traditional system:
Your goal in reading the literature is to figure out which results to build on in your own quest for discoveries and recognition.
Publishing scientists want to be just as careful as necessary to get people to cite them and build on their work, but not so careful that they slow themselves down and get scooped.
Journal editors use peer review so that scientists will want to read their publications in their search for useful clues for their own research.
As long as everyone is primarily motivated to make discoveries and get cited, the incentives line up very well. Read Michael Polyani’s “Republic of Science” essay for a good description of this world and all the optimal sloppy shortcuts it entails.
The problem is when the incentives and consequences change, so that scientists want to influence policy and attract funding by taking certain positions. Then 1) biases are likely to creep in and 2) the traditional level of reproducibility of results will be below that necessary for policy makers to make costly decisions. What’s good enough for curiosity-driven debates is nowhere near good enough for redirecting industrial civilization.
I recently produced an article on my views of the peer review process on my own irregularly updated blog site.
Some of you may be interested in it:
http://climatebalance.wordpress.com/2009/02/04/scientific-peer-review-is-it-a-corrupted-process/
Follow the money!
Look, peer-review in reality is a control gate to publishing, and some people must do so or perish, meaning not gain tenure and lose funding. The people who control the gate ultimately control the money flow.
It’s all about the funding. If you’re in with the CAGW cabal, you get funded and published. If you’re out, you’re out in every way imaginable, including losing your job.
That’s what’s wrong with the system. That’s how it is politicized. Follow the money.
[snip] The BBC now steps into the fold big time after keeping mum for so long, and what ist their message? They advocate more of this “post-normal science” snake-oil, so that we won’t have all this trouble about data and complicated old-fashioned science in the future but can instead rely on consensus of “social actors”!!! And they make it sound all roses and kittens, to boot.
http://news.bbc.co.uk/2/hi/science/nature/8388485.stm
Oh dear, oh dear. This CRU data leak may badly, badly backfire on us!
For more, and mighty revealing, qoutes of the BBC authors’ (Hulme and Ravetz) , pls see here:
http://buythetruth.wordpress.com/2009/10/31/climate-change-and-the-death-of-science/
Welcome to the Sausage Factory…
What should you do when you’ve lost your notes and cant reproduce the data?
HINT
“is peer review in need of change”?
I laughed when I read this.
Changing the system won’t eliminate corruption. Corrupt players will angle any system!
The most that can be achieved: A change in whose turn it is to be dominant, perhaps…
the ebb & the flow – another phase …
My preference: Liberalize the d*mn system! so that some fresh ideas can stir the stagnation. Tight-*sses can have their turn running the system again some years out after creative forces have a turn.
Peer review has never been pristine. It has been used by one cabal to limit the publication of other cabals since the beginning of time (i.e. 1600). Sir Isaac Newton, one of the greatest scientists Mankind has ever produced, was not above using position to stifle a competitor. I have been subject to this type of attack in my field of engineering and I must admit that I took glee in returning the favor when the oppotunity arose. How many of us in many fields of study know of researchers, scientists, or engineers who will never sit at the same table with each other, ever! Despite the conflict, it is part of how science moves forward because the arguments get aired. BUT, it is effective only as long as the journal editors stay above the fray so everyone gets heard eventually. Have the editors of the climate journals maintained an honest debate? It does not appear so.
Whether we want a mess around us is one thing. Needlessly killing animals until they become extinct is another of those one things. “Preserving as existing” is quite another. Causing the water in rivers and lakes to be so pure that it provides little or no nourishment to various “critters” is stupid. Whenever humans have attempted to do what is obviously stupid, it never worked.
The planet turned quite nicely all by itself until humans came along, and will continue to turn quite nicely if humans get themselves gone through human stupidity.
Having seen both sides of the review process, I offer the following thoughts:
1. For the most part, serving as a reviewer is a labor of love – the principal returns being the satisfaction of doing one’s civic duty and getting an advanced peek at something new and interesting.
2. Sometimes, however, it is also a means of promoting self-interest. This certainly seems to have been the case with Jones, Mann, etc., whose behavior is really a stain on the profession.
3. Double blinding the process, as suggested by other respondents, sounds fine in principal, but in practice probably won’t prevent the kind of abuses revealed by the CRU emails. After all, if you’re object is to suppress results inimical to your own ideas, the author’s identity is immaterial.
4. Archiving raw data, computer code, etc. – enough to allow the interested reader to replicate the results – is a reasonable requirement. But, under the present system, I doubt that most reviewers, having neither the time nor the incentive, would avail themselves of the opportunities access to such information would provide.
5. Publishing reviewers’ names along with the articles might provide some such incentive; likewise, publishing the reviews themselves. In effect, one’s reviews would become a form of publication to be added to one’s CV, clucked over in the course of annual fitness evaluations, etc.
6. The fundamental point about CRUgate, or whatever you call it, is that it’s an exceptional case. Both paleoclimate reconstructions and climate modeling are in the hands of small, tightly knit communities. Moreover, the essence of what they do cannot be replicated by individuals. To check the validity of CRUs gridding procedures, you need an outfit of roughly equivalent size and funding; ditto, for the large scale models. This allows for what amounts to monopolistic practices, the pursuit of which is favored by the enormous sums of money involved, both in the form of direct support to the investigators and overhead returns to their respective universities.
7. Regarding the latter (overhead), I have seen little, if any discussion. Yet it is really central. If, as a Professor at the University of the Antipodes, I receive $100K to study the turtle that holds up the earth , my university tacks on an additional $50K as overhead. Overhead is the crack cocaine of the academy. The original rationale was to enable universities to provide laboratory facilities, etc., to support the research being funded. The present-day reality is that these moneys effectively go into the general fund. Many (most?) research universities are utterly dependent on them. If meaningful investigations of CRUgate go forward, one of the things that will emerge is the fact that the universities involved turned a blind eye to abuses that were apparent to anyone willing to keep his eyes open.
8. Equally corrupting is the ideological mindset that makes AGW, and more generally, sustainability, so attractive to both faculty and administrators. Many (most?) universities are up to their eyeballs in this sort of thing; likewise, the funding institutions. So you get positive feedback: money and ideology foster the promulgation of “correct” results, which, in turn, generate more money and reinforce the ideology. The parallel with Lysenkoism is instructive.
9. There are already rules in place that forbid practices of the sort that seem to have been revealed by the CRUgate emails. The single most important thing that can be done to restore the integrity of the science is to penalize those who have broken the rules: ineligibility for future funding, say for the next 10 years, would be a good first step, by providing an example both to individual investigators and to their institutions of what can happen when scientists and their universities become just another interest group. Michael Ruse, the philosopher (and no Creationist), has discussed this sort of thing in the context of “evolutionism,” which he characterizes as a secular religion. Essentially, Ruse argues that the people doing the science are often them same individuals promulgating this, that or the other social policy and that, in the classroom, the two often get conflated. Ruse’s point, I believe, applies with equal, if not greater, force to environmental studies generally.
10. The other reform I would recommend is prohibiting participation of working scientists in groups such as the IPCC that, claims to the contrary notwithstanding, are effectively in the business of proposing policy. Here again, the current state of affairs, whereby individuals wear multiple hats, generates positive feedback that encourages self-serving and unethical behavior.
With apologies for having gone on at such length.
IMHO, the end state will be “Public Review”.
We have a competition now between “blind review” and “blogs” and the blind peer review is losing. Given the petty political infighting that people seem unable to avoid, any “blind” process will end up in that state. Even double blind is no cure. Folks have a “voice” in their writings and in small fields, the positions folks have taken is pretty clear.
So the web based review will get ever more of the “suppressed” or cutting edge raptid breaking stuff. The “end around” the political road blocks will mean that more of the interesting stuff will have bypassed the slow and politicized track.
In the end, someone will make a “peer reviewed web publishing service” where the “reviewing” will be done real time and in the open on discussion pages. Once thrashed out (when a tread has gone stale) it will be ranked and added to the archive. Some will get highest honors, some honorable mention, and some listed under “nevermind…”. WIth all the comments, commenters, and their biases and affiliations, visible to all.
And in that context science will advance faster and further than ever before.
And nothing can stop it for happening. The only question is “when”.
Frankly, the whole “secret handshake blind reviewers on paper” thing is just so archaic. It smells of old boy networks and folks wanting a political angle to work…
Thanks to those who linked to UNEP.
I was reading some of UNEP’s online material and there is mention of a strategy of uniting the world by reaching down past the government level and into the institutions in society, like academia and businesses and interest groups. In other words, I can see how this is an effort to water down the importance of nation states and their boundaries.
There was a time when the city state was the largest social power structure, and nowadays, whilst I still pay taxes to the city or county, I also pay takes to the nation’s government. For a global system, you can add a new level of taxes on top of that, and for different things.
We will all continue to be members of a nation, but it won’t matter much, just like I am a member of a city or county but that doesn’t matter much.
The new global system will reach down into academia (and hence, “the science”) and business (and hence, “big energy”), and draw power from allegiances built there. In turn the global government will use controls to alter the flow of resources globally, and “manage” the world “ecosystem”.
If we need more “rainforest” over here, we’ll pay the locals to plant trees rather than build factories—whole areas will have their development directed and coerced into whatever direction is deemed appropriate from above.
I can appreciate the vision of watering down the nation state. After all, why should I get to have a better life just by the accident of where I happened to be born? I could easily have been born a starving kid in Africa. So I don’t want to be too pessimistic about this idea of watering down the nation state in favour of a global system; I’m sure it will work pretty well after a thousand years of trial and error.
Science used to be ratified by a system of “peer usage” – that is to say other scientists actually had to find a real use of the original science in a different application – or ideally in a number of different applications. Hence Newton’s theories on gravity became accepted science because the theories could be used to make calculations in a variety of other areas where the calculations worked and worked well.
“peer review” is a relatively new process that has only gained traction in the last 40 years or so. It seems to me to be driven by economics rather than good science because obviously funding research on the basis that the science can be proven to be worthwhile (via peer usuage) is so much more difficult than doing science and have a number of your mates give it the thumbs up and then all get funding. Peer review lends itself to exactly the kind of financiallly motivated corruption we have seen in the AGW camp. It also leads itself to what psychologists and socialogists call “an escalating commitment to a particular course of action”, since it tends to result in informal teams of pals getting together with a particular belief system all inadvertently encouraging each other to validate each other’s erroneous conclusions in order to remain part of the in-crowd. They actually don’t realise that they are lying to each other and they have built a whole set of conclusions based on lying to each other and now they can escape from their own tower of lies.
A software audit could be done by “non-experts” quite easily. The authors publish what the code is supposed to do and the software auditor can review and test the software to ensure it does exactly that. The auditor has no need to know why it is being done or if it is even the right way to do it, they would simply verify that the code does what the author claims. That way the peer reviewer who can read the authors explanation of what the software is supposed to do is freed from checking if it is in fact doing it. The qualifed peer reveiwer can judge if the why is reasonable and justified.
This is simple software quality assurance that is done everyday at thousands of software development shops, firms and independent developers.
A software audit would include actually running the program against the raw input data the author provided and comparing that to the published data points and a review of the code itself to ensure that “special case” subroutines do not exist (i.e. ignore the post 1960 tree data) …
Call it the pre-peer review audit, any work submitted must pass at 100% before being actually submitted for peer review.
There are no doubt some improvements that can be made in the peer review process but the true nature of the beast that is besetting us is all too human. Professional societies, organizations (to include those of the hard sciences) and their “journals” can rarely take political positions or make endorsements and retain their credibility with the public (not to mention their own members). When the leadership –or outspoken members– of these organizations have obviously begun to take political stances they have currupted themselves and their organization. Consider this: the military in most advanced nations are allowed to vote freely for whomever they choose, but they are forbidden to wear their uniform and endorse –or actively canvass/campaign for– any political party or candidate over another (while in uniform). Professionals –especially hardcore scientists– and their professional organization(s) actually face similar restrictions when dealing with the public and political issues whether they want them or not. As soon as a scientist (or a professional organizations “leadership”) takes a political stand (or begins to exclude their fellows for their diverging views) the entire membership is painted with the same brush in the eyes of the public. There are “laws” in science, and there are “laws” of human nature. Unfortunately for all of us, some scientists and the leadership/editors of some professional organizations believe that they are special, that the laws don’t apply to them, that they are better and smarter than everyone else is. Life isn’t that simple. No one gets to have it both ways. No doubt, for some, the damage done will necessitate renaming their journel and/or deactivating the organization itself and beginning afresh. The two orgainzations I love to hate the most serve as perfect examples: The American Medical Association and The American Bar Association (I didn’t want to hit too close to home but I think the point is clear to all. There are others.)
A small team (<12) of software developers/business analysts could be easily assembled and paid for by the vartious publications and submitting institutions to be the software audit team for all publications. Would cost about 2-3 million per year and would have caught this nonsense of Climategate years ago.
Oh and all sumbissions and their test results would be made public. That way the world could see if a particular scientist had a track record of submitting sloppy software (i.e. code not match the explanation or hidden special case subroutines) … I'll call it sloppy software vs fraud, but in the world of software they amount to the same thing. Clean in = Garbage out.
sorry for the poor spell check in my original comment 🙁
As an aside, this software audit team could be hired directly by the various submitting scientists to do a private review of the software as well. They would be charged a fixed fee and the results would remain private. Many academic institutions would not have the QA staff available to do a good audit of their own and would benefit from this sort of division of labor. As an incentive to maintain their adversarial relationship with the scientists any submitted software that passes the data tests would be given a refund of a portion of their fee. That way the audit team would have an incentive to find errors as opposed to having an incentive to ignore them.