A poll follows.
Over at Bishop Hill, he’s listed some quotes from Geoffry Boulton on scientific integrity that I found interesting. He writes (with apologies for posting in full, I couldn’t see any way to excerpt this short article):
==============================================================
Geoffrey Boulton is giving a speech to JISC, the goverment body which “inspires UK colleges and universities in the innovative use of digital technologies, helping to maintain the UK’s position as a global leader in education”. (Austerity, what austerity?). His comments are being widely tweeted under the hashtag #jiscmrd. Here are a few interesting ones:
Bolton from Royal Society saying that its “malpractice” to not publish underlying data to research at same time as paper published
“@ScienceBL: Boulton says publishing of data should be concurrent with the paper. #jiscmrd #datacite” <- very much agreed.
Boulton: cures for scientific fraud: open data for replication, transparent peer review, personal and system integrity #jiscmrd
#jiscmrd Geoffrey Boulton: open data is our responsibility to citizen science.
It’s funny to see Boulton calling for transparent peer review after failing to investigate allegations of journal nobbling – probably the single most important issue to have emerged from Climategate – during the Russell inquiry.
===============================================================
The idea of having all the data and methods up front ahead of time make a lot of sense. In my view, this is central to effective peer review. Without it, it boils down to a “trust us” situation with the authors of the paper. Given all of the mess surrounding Marcott et al and the failures of peer review to catch its problems, and the uptrend in paper retraction in science in general, I thought it might be a good time to ask this question.
UPDATE: Some people wondered about whether they could join such a professional organization or not if it existed, not being accredited in the field. It should be noted organizations like the AGU and the AMS accept “associate members” i.e. people that have an interest in the science but who may not be accredited in the field. There’s no reason to consider why that could not be the case for a new organization. – Anthony
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Boulton is to scientific integrity what Charles Manson is to happy families.
“Western populations would accept serfdom if it is packaged as saving the Earth”…Bertram Russell
Reject the hypothesis that humans CAN save the Earth and you are on your way to avoiding serfdom….meanwhile we CAN reduce our minor negative impacts. Thinking humans make lousy slaves.
Well, this is truly rich. These statements may set a new standard in “hubris”.
McIntyre, rightly admired for his considerable restraint in dealing with pompous buffoons, must be experiencing a rise in his annoy-o-meter.
This must be the face of post-normal science – where facts, truth, integrity, and logic “don’t really matter anymore” (TM – Climate Science)
My specialty isn’t atmospheric sciences, so I am not sure I could check “Yes” on the poll. Replace that with RF Propagation and I’d definitely join. The problem is that while I follow this issue and it impacts tangentially on my main research in RF propagation, it is the politics of it that inspired me to look into it farther. When someone claims that we all have to live as cavemen to save the planet, I want to check their data. When the data (when one can get it) and procedures (again if one can get it) appears to not support their conclusions I get curious and start digging. When the further I dig the less supported the position appears, I tend to get angry at those who are muddying up the reputations of all scientists.
Is the poll for people working in the sciences only? I’m not in that group, so rather than skew the results by voting “no,” I’ll refrain from voting. If I were a scientist, yes, I would join.
REPLY: actually, organizations like the AGU and the AMS accept “associate members” i.e. people that have an interest in the science but who may not be accredited in the field. There’s no reason to consider why that could not be the case for a new organization – Anthony
Engineers generally have high integrity because the proof of their good or bad work is in the product. You can’t get away with cooking a design for a bridge or airplane, nor would one want to. It also helps that they belong to associations created by statutes that require a license to practice which requires certification, passage of an ethics examination and which has the power to discipline up to and including loss of license, remedial education and supervised re-entry into practice. I think the time is ripe, in an era of moral laxity, to require similar statutes for scientists. In Canada, we have already included geologists and geophysicists in these associations. Re Boulton and posting data. Lets hope its ‘data, the whole data and nothing but the data’. We have seen posting of data in the Shakun paper that is tailor made for it and different from that of his just completed thesis on the subject with no additional research done. Boulton would (from his past partisan position with climate felons) have been a candidate for re-education if he were an engineer.
Come on, you yes-voters, what does the quibble “professional” mean but authoritarianism and credentialism? How many vote is not so significant as who counts the enfranchised-votes. Professional democracy is not different from professional mob-rule. The Ancient Greeks mooted elitism through Sortition.
If I voted Yes it would really indicate something between “Confirmation bias” and “Conventional Wisdom”. If I voted No it would not mean I didn’t want more transparency but simply that I didn’t have the interest or knowledge to contribute.
Then again, like one of the Marx Bros said, it might indicate that I wouldn’t want to join any club that would accept me.
JC
I would welcome any scientific journal that had that requirement – I would humbly suggest it would make a great standard for a scientific magazine, online or otherwise. In time I would think it would be far more popular due to a history of integrity that some would say is lacking in the current offerings.
I think his suggestion is a start. But it does not deal with one of the erroneous methods used by the warmist lobby which now seems to regard the number of “peer reviewed” papers as being critical to an author’s credentials and correctness. This has been taken a step further by suggesting that because a large number of such papers support the AGW view it naturally follows that there is a consensus that the AGW belief is right and that there are only a few cranks, who should be disregarded, who disagree.
A peer review by somebody who agrees with your point of view is unlikely to advance things very much. It is a bit like asking a group of church ministers to review papers on the existence of god and concluding, because they give favorable reviews, that god therefore exists. Then, because only true believers are likely to publish such papers, you say that the overwhelming number of papers support the existence of god, therefore there is a consensus that he exists.
Boulton is an international expert on ethics. Here, for example:
http://www.raeng.org.uk/news/publications/list/reports/Ethics_transcripts.pdf
…where Boulton discusses the importance of acting with “honesty and care, not committing plagiarism and declaring conflicts of interest” (much like when Boulton was presented as an IPCC author in his resume, as the UK Government’s Chief Advisor on Climate Change (http://youtu.be/2VFWYfBEtJ8) (which he was not), and when he neglected to mention his time at the UEA when serving as a lead in the Muir Russell ‘independent’ inquiry into the CRU. Who better to comment on matters relating to honest peer review? (ho ho)
If a professional atmospheric sciences organization existed that didn’t enforce it’s declared standards, would you resign?
I strongly recommend the process of open and transparent peer review as I describe in the following (in the case of publicaly funded research).
Peer review should be performed as it is currently done in a private environment but then the proceeding of the peer review should be made accessible to the public when the paper is published. The reviewers should also be identified as a mandatory part of the public access to the peer review proceedings.
I would not consider incremental cost to do that as a a negative attribute.
John
It’s been noted in previous posts, but I will repeat it. Geoffrey Boulton has absolutely no credibility with regards to ethics and integrity. The Muir Russell inquiry proves that. Requiring the publishing of data with the research paper to facilitate replication and open peer review is a great idea, but Boulton is absolutely the wrong messenger.
I don’t twit myself, but is anyone going to do the Phil Jones quote about ‘you will only try to find fault with my data so you aren’t having it..’
I would go so far as to say that “fake” should be the default assumption for any paper that does NOT publish its data when it publishes its conclusions.
A case of ” Why do you see the splinter in your brother’s eye but not notice the log in your own eye?”, even though the splinter is a very real problem here.
John Whitman says:
March 26, 2013 at 9:25 am
Peer review should be performed as it is currently done in a private environment but then the proceeding of the peer review should be made accessible to the public when the paper is published. The reviewers should also be identified as a mandatory part of the public access to the peer review proceedings.
If the journal will not cooperate in this regard, the author can simply put the review on his/hers website. Examples here: http://www.leif.org/research/
Reblogged this on Climate Ponderings and commented:
TAKE THE POLL
John Whitman says:
March 26, 2013 at 9:25 am
I think that no review or evaluation should be anonymous. Not infrequently, applying for a grant, even a very small one, for instance to cover costs of radiocarbon dating, can meet with a completely opaque “not recommended” or similar response with no explanation why. Without an explanation, there can be no path forward if the researcher cannot fund the work. Not infrequently, back stories and grapevine information will often indicate that “x” review the application. He/she – hates the committee chair of that student/automatically – rejects research without an inherent touchy-feely/component – wants the project and has in fact already hijacked the idea. Requiring a clear justification for why a proposal is reject and that the individual objecting put their name on the line seems to be a clear method of forcing a more objective review process.
This approach should be expanded to all peer reviewed science. Although readers here known & are focused on atmospheric science , similar issues exist in other sciences as well. Are more open process would be beneficial to all science.
Then the next big problem to tackle would be grant funding – needs to be double blind. We do these 2 things & we’ll be a long ways toward more credible science.
Given that I already belong to an organization (contract engineering for mining firms) that requires this by law, I have voted yes. I’ve said this before. If climate scientists had to publish under the same laws that geologists do (hmm I can think of one of those!) they would panic. Indeed, if Mann “hid the decline” in a NI-43-101 report he would be subject to prosecution. A geologist would be required to highlight the decline and explain why it wasn’t important rather than hiding it.
In some cases I run through hundreds of GB of raw data before I come to a simple final plot. I actually don’t store the raw data many times, since it would take too much space to store it in the first place. This would also be the case for DNA studies. While raw data publication should be encouraged, it should not be required.
Anthony, how do we know this is not another “Lewcook” production? 😉
… JISC, the goverment body which ”inspires UK colleges and universities in the innovative use of digital technologies, helping to maintain the UK’s position as a global leader in education”
With all due respect AND apologies to my English friends, especially those in Hartlepool, the UK “needs” a government body to inspire the use of digital technologies in higher education? And IF the UK IS a global leader in education, why are so many Brits coming over to the US to finish their education? Part of being a GREAT, LEADING education system is having the capacity, in space, cost AND imagination, to “educate” a vast number people to whatever level they so desire and can achieve. Secondly, education is NOT just about scoring high on paper tests, in school. THE REAL test is what you are capable of accomplishing (and do accomplish) in THE REAL world once you are out of school, that of course is IF you ever leave school.
And “JUST” publishing the (some) data with/when a paper is submitted is not going to solve the inherent problems with the cloistered per-review system.