Quote of the week – solving the peer review integrity issue

qotw_cropped

A poll follows.

Over at Bishop Hill, he’s listed some quotes from Geoffry Boulton on scientific integrity that I found interesting. He writes (with apologies for posting in full, I couldn’t see any way to excerpt this short article):

==============================================================

Geoffrey Boulton is giving a speech to JISC, the goverment body which “inspires UK colleges and universities in the innovative use of digital technologies, helping to maintain the UK’s position as a global leader in education”. (Austerity, what austerity?). His comments are being widely tweeted under the hashtag #jiscmrd. Here are a few interesting ones: 

Bolton from Royal Society saying that its “malpractice” to not publish underlying data to research at same time as paper published

@ScienceBL: Boulton says publishing of data should be concurrent with the paper. #jiscmrd #datacite” <- very much agreed.

Boulton: cures for scientific fraud: open data for replication, transparent peer review, personal and system integrity #jiscmrd

#jiscmrd Geoffrey Boulton: open data is our responsibility to citizen science.

It’s funny to see Boulton calling for transparent peer review after failing to investigate allegations of journal nobbling – probably the single most important issue to have emerged from Climategate – during the Russell inquiry.

===============================================================

The idea of having all the data and methods up front ahead of time make a lot of sense. In my view, this is central to effective peer review. Without it, it boils down to a “trust us” situation with the authors of the paper. Given all of the mess surrounding Marcott et al and the failures of peer review to catch its problems, and the uptrend in paper retraction in science in general, I thought it might be a good time to ask this question.

UPDATE: Some people wondered about whether they could join such a professional organization or not if it existed, not being accredited in the field. It should be noted organizations like the AGU and the AMS accept “associate members” i.e. people that have an interest in the science but who may not be accredited in the field. There’s no reason to consider why that could not be the case for a new organization. – Anthony

The climate data they don't want you to find — free, to your inbox.
Join readers who get 5–8 new articles daily — no algorithms, no shadow bans.
0 0 votes
Article Rating
94 Comments
Inline Feedbacks
View all comments
JudyW
March 26, 2013 10:08 am

The UK is absolutely the best at self aggrandizing propaganda.

oldfossil
March 26, 2013 10:18 am

Perhaps Geoffrey Boulton has experienced a conversion on the road to Damascus.
As a parent of two and grandparent of five I can attest that the best way to reinforce positive behavior is to reward it.

Sundance
March 26, 2013 10:21 am

Anthony an equally important questions is “Would you see an organization or science journal providing replicable papers up front as better than those that don’t?”

March 26, 2013 10:39 am

Another “quote of the Climate Gate Cult’s elite:
http://www.wsj.com
March 25, 2013 ” Natural Gas Rocks The Energy World”
by:
David Crane , NDG (Natural Gas Producer)
Tim Rosenzweig (Goldwind Inc.) must be wind energy guy.
Michael Brune (Sierria Club)
Best quote ” by the Sierria Club guy,,
“why spend hundreds of billions of dollars in a “fossil-fuel” infrastructure when
WE KNOW THAT WE HAVE TO DECARBONIZE AS QUICKLY AS POSSBLE.”
He better check things out, he be % carbon too.
Carbon feeds corn , corn feeds him.

jorgekafkazar
March 26, 2013 10:47 am

Science is about replicability, not hubristic assertion. If it’s not replicable, it’s not science, and no amount of statistical mummery will make it so. It may be religion or politics or pandering, but not science of any sort.

March 26, 2013 11:02 am

To appreciate the importance of JISC to this story it important to remember that CAGW is relying on a modelling, constructivist, social inquiry approach to education replacing the historical transmission of a body of knowledge approach. Then you simply make catastrophe due to . . . as politically desired part of the software. Visually the student keeps hearing and seeing that there will be a catastrophe and enough repetitions make it a believed “fact.” remember false beliefs are still consequential in altering future behavior.
The Frankfurt School in their Radio Project research in the 30s established that useful fact. It’s in the Primer for all aspiring Statists I believe. Much of the push in digital learning is really to make education vocational. Use of a tool that does the mental work. So a computer as the center of education is a cognitive tool that is actually weakening most students who simply treat the interaction as visual and interactive, not mental as a book or a lecture.
That’s also in the Primer. Use education practices that take away the masses ability to think accurately and abstractly. Seriously. It comes from Soviet psychologists Vygotsky and Luria and especially Leontiev. I actually explained symbolic cultural tools here months ago http://www.invisibleserfscollar.com/creating-new-minds-different-values-equity-in-credentials-can-this-really-lead-us-to-prosperity/ .
Plus there is the fact that to a committed Marxist (and they do exist and they are getting bolder in saying so), ICT is the dream transformative technology that Marx and Engels always dreamed of in their vision of history. The idea that consciousness changes to reflect the dominant technology.
So if education is the vehicle for transformation and CAGW is the excuse as I have been arguing, the Digital Learning push is another vehicle for massive noetic change. At the individual and then society levels. This is not just another speaking engagement. Digital learning and cyberlearning are very connected to CAGW as a means of sociocultural change. CAGW is the reason for the accompanying economic change. Together you get the Total Transformation or an Administered State. Every bureaucrat’s and their cronies dream.

March 26, 2013 11:18 am

Ten Commandments of the Climate Cult.
1. Thou shall be cold if the cult says so.
2. Thou shall be hot if the cult says so.
3. Thou shall not eat any thing on our no eat list. (We make the list)
4. Thou shall walk or ride a donkey or horse to work and play.
5. Thous shall live in a small hut and be grateful for that to the Climate Gods.
6. Thou shall not take Michael Manns name in vain.
7. Thou shall not praise the use the Carbon Energy of Mother earth.
8. Thou shall not demand proven facts from the climate gods just obey.
9. Thou shall not do unto the climate gods as they do unto you.
10. THOU SHALL DECARBONIZE OR ELSE.

Theo Goodwin
March 26, 2013 11:25 am

Gary Pearse says:
March 26, 2013 at 8:40 am
“Engineers generally have high integrity because the proof of their good or bad work is in the product.”
Also, many engineers belong to professional organizations of engineers that teach ethics, have ethics boards, and have something to say about licensing. The standards for Civil Engineers, Materials Engineers, and quite a few others are very high. Most important of all, these engineers are quite comfortable talking to one another about ethics.

Wamron
March 26, 2013 11:29 am

Re Joseph Oulson: “Thinking humans make lousy slaves”
Not if the Ottoman empire was anything to go by. Their entire state beurocracy was administered by slaves, up to and including the status of regional governor. These were very educated , thinking slaves. Most were raised from childhood for their roles having been taken as human tax tribute from East European states defeated in slave yielding military campaigns. Those campaigns in turn waged by slave armies, most notably the elite Janisserie and the Horsemen of The Porch.
Chattel slavery continued in Turkey until the 20th Century. The last slave market in Istanbul was in 1904.
The vision of slavery that guilt-whipped Westerners are perpetually fed is very, very distorted.

March 26, 2013 11:33 am

The thought has just occurred to me that more people read this blog than anything Boulton writes.
Indeed, after thinking: “they will soon start coming to you begging to be allowed to speak”.
I began wondering just how much commercial value there is in WUWT.

March 26, 2013 11:58 am

The climate waterheads have turned “peer review” into “pal review”.
The pretense of impartiality still exists, for some unfathomable reason. It doesn’t make bloody sense.
It’s like catching your child with his hand stuck in the cookie jar, chocolate cookie well in hand, wrist stuck in the bottleneck; yet, the child insists that what you’re seeing ISN’T what you’re seeing.
Guys caught in the family bed banging away at a chick have more credibility than Albert “Global warming’s gonna make me rich, beyotch!” Gore.

Dave
March 26, 2013 12:13 pm

Boulton is to scientific integrity as Gliek is to ethics…

March 26, 2013 12:26 pm

Geoffrey Boulton, the poster-child for post-normal science and the Theory of Inverse Reality. Invert what he says and then discern the truth of whatever matter.

Stephen Richards
March 26, 2013 12:29 pm

I wouldn’t join but I would buy their journal. I buy none now. Stopped physics, SCIAM, Nature, New scientist. Had them all at one time.

john robertson
March 26, 2013 12:33 pm

Boulton speaking on ethics and science is like Al Gorical speaking about energy conservations and honesty.

John Whitman
March 26, 2013 12:39 pm

lsvalgaard on March 26, 2013 at 9:42 am
If the journal will not cooperate in this regard, the author can simply put the review on his/hers website. Examples here: http://www.leif.org/research/
– – – – – – –
Leif,
Thanks for your comment.
I noticed your “320 Dikpati Referee Report.pdf (My Referee’s Report on Dikpati et al., GRL 2006) “.
As a referee (reviewer) option, publishing their reviews on a website does look like it contributes toward opening up the journal (peer) review process.
John

Wamron
March 26, 2013 12:40 pm

Milwaukee Bob…all accepted and with no offence taken.
That thing about British education is one of a number of articles of faith that form unexamined wallpaper to life in Britain. Its an identity narrative.
What I really find questionable is the assumption that encouraging the use of digital technology in education is necessarily a good thing.
I very much disagree.

Zeke
March 26, 2013 12:54 pm

“data for replication”
Why couldn’t he say “Preservation of original data is supposed to be one of the highest priorities for scientists. Preserving data allows others to replicate results, and it also allows for the possibility that in the future scientists and interested people may make observations from that data that are not anticipated by today’s limited understanding. Preservation of unaltered data is a safeguard against many scientific mistakes and abuses, such as carefully selecting only certain data points and signals, while excluding others which would not confirm the the theory of the scientists.” Probably because it was just too long for a Tweet. (:

John Whitman
March 26, 2013 12:55 pm

Duster on March 26, 2013 at 9:47 am

John Whitman says:
March 26, 2013 at 9:25 am

I think that no review or evaluation should be anonymous. Not infrequently, applying for a grant, even a very small one, for instance to cover costs of radiocarbon dating, can meet with a completely opaque “not recommended” or similar response with no explanation why. Without an explanation, there can be no path forward if the researcher cannot fund the work. Not infrequently, back stories and grapevine information will often indicate that “x” review the application. He/she – hates the committee chair of that student/automatically – rejects research without an inherent touchy-feely/component – wants the project and has in fact already hijacked the idea. Requiring a clear justification for why a proposal is reject and that the individual objecting put their name on the line seems to be a clear method of forcing a more objective review process.

– – – – – – –
Duster,
Thanks for your comment.
I think you are suggesting that at a the request of the funding applicant any decision not to fund by the funding body should be explained in at an accessible public link.
It would tend to contribute toward integrity auditing of the funding body.
John

Roy
March 26, 2013 12:56 pm

Milwaukee Bob says:
March 26, 2013 at 10:04 am
And IF the UK IS a global leader in education, why are so many Brits coming over to the US to finish their education?
For much the same reason as American students go to universities in Britain and other countries.

Theo Goodwin
March 26, 2013 1:27 pm

Doug Huffman says:
March 26, 2013 at 8:52 am
Please do not panic. Lots of ideas are floating around here. The idea that data must be published along with the publication of a peer reviewed article does not threaten the liberties of anyone.
As regards professional associations, they are found at various places along a spectrum. An association that teaches ethics and encourages its members to become comfortable discussing ethical issues does not threaten the liberties of anyone.
Keep in mind the context. The so-called “investigations” that white washed the Climategaters could achieve their end only because there were no widely accepted standards that could be applied to their work. In effect, the investigators made up their standards or non-standards during their investigations. That cannot be allowed to stand. The behavior of the Climategaters and those who exonerated them raised the questions that we are discussing.

Puppet_Master_Blaster_Master
March 26, 2013 1:38 pm

The AGU and AMS “may” be ‘professional’ organizations but THEY ARE THE CENTAL PROBLEM with “peer-review”, including their pay-to-play systems and ‘buddy-check’ reviewers.

March 26, 2013 1:44 pm

Certainly the situation as it stands has to turn around or all of science loses out.
I find it personally interesting that while I know it is the scientists amongst us who are doing some heavy duty work to turn this scam around, and it is scientists who really will save this world from mad green policy – I nevertheless have a default reaction of mistrust whenever I hear “science says” now.
I hate that I feel that way. It surprised me that I should have that initial reaction to reject something outright simply because “science says”. It doesn’t matter what the field is. On some kind of subconscious level “Science = Dodgy” and that is just SO WRONG.
I’ve been coming in here daily for quite some time now – I’m addicted to this place. I love what I learn here, I love reading the comments and interacting with you all. I know that we will win out through using real science and the scientific method. You are all doing exactly what’s needed.
Yet there is a huge percentage of the population that does not come in here and does not know the good work being carried out to stop the madness that’s going on in the world. When this is all done and finished, it will be the “skeptical” aspect that will be remembered. “Science” will be remembered as being on the Bad Guy side.
So, if I’m feeling “science = dodgy” and “don’t trust ‘science says’,” and I KNOW that real science is winning – what the heck are THEY going to think? Especially when the only science that’s been rammed down their throats for 30 years has been the FALSE sort.
So, yes, this situation has to be turned around to keep ALL of science going down the toilet completely for the next hudred years.
After my initial mistrust when I hear “science says,” I have only one question and that is, “Where’s the data?”
Data and methods ALWAYS supplied up front is the way to go. It is the vital step. That, I would always trust.

Ray Donahue
March 26, 2013 2:33 pm

Hi Warmron, RE: Slavery
Absolutely correct. A very (!) complicated and embarassing subject (not just to the Western World
Ray

Vincent
March 26, 2013 2:33 pm

Pressure of work prevents me from reading previous posts, so I may be repeating points already made.
The questions being asked in this poll makes it irrelevant for me to respond, but it raises issues that I have a strong opinion on. The problem is the phrasing of the question:
1. I’m an Electronic Engineer, so I would never be likely to join an atmospheric sciences organization.
2. Even if it was an engineering organization, I would not be likely to join either way. After more than 30 years in practice, I’ve not joined one yet.
However I do read papers from these organizations and I need to trust what I read.
What is perhaps more relevant to this poll is if it had asked whether I would give credence to a paper published by an organization that only published papers that were replicable up front.
Then my answer would be a definite “Yes”! As it stands, I can’t even say “Maybe”.