Quote of the week – solving the peer review integrity issue

“Western populations would accept serfdom if it is packaged as saving the Earth”…Bertram Russell
Reject the hypothesis that humans CAN save the Earth and you are on your way to avoiding serfdom….meanwhile we CAN reduce our minor negative impacts. Thinking humans make lousy slaves.

If I voted Yes it would really indicate something between “Confirmation bias” and “Conventional Wisdom”. If I voted No it would not mean I didn’t want more transparency but simply that I didn’t have the interest or knowledge to contribute.
Then again, like one of the Marx Bros said, it might indicate that I wouldn’t want to join any club that would accept me.
JC

I think his suggestion is a start. But it does not deal with one of the erroneous methods used by the warmist lobby which now seems to regard the number of “peer reviewed” papers as being critical to an author’s credentials and correctness. This has been taken a step further by suggesting that because a large number of such papers support the AGW view it naturally follows that there is a consensus that the AGW belief is right and that there are only a few cranks, who should be disregarded, who disagree.
A peer review by somebody who agrees with your point of view is unlikely to advance things very much. It is a bit like asking a group of church ministers to review papers on the existence of god and concluding, because they give favorable reviews, that god therefore exists. Then, because only true believers are likely to publish such papers, you say that the overwhelming number of papers support the existence of god, therefore there is a consensus that he exists.

It’s been noted in previous posts, but I will repeat it. Geoffrey Boulton has absolutely no credibility with regards to ethics and integrity. The Muir Russell inquiry proves that. Requiring the publishing of data with the research paper to facilitate replication and open peer review is a great idea, but Boulton is absolutely the wrong messenger.

I would go so far as to say that “fake” should be the default assumption for any paper that does NOT publish its data when it publishes its conclusions.

John Whitman says:
March 26, 2013 at 9:25 am
I think that no review or evaluation should be anonymous. Not infrequently, applying for a grant, even a very small one, for instance to cover costs of radiocarbon dating, can meet with a completely opaque “not recommended” or similar response with no explanation why. Without an explanation, there can be no path forward if the researcher cannot fund the work. Not infrequently, back stories and grapevine information will often indicate that “x” review the application. He/she – hates the committee chair of that student/automatically – rejects research without an inherent touchy-feely/component – wants the project and has in fact already hijacked the idea. Requiring a clear justification for why a proposal is reject and that the individual objecting put their name on the line seems to be a clear method of forcing a more objective review process.

Given that I already belong to an organization (contract engineering for mining firms) that requires this by law, I have voted yes. I’ve said this before. If climate scientists had to publish under the same laws that geologists do (hmm I can think of one of those!) they would panic. Indeed, if Mann “hid the decline” in a NI-43-101 report he would be subject to prosecution. A geologist would be required to highlight the decline and explain why it wasn’t important rather than hiding it.

The UK is absolutely the best at self aggrandizing propaganda.

Anthony an equally important questions is “Would you see an organization or science journal providing replicable papers up front as better than those that don’t?”

Science is about replicability, not hubristic assertion. If it’s not replicable, it’s not science, and no amount of statistical mummery will make it so. It may be religion or politics or pandering, but not science of any sort.

Ten Commandments of the Climate Cult.
1. Thou shall be cold if the cult says so.
2. Thou shall be hot if the cult says so.
3. Thou shall not eat any thing on our no eat list. (We make the list)
4. Thou shall walk or ride a donkey or horse to work and play.
5. Thous shall live in a small hut and be grateful for that to the Climate Gods.
6. Thou shall not take Michael Manns name in vain.
7. Thou shall not praise the use the Carbon Energy of Mother earth.
8. Thou shall not demand proven facts from the climate gods just obey.
9. Thou shall not do unto the climate gods as they do unto you.
10. THOU SHALL DECARBONIZE OR ELSE.

Re Joseph Oulson: “Thinking humans make lousy slaves”
Not if the Ottoman empire was anything to go by. Their entire state beurocracy was administered by slaves, up to and including the status of regional governor. These were very educated , thinking slaves. Most were raised from childhood for their roles having been taken as human tax tribute from East European states defeated in slave yielding military campaigns. Those campaigns in turn waged by slave armies, most notably the elite Janisserie and the Horsemen of The Porch.
Chattel slavery continued in Turkey until the 20th Century. The last slave market in Istanbul was in 1904.
The vision of slavery that guilt-whipped Westerners are perpetually fed is very, very distorted.

The climate waterheads have turned “peer review” into “pal review”.
The pretense of impartiality still exists, for some unfathomable reason. It doesn’t make bloody sense.
It’s like catching your child with his hand stuck in the cookie jar, chocolate cookie well in hand, wrist stuck in the bottleneck; yet, the child insists that what you’re seeing ISN’T what you’re seeing.
Guys caught in the family bed banging away at a chick have more credibility than Albert “Global warming’s gonna make me rich, beyotch!” Gore.

Boulton speaking on ethics and science is like Al Gorical speaking about energy conservations and honesty.

Milwaukee Bob…all accepted and with no offence taken.
That thing about British education is one of a number of articles of faith that form unexamined wallpaper to life in Britain. Its an identity narrative.
What I really find questionable is the assumption that encouraging the use of digital technology in education is necessarily a good thing.
I very much disagree.

Duster on March 26, 2013 at 9:47 am

John Whitman says:
March 26, 2013 at 9:25 am

I think that no review or evaluation should be anonymous. Not infrequently, applying for a grant, even a very small one, for instance to cover costs of radiocarbon dating, can meet with a completely opaque “not recommended” or similar response with no explanation why. Without an explanation, there can be no path forward if the researcher cannot fund the work. Not infrequently, back stories and grapevine information will often indicate that “x” review the application. He/she – hates the committee chair of that student/automatically – rejects research without an inherent touchy-feely/component – wants the project and has in fact already hijacked the idea. Requiring a clear justification for why a proposal is reject and that the individual objecting put their name on the line seems to be a clear method of forcing a more objective review process.

– – – – – – –
Duster,
Thanks for your comment.
I think you are suggesting that at a the request of the funding applicant any decision not to fund by the funding body should be explained in at an accessible public link.
It would tend to contribute toward integrity auditing of the funding body.
John

Doug Huffman says:
March 26, 2013 at 8:52 am
Please do not panic. Lots of ideas are floating around here. The idea that data must be published along with the publication of a peer reviewed article does not threaten the liberties of anyone.
As regards professional associations, they are found at various places along a spectrum. An association that teaches ethics and encourages its members to become comfortable discussing ethical issues does not threaten the liberties of anyone.
Keep in mind the context. The so-called “investigations” that white washed the Climategaters could achieve their end only because there were no widely accepted standards that could be applied to their work. In effect, the investigators made up their standards or non-standards during their investigations. That cannot be allowed to stand. The behavior of the Climategaters and those who exonerated them raised the questions that we are discussing.

Certainly the situation as it stands has to turn around or all of science loses out.
I find it personally interesting that while I know it is the scientists amongst us who are doing some heavy duty work to turn this scam around, and it is scientists who really will save this world from mad green policy – I nevertheless have a default reaction of mistrust whenever I hear “science says” now.
I hate that I feel that way. It surprised me that I should have that initial reaction to reject something outright simply because “science says”. It doesn’t matter what the field is. On some kind of subconscious level “Science = Dodgy” and that is just SO WRONG.
I’ve been coming in here daily for quite some time now – I’m addicted to this place. I love what I learn here, I love reading the comments and interacting with you all. I know that we will win out through using real science and the scientific method. You are all doing exactly what’s needed.
Yet there is a huge percentage of the population that does not come in here and does not know the good work being carried out to stop the madness that’s going on in the world. When this is all done and finished, it will be the “skeptical” aspect that will be remembered. “Science” will be remembered as being on the Bad Guy side.
So, if I’m feeling “science = dodgy” and “don’t trust ‘science says’,” and I KNOW that real science is winning – what the heck are THEY going to think? Especially when the only science that’s been rammed down their throats for 30 years has been the FALSE sort.
So, yes, this situation has to be turned around to keep ALL of science going down the toilet completely for the next hudred years.
After my initial mistrust when I hear “science says,” I have only one question and that is, “Where’s the data?”
Data and methods ALWAYS supplied up front is the way to go. It is the vital step. That, I would always trust.

Pressure of work prevents me from reading previous posts, so I may be repeating points already made.
The questions being asked in this poll makes it irrelevant for me to respond, but it raises issues that I have a strong opinion on. The problem is the phrasing of the question:
1. I’m an Electronic Engineer, so I would never be likely to join an atmospheric sciences organization.
2. Even if it was an engineering organization, I would not be likely to join either way. After more than 30 years in practice, I’ve not joined one yet.
However I do read papers from these organizations and I need to trust what I read.
What is perhaps more relevant to this poll is if it had asked whether I would give credence to a paper published by an organization that only published papers that were replicable up front.
Then my answer would be a definite “Yes”! As it stands, I can’t even say “Maybe”.

Gary Pearse says:
March 26, 2013 at 8:40 am
“Engineers generally have high integrity because the proof of their good or bad work is in the product. You can’t get away with cooking a design for a bridge or airplane, nor would one want to.”
Having worked for the government, I can’t say that about military contracts. As you are usually punished for making a design error by being paid even more money to correct it. Even belonging to a professional organization would have little effect in these cases as the government contractors are almost always sheltered from lawsuits. Me thinks that this is probably the major issue with the climatologists, too much grant & contract dollars from government sources. Now providing the data requirements along with the research paper (and any other requirement to try to ensure integrity) could easily be made a legal requirement in the grants/contracts. Interesting that this is not so.

Re AD Everard…all of science going down the pan.
This has recently become a recurring and very pertinent theme on this and other web forums.
What I have not often seen mentioned is that there is actually an agenda among so-called “Post Modernist” science academics to actively pursue the destruction of what has been known as “science”.
They do not balk at refuting the very existence of objective reality. It is actually taught now that the “discovery” of facts is an out-moded paradigm and that “science” is in future to be regarded as the “creation of knowledge” and implicitly, a cultural practice indistinuishable from argumentation about belief, based in ideology.
This is of course principally a disease of the Social Sciences. We see it in Lewandowsky. But it has metastised into forms now migrating into all other areas of activity. For example, via the generation of particular concepts and interpretations of “ethics” that will circumscribe the pursuit of knowledge even in engineering. More and more often we will discover that certain discoveries (or lines of development) have not been pursued because their implications have been deemed liable to”unethical” effect. “Ethics” is in my opinion purely ideological.
No amount of concern for the continued practice of science will avert the disaster this is going to bring upon Western societies. Its like worrying about decorating the facade of a building from which the internal structure is being quietly removed out the back door, beam by beam.
As I have said before, in my opinion Western culture is in its declining phase, but we can hope that the good it produced will be inherited by its successors in the East.

PS: NRC = Nuclear Regulatory Commission

“This poll did not load properly”
When clicking “Vote”

Maybe I am going against the grain here, but I do not think that being able to replicate the results from the published source code and data is not the most important aspect to improve. Most people – even scientists in the field – cannot spend time to replicate all but the most important papers. This includes the peer reviewers. Most people will just look at the conclusions.
There are three other ways that are much easier to understand.
Firstly is making novel statements – predictions about the world. This means making the hypotheses vulnerable to refutation.
Secondly is encouraging diverse approaches and different hypotheses to explain the same phenomena. Competition should lead to higher standards, including clearer presentation of results.
Thirdly is encouraging more “literature” reviews on the state of a particular topic. In economics there were a number of journals that did just that – The Journal of Economic Literature being one of the most prominent. This encourages comparing and contrasting different papers, making the subject more accessible, and also pointing to gaps in the knowledge. The UNIPCC reporting every seven years from a particular perspective is a much inferior outlet to many diversified approaches.
In summary there should be competition and pluralism – a reversal of the trend to block out contrary views and get scientific organisations to promote unitary views.

RE feet2thefire:
I don’t believe a Journal should limit itself to only replicated papers. A Journal exists to give desemination to observations and theories for which other people can replicate and build upon.
That said, I would certainly applaud an editorial / peer review comment about the reliability of each paper. In fact, Add it as a standard clause to the abstract. It can be a set of standard clauses such as
“This paper describes a one-time event or observation that cannot be replicated”
“Models from top to bottom. No raw data. Models not reviewable.”
“Raw data is not publically available”
“Raw data is provided. Process is not in public domain”
“Raw data and Processing code provided”
“Raw data and Processing code provided, results replicated by reviewers”
Remember the quote from Asimov: ”
The most exciting phrase to hear in science, the one that heralds new discoveries, is not “Eureka” but “That’s funny…” There needs to be a place to publish “something weird just happened. Any of you seen it, too?” The issue is that the paper needs to be clearly identified as to how established, how well founded, how well checked, or how one-off-weird is the phenomenon about to be described in the paper.

To be honest, I would prefer if they skipped the study/publication step and just gave us the raw data.
The study always has to say “global warming” while the processed data and the raw data that comes along with.the study says the opposite. So, I’d rather not have to read the same garbage over and over again only to find out that the data says the opposite.
The science does not allow the scientists to be objective. But the numbers are objective entities in and of themselves.

Tobias..your comment makes no sense; neither of us said anything about modern slavery.
On the contrary, the continued large scale practice of slavery throughout large parts of Africa, the Middle East and Asia is one more reason why the perpetual harping on about the relatively short-lived trans-Atlantic slave trade is a sick joke. Why the hell arent those people,instead of ranting about crimes of two centuries past applying their attention to the current practice of slavery?
Partly for the same reason that we have had slavery being practiced in Britain for more than the past decade with thousands of teenage girls being passed about as chattels without the authorities, until recently, doing a damn thing about it. That shibboleth has now been broken and without identifying the community responsible for sexual slavery in Modern Britain members solely of that community (whose victims came solely fromthe host community) have nonetheless been convicted of the crime in droves.
It is very useful for certain parties toharangue us continually with the historicalguilt about long ended versions of slavery. It would be very unhelpful to the agenda of those parties, on the other hand, to shine a spotlight on the current practice of slavery and invariable identity of those responsible.
One point in that regards: Saudi Arabia refuses to sign the relevant UN declarations on ending slavery.

wamron, sorry I expressed myself poorly, what I meant was that educated slaves years ago did a lot of the work as you mentioned, what I was (poorly) trying to say that currently today’s educated workforce is in much the same situation. They are slaves to their jobs and the companies they work for, trapped in circumstances that may have looked within their control as they started out with high hopes and ideals but soon are bound by forces beyond their control just a few years later, I am sorry but that is the best way I can describe it.

RE: Dan Hughes – “some thoughts” link to Rajagopal editorial
WHEW! That was tough sledding!
And for the most part…. Doubletalk!… Hiding behind words like “autoschediastically”, “ephectic”, “sagacious”, and “tyranny of consuetude” (for which Google could find usage only if transposed)
For the most part, he was complaining that there are commonly used metrics that purport to give an importance to papers, authors, and journals by their citations and linkages. By these very metrics, the University of Alexandria, Egypt is the 4th most important research university in the world. The citation metric system is being gamed.
His fourth paragraph is a beaut of obfustication. I distill it down to: We need metrics to rate things. So we have to quantify the unquantifiable. He conflates mathematical proof, established fact in science, with these heuristic ephemeral metrics so subject to gaming.

In the hands of a person of good judgment, even these seemingly unreliable metrics can prove to be of some use.
[his last sentence:] We should subject our subjectivity to face the light of objective metrics while tempering supposedly objective metrics by appealing to well-reasoned personal judgment.

And this guy is editor of an Engineering Science journal?
The famous speech from “I, Claudius” came to my mind. Citation metrics fail because all papers are not created equal. One perceptive paper is worth ten good papers and far better than a hundreds of junk papers.

Claudius: Senators, it is true I’m hard of hearing, but you will find it’s not for want of listening.
As for speaking, again it’s true I have an impediment. But isn’t what a man has to say more important than how long he takes to say it?
It’s true again I have little experience of government, but then have you more? I at least have lived with the imperial family who have ruled this empire, ever since you so spinelessly handed it over to us! I have observed it working more closely than any of you. Is your experience better than that?
As for being half-witted, well, what can I say? … Except that I have SURVIVED to middle age with half my wits while thousands have died with all of theirs intact! Evidently quality of wits is more important than quantity!

By now the question of peer review is almost obsolete. The areas of science that have ACTUAL UTILITY for humans have been operating outside the system of Publish Or Perish for a long time.
Medicine has its own clinical trial rules; engineering depends on acceptance by the customer; materials science is the same; agricultural research has close contact with farmers.
All other branches of science are basically forms of mental masturbation, and their little games are irrelevant to human progress.

Second try….I deal a lot with peer review and would warn aganst rating it too highly. It improves papers and may encourage debate, but really new ideas cannot be peer reviewed. I am worried what more emphasis on peer review (as promising more ‘truth’) would mean for the social sciences. Providing all evidence would require opening a ‘brain’ – all thougth processes – and a great deal of human thought would be dismissed altogether (see Polistra above). Instead patience is needed, natural science will win out in the end, but this may take a long time. Re CAGW, one hint….this is best explained as a social science phenomena , a part of history when solutions were looking for a problem. Here social sciences , especially interest analysis, provide better insights into what went on and who drove the AGW ‘scare’. But this is another story……its summary is that while the science debate keeps many people engaged and funded, no decison will be possible for a long time .If the ‘sceptics’ want to win the argument now, they must venture into the social sciences and analyse ‘science’ as a part of market-forces, selected and funded to support other groups. Big research science is in need of political support and this will only be ‘delivered’ if solutions are offered as well as the problem. Why was there so little response to the threat of global cooling, in the late 1980s, while that of global warming was welcomed with open arms by assorted bureuacracies?

Sonja Boehmer-Christiansen says:
March 27, 2013 at 4:09 am
I hope you will watch all http://en.wikipedia.org/wiki/Hjernevask and please read Richard Feynman’s original adress http://www.lhup.edu/~DSIMANEK/cargocul.htm. You will understand why I am so harsh on social science.

Sonja-B.C….how does it escape your attention that the Eugenics era culminated in the National SOCIALIST Party?
Why do you presuppose that rigour is necessarily “righ wing”?
Indeed, what do you even mean by “the Right”.Who, specificallyare these “Right” people who believe the things you say?
I think your comments only illustrate the problem in Social Science that the “correct” view / result / belief / finding is only ever to be those which are thought “better” or “preferable” for “ethical” reasons.

Sonja BC……………”I AM SHOCKED BY THE LOW QUALITY of debates…”
Well thats exactly what any of the thousandsof non-commenting readers of your comments will conclude about you.
we posit arguments…you disregard them.
Your pomposity is IMMENSE: “…here we get inti philosophy, not wanted here I imagine…”
well I was reading Popper before you were born…. I suggest you digest theTractatus b efore you pronounce yourself versed in philosophy.
You know what…I DIDNT BOTHER reading the rest of your adolescent drivel.
If you want to “debate” you have to abide by this basic principle: each person respondst o the preceding points of the other.
Anyone starting at the top and reading down will see how I comprehensively took you apart and stomped all over you in my previous comment. But you havent the balls to try to counter my pints or the savvy to recognise that you are now embarrassing yourself by continuing without so trying.
Keep coming here little child. keep posting. Keep demonstrating to the silent lurkers as well as the few commenters the kind of puillanimous excuse for a person that is destroying our culture every time you draw a breath.

Gary Pearse, dont waste your fingers typing, she’s just going to pigeonhole you as another “Right wing” OLD guy.
On second thoughts, keep typing………..every opportunity you give her to display her lethal combination of naiivete and arrogance just proves my assessment.
That’s why I’ve given her comments attention. I hope to elicit more gems like her previous ones.

These tweets and comments conflate peer review and replication. Peer review is a lightly-once-over pre-publication check for egregious boners and excessively unorthodox findings. Not the core issue.