A look at the original sting operation John Bohannon did on open access scientific journal review processing.
Guest essay by Howard Booth
In reading some of the responses to Chocolate covered science: The terrible state of scientific publishing that WUWT put up on May 30th, some respondents correctly alluded to the fact that John Bohannon’s target in that sting was more precisely the media not the process of getting published into a scientific journal. Mr. Bohannon was under a deadline to support the mocumentary being produced and didn’t want to get slowed down by a peer review process, so he short-cut the process by selecting journals he knew would have low publishing standards knowing that at least one of the 20 to which he submitted would surely pick him up. Several did, so the real sting of the media could begin.
But in rereading the article, it is interesting to note how and why he knew which journals to select to short-cut the process, which is related to why he was approached to do this mocumentary. Bohannon had done a previous sting operation on the open access journal publication process, which is what prompted the producers of the television show to reach out to him. In that original sting, he explored the somewhat murky world of open-access scientific publishers. The effort amounts to a research survey project of its own, and the results were published in Science Magazine article Bohannon wrote in October of 2013.
He created a hopelessly flawed research paper entirely from thin air making sure to splash it with enough obvious red flags that it shouldn’t have passed initial editorial consideration, let alone assignment for peer review. If somehow a journal would still offer to publish, he would then claim that he had found a fundamental flaw in the research which may mean it should be withdrawn, and of course some still offered to publish for a fee. Furthermore he gathered meta-data on the publishers to see how real they were, tracking email headers, web site IP addresses, and bank clearing house locations to see if their location matched their claims.
He plotted their locations and the results are quite interesting, with places like Africa, China, and India have some of the lower standards, but a significant percentage of western based operations also had poor records. He offered some anecdotes about interaction with respected publishing houses, and a couple of closures that happened as a result of his sting.
In total the “paper” was accepted by 157 journals, many of which have very official sounding names that parallel legitimate venues. Interestingly the pass/fail rate of approved journals in the Directory of Open Access Journals (DOAJ) was not that much different than the “predatory” list that Jeffery Beall maintains of dubious journals. A peer also suggested that the results would not have been that different had he taken the paywalled subscription-based group on, though that was not studied. The bottom line appears to be that if you know how to navigate the scientific publishing world, you can get almost anything published for a fee.
I recommend you review the entire article over at Sciencemag.org.
it’s all a business
Like organized crime
There must be a Racket, if there is to be any Racketeering.
No, it isn’t all business.
I’ve been a journal editor for a decade, a reviewer for two, and an author for three. I can tell you from experience there are plenty of academics and scientists out there with standards.
Yes, it is true that there are many others without any standards. It is indeed becoming a straight business issue for an increasing number of people.The new pay-to-publish journals are a mockery. Traditionally, the economic side only affected career advancement and questionable research took on a different form.
Nonetheless, the brunt of mishandling in academic/publishing is the same today as decades ago, namely, gate-keeping. I’ve lost count of the number of reviewers that through ignorance and conceit have torpedoed valid manuscripts while fast-tracking others without merit. It is a constant struggle.
(Note to academics/scientists starting out. Learn from rejections but don’t let them deter you. Revise and then resubmit elsewhere. If an author has 100 publications and another 10, the first one has been rejected far, far, far more times than the second. Take it to heart.)
Sadly, perhaps even tragically, this article (IMO) explains the problem. http://science.tdprofiti.com/sequestration-ushers-in-a-dark-age-for-science-in-america.html
Maybe a rethinking of how funds are allocated is required. Let’s all hold our breath for that.(sarc off)
There are lots of fake .pdf papers that have a professional format.
And then there’s Prof. Rick Trebino’s journal submission requesting a correction.
“66. Vow that, in the future, you will collaborate only with scientists
with short names (Russians are definitely out).
67. Thank your Chinese grad-student coauthor for having a last
name only two letters long. Make a mental note to include this
important fact in recommendations you will someday write to
her potential employers. ”
Absurdly brilliant, thanks for sharing Trebino with us.
Well, Science is a rotting corpse. Trebino’s complaint, however accurate and witty, merely affirms that it died from the head down.
This is the first time I saw Trebino’s article; the publication situation is worse than I imagined.
The Science Journals editors showed a consistent incompetence and malfeasance towards truth only rivaled by members of Congress.
Was wondering when this story was going to percolate up to a WUWT post.
Another someone in another thread — perhaps here even — noted that peer review is about ‘acceptability, not validity.’ It’s undeniably true that any vanity or other pay-for-print business model is not terribly interested in the role of gatekeeping any but the insufficiently profligate. But it’s just as true that a peer review model is not terribly interested in the role of gatekeeping any but the insufficiently conformist.
The entire cat fight about open-access vs. peer-review is whether a lack of peer pressure will produce more dissemination of scientifically sound results than that of institutionalizing peer pressure. But the entire purpose of the argument at all is to claim that science is holy based solely on the pronouncements of a scientist. And we know he is a scientist, because his friends say he is.
This is akin to the ‘hacker’ problem in software lore. That you are not a hacker unless a hacker calls you a hacker. So who then could possibly be the first hacker? And that since hackers are experts, as they are called experts by experts — save for the first hacker — then we need only believe the hacks they claim they’ve done.
The entire idea that anyone should be able to claim that hacks have been accomplished, or that anyone can or should implement the hack is completely beside the point. We know the hack works because a hacker claimed it did.
All of it, from get to go, is a textbook case of the fallacy known as Appeal to Authority.
The Media being itself just one more self-appointed authority. We know that journalists report the facts, journalists tell us so.
Hacking is what an amateur lumberman does to a tree trying to chop it down. Hitting the tree randomly with a sharp blade, or maybe not so sharp, hoping to leverage a weakness. Almost by definition then “hacking” is simply chopping away at someone else’s program, sometimes for worthy purposes and sometimes not.
Since it is sometimes illegal it is wise to not be too boastful about it. It could turn out the person that gave you permission doesn’t have authority to do so.
Science Fiction fans will immediately think of Thiotimoline
http://en.wikipedia.org/wiki/Thiotimoline
Ah, Theobromine, the Food of the Gods. And who may these mere mortals be who profane this substance?
================
Besides writing that fiction to gear up for his doctoral presentation, Asimov seems to have a sense of humor. In a work related environment, I created some fiction appearing for division wide management – nobody ever questioned my writeup.
m a liberal professor, and my liberal students terrify me
by Edward Schlosser on June 3, 2015
The press for actionability, or even for comprehensive analyses that go beyond personal testimony, is hereby considered redundant, since all we need to do to fix the world’s problems is adjust the feelings attached to them and open up the floor for various identity groups to have their say. All the old, enlightened means of discussion and analysis –from due process to scientific method — are dismissed as being blind to emotional concerns and therefore unfairly skewed toward the interest of straight white males. All that matters is that people are allowed to speak, that their narratives are accepted without question, and that the bad feelings go away.
So it’s not just that students refuse to countenance uncomfortable ideas –they refuse to engage them, period. Engagement is considered unnecessary, as the immediate, emotional reactions of students contain all the analysis and judgment that sensitive issues demand.
where did you get this quote, please and thanks?
Quick Google search turned up this, interesting read.
http://www.vox.com/2015/6/3/8706323/college-professor-afraid
Thank you Darrin – I had looked for Edward Schlosser on Duckduckgo and only found obituaries …
NRO, I think, on 3 Jun 15.
He’s taught his brand of tolerance, now let him tolerate it.
My thoughts as well. Solely blaming the Evil Wall Street Bankers for the collapse of Housing market is, to use his own words, “an oversimplification”, and ignores the millions of fraudulent mortgage applications. These applicants and the brokers who coaxed them were never prosecuted.
No the shoes on the other foot and the professor isn’t happy.
See the Boston Fed Study in:
http://www.rossmckitrick.com/uploads/4/8/0/8/4808045/thecaseforduediligence_fiedition.pdf
Reg, not just the mortgage applications, or the bankers, who, BTW, were forced into giving absurd loans. Fannie and Freddy led the way in piss poor loans and bundling MBS packages, and in extreme leveraging, and Janet Reno instituted a dozen major law suits against the banks to encourage these loans. Ivy league MBA also certified these as triple A.
Indeed many consumers were happy to take out tens to hundreds of thousands in loans, and then walk away from any repayments, while living for years in those homes. Many, after the collapse, then used that money to buy different real-estate. Greed appears to be a human quality infecting all human groups, be they corporate, Government, or individual.
It’s hard to figure where to draw the line, but 3 and 4th tier journals have always had a problem with quality control. Fortunately, when the reviewers make a mistake, it is corrected by subsequent submission. A good example is the recent paper by Richardson et al., 2015. (http://link.springer.com/article/10.1007/s11434-015-0806-z)
trafamadore says that the “3 and 4th tier journals have always had a problem with quality control.”
First off, where is the lack of “quality control”? Show us. Your comment is nothing but a drive-by ad hominem hit because your link goes to a Lord Monckton paper. It is disingenuous, and makes you no more than a troll.
Would you consider the journal Science to be “first tier”? How about Nature ?
Between them, those two journals published a dozen peer reviewed papers that in retrospect should have been questioned by the reviewers. They were fraudulent. Whereas the paper you linked to was never withdrawn like those in Nature and Science.
In fact, it is the first tier journals that have had the most trouble with fraudulent submissions, and of those, their pal reviewed ‘climate’ papers appear to be the most questionable.
Have you never read the Climategate emails? Michael Mann and others brag about controlling the climate peer review process. Any neutral observer would agree that climate peer review process is broken.
You found one series of papers (by one scientist) that were published in Science and Nature and then retracted. I think that’s a pretty good track record for journals that have published 100s of papers if not thousands.
The very topic of this posting is the lack of quality in the 3rd and 4th tier journals that Bohannon targeted with his fraudulent paper, and yet you attack trafamadore for merely confirming that fact.
You say the peer review process is broken. What is your specific proposal for a better method?
“Between them, those two journals published a dozen peer reviewed papers that in retrospect should have been questioned by the reviewers.”
And, for the record, John Bohannon did not identify Science or Nature in his sting operation. Or any top tier journal.
For decades and decades I have heard complaints about peer review. And for decades and decades, no one has come up with a better system.
“Your comment is nothing but a drive-by ad hominem hit because your link goes to a Lord Monckton paper.”
Actually, not. My link goes to a fine peer reviewed paper.
For the record, John Bohannon didn’t even try, due to time pressure. He explicitly says so. Your observation is pointless.
trafamadore
Actually, your link goes to a paper that is almost useless because the material is not very absorbent and is so course that it scratches.
Also, the paper is covered with nonsensical printed words which are not good decoration but add to the discomfort of using the paper for its only fit purpose.
Richard
In the end, it depends on having enough willing to take flac for opposing the group-minds. Kipling’s “If” comes to mind.
I call BS on this one. This is a “science orthodoxy” attempt.
Please look up a copy of “On the Electrodynamics of Moving Media”. It is 14 pages long, has NO references (there are some FOOTNOTES in published versions of it, they were NOT in the Original.) It had ONE reviewer, the editor of the “journal” it was published in.
IT WOULD NOT BE ACCEPTED OR PUBLISHED IN THE FORM IT WAS SUBMITTED IN ANY MODERN PEER REVIEWED JOURNAL TODAY.
Then I ask you, what is the more “modern” name of that “paper” in today’s assessment of it?
From an article in Slate, “How Gobbledygook Ended Up in Respected Scientific Journals” (my bold):
Of course it’s all about money. You don’t have to tell us paid shills of Big Oil, professional climate deniers, who are secretly terrified about our impending doom, only we can’t show our fear, and must always pretend to be unafraid, like the brave happy face of a clown who really wants to cry, when we’re really feeling totally opposite to what we say on WUWT, just so that our poor families have something to eat and we can drive Bentleys. /sarc /anti-sarc /sub-sarc /unsarc /BFsarc /sarclolcat /dancinghamstersarc /SkSsarc and /DeSmogSarc.
[Dancing hamsters? .mod]
A gratuitous reference to one of the “world wide web’s” first viral multimedia experiences. http://www.hampsterdance.com/ i.e. an ancient and powerful sarc.
From: http://www.sciencemag.org/content/342/6154/60.full
Who’s Afraid of Peer Review?
“Some open-access journals that have been criticized for poor quality control provided the most rigorous peer review of all. For example, the flagship journal of the Public Library of Science, PLOS ONE, was the only journal that called attention to the paper’s potential ethical problems, such as its lack of documentation about the treatment of animals used to generate cells for the experiment. The journal meticulously checked with the fictional authors that this and other prerequisites of a proper scientific study were met before sending it out for review. PLOS ONE rejected the paper 2 weeks later on the basis of its scientific quality. https://www.plos.org/open-access/ .”
Good to know.
…and this: http://www.sparc.arl.org/blog/science-magazine-open-access-sting
I see no ONE has noted that “On the Electrodynamics of Moving Media” is the 1905 paper by Einstein which is otherwise known as the “Special Theory of Relativity”.
TSK! It CLEARLY would NOT be publishable in modern “peer” (read CRONY) reviewed journals.