A scientist / journalist shares his story of a sting operation on the scientific publishing process with frightening results.
Howard Booth writes:
An interesting sting was done on scientific publishing to show how remarkably easy it is to get published despite doing a poor research project (in this case on purpose). John Bohannon, a PhD chemist and science journalist, was contracted to do a documentary for German television on the poor research in nutrition that produces outlandish claims about diet and dietary supplements. They conspired to create a research study intended to produce fantastic results, and see how often their poor research would get called out through the entire publishing process.
The study had a terrible sample size, poor quality control, and zero peer review. They managed to produce factually correct results that made an outlandish claim, write it up, get it published (with their choice of venues as many offered to pick it up), and get it splashed into the media with virtually no challenge to their methodology. It was picked up by several noteworthy outlets and on track to get picked up by Men’s Health magazine, but no doubt John’s confession of the sting will forestall that.
I know that WUWT is primarily concerned with CAGW, but I thought this was an interesting peek into the terrible state of the scientific publishing process, and the media’s inability to hold scientists accountable.
The full article can be read over at io9
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
What’s missing from the summary here is that they also set up an activist organization to promote the idea and feed talking points to media types. And *that* is what really closes the circle on it with respect to the IPCC.
But given that, I’m not at all certain how much weight we can place on fraudulent papers and/or fraudulent journals. Given that a whole activist org was constructed for the sake of duping people long term.
Just discovered a promising new word in the comments thread – ‘scientician’. A great descriptor for Michael Mann etc. I think the commenter was using it in the sense of diet scientists being more like dieticians, but to me it immediately the abuse of climate ‘science’ by politicians, climate “scientists”‘ political opportunism.
eek – immediately ‘brought to mind’ the abuse of climate ‘science’ etc.
The really sad part is mainstream journal editors at Science and Nature are fully in on the pseudoscience of today’s climate science, at least where anything related to the CMIP 3/5 climate models outputs are concerned.
Even Dr. Oz pushes pills on T.V.
Yes … now read the article at the following link and see how, at long last, the end has come for the greatest scientific fraud that has affected so many peoples’ lives:
http://www.rationaloptimist.com/blog/cholesterol-is-not-bad-for-you.aspx
I remember reading about this on the German MSM but can’t seem to find it right now.
“and on track to get picked up by Men’s Health magazine”
I am pretty sure that if the figures have numbers, that magazine would publish it. What an amazing “sting”.
Howard Booth said: “I know that WUWT is primarily concerned with CAGW, but I thought this was an interesting peek into the terrible state of the scientific publishing process, and the media’s inability to hold scientists accountable.”
Howard calls it the scientific publishing process, yet the sites included “the Daily Star, the Irish Examiner, Cosmopolitan’s German website, the Times of India, both the German and Indian site of the Huffington Post, and even television news in Texas and an Australian morning talk show.” I don’t see a single scientific journal in that list, just general news sites.
The very process – scientific peer review – which is decried by many WUWT commmenters as being riigged or exclusionary, is the only way to keep the quality of research papers high. It’s not perfect, but does someone have a better suggestion?
In the case of the chocolate paper, the folks who asked pertinent questions were online commenters. The effect is similar to something Eric Raymond said:
He’s referring to the way software is written but it applies equally to science. ‘Open science’ may be the cure for the abuses of the conventional peer review process. http://en.wikipedia.org/wiki/Open_science
The issues is that in public all those ‘peer review ‘ problems are kept of the radar while want is sold to the public is the ‘perfection of peer review’
To be fair that true is of all acedmic areas but climate ‘science’ is special so it has it own little issues , like the leaders in the filed trying to ring the system so only supporting papers are published, by such approaches has bullying journal editors and calling for them to be sacked if they fail to compile.
It is almost a given now that those who are brave enough to challenge ‘the cause ‘ by publishing research that fails to follow the consensus, have also to make some form of public admission of their ‘sins ‘ and are required to confirm that ‘faith ‘ in the cause . That is not a sign of healthy process in action, as the paper should stand on its own merits .
Chris,
You seem to have missed the fact that the research was published in a scientific journal:
Our paper was accepted for publication by multiple journals within 24 hours. Needless to say, we faced no peer review at all. The eager suitor we ultimately chose was the the International Archives of Medicine. It used to be run by the giant publisher BioMedCentral, but recently changed hands. The new publisher’s CEO, Carlos Vasquez, emailed Johannes to let him know that we had produced an “outstanding manuscript,” and that for just 600 Euros it “could be accepted directly in our premier journal.”
Although the Archives’ editor claims that “all articles submitted to the journal are reviewed in a rigorous way,” our paper was published less than 2 weeks after Onneken’s credit card was charged. Not a single word was changed.
The newspapers etc were responding to the press release announcing the publication of the paper.
It helps if you read the story to which the post refers.
Billy, I did read the story – but thanks for checking. I don’t consider online “pay for posting” sites to be scientific journals. That may meet your standard of quality, but not mine. Anyone remotely familiar with the trend in online publishing knows that there are now a myriad of sites that will take paid articles and papers, analagous to the advertorials published in commercial print magazines in the past.
In response to Chris:
Be sure to read the article for the details. Before going to media outlets, they first sought to get published in scientific journals. They submitted to 20 different applications, were picked up within 24 hours by several with offers, and chose the one that fitted their needs to give it enough authenticity before going to the media:
“Our paper was accepted for publication by multiple journals within 24 hours. Needless to say, we faced no peer review at all. The eager suitor we ultimately chose was the the International Archives of Medicine.”
The fact that the media didn’t bother to check the reputation of these journals or do any direct fact checking of the actual study shows just how easy it is to make headlines across the world with fantastic “science”. The only thing that stopped this train was the fact that the Bohannan didn’t want it to get too far and pulled the plug. My guess is that once it gets to something like Men’s Health, the Today Show and who knows the Nightly News could have been next.
I wonder if he could have done it with a sample size of one….
More of the same:
http://health.heraldtribune.com/2011/11/03/scientist-who-linked-meat-eating-to-selfishness-is-disgraced/
I wonder if there’s data on the number of papers whose conclusions differ from those they set out to show? Not many I’d wager.
Most of the science news in the media looks like agenda science, across the board.
retractionwatch @ur momisugly wordpress is rather handy 🙂
[…] Most scientists are honest and do it unconsciously. They get negative results, convince themselves they goofed, and repeat the experiment until it “works.” Or they drop “outlier” data points.
[…]The key is to exploit journalists’ incredible laziness. If you lay out the information just right, you can shape the story that emerges in the media almost like you were writing those stories yourself. In fact, that’s literally what you’re doing, since many reporters just copied and pasted our text.
All sounds a little too familiar.
Don’t forget to give them the PowerPoint so they can pitch the story to their editor
Reblogged this on gottadobetterthanthis and commented:
–
So, did you see the study that said you should eat chocolate to lose weight? Well, it was fake.
That is, a group set out to see if they could scam the science and nutrition publishers. They did.
Quoting Howard Booth, “They managed to produce factually correct results that made an outlandish claim, write it up, get it published”. In other words, they lied without really telling any lies, and nobody caught it.
It is tempting to think science for profit is to blame, but this still goes to the government. Sure, the profit motive blinded the publications and those involved, but the blame for the mindset and culture of publishing drivel lies squarely at the governments that fund so much of the related research that provides the grist for these publishing mills.
Governments should protect our property. Nothing more. When they try to help us, they always do more harm than good.
This:
http://io9.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800
is the same reference given in the main post at WUWT.
I love dark chocolate. I bought it, dark chocolate that is. Now I try to reproduce the results with chocolate covered strawberries. So far, the results are inconclusive. More research is needed.
“The study had a terrible sample size, poor quality control, and zero peer review.”
Sounds like most sceptic paopers to me
‘paopers’ ?, basic trolling is much improved if you spell correctly , otherwise you make your self look like a fool , HTH .
Anyone can get any rubbish published in the science journals.
https://thepointman.wordpress.com/2013/10/25/the-climate-wars-and-agent-deep-woolabra-wonga/
Pointman
Sometimes I’ll eat an entire 100g bar of 80/90% dark chocolate because its pure fat goodness. I’ve upped my saturated fat and cholesterol intake considerably while avoiding any grains and never looked of felt better.
I do the same. Works for me. High fat, low carb
Hey Jai,
Thanks for pointing out another example of surface warming without tropospheric warming! I don’t understand why you folks keep citing evidence that proves that your CO2 greenhouse theory is wrong? It might be about time for you to start looking for another explanation for the multitude of ‘evidence’ of surface warming that you are so obsessed with. Here’s a hint. It ain’t man made!
This is great news. If I eat more chocolate I will lose more weight.
Sorry, didn’t read past the Express headline.
Actually, I did, but even if the reporter did the actual leg work and got the story correct, the healine writer could blow it all away for a large percentage of the population.
Julian Simon, writing in his response to critics, which i think is pertinent to this discussion;
“Famous science writer Isaac Asimov expressed the bewilderment of a person who at least faced up to this intellectual predicament, as Ehrlich et. al. do not. Asimov read about the resources bet and then wrote:
Naturally, I was all on the side of the pessimist and judge my surprise when it turned out he had lost the bet; that the prices of the metals had indeed fallen; that grain was cheaper; that oil…was cheaper; and so on.
I was thunderstruck. Was it possible, I thought, that something that seemed so obvious to me – that a steadily rising population is deadly – can be wrong?
Yes, it could be. I am frequently wrong.
Asimov permitted himself to be bewildered. “I don’t understand this,” he wrote. And he says about economics in general: “I cannot understand it, and I cannot believe that anyone else understands it, either. People may say they understand it…but I think it is all a fake.”
Unlike Asimov, the doomsayers refuse to allow themselves to be bewildered by the facts. Instead, they simply reject the facts and deride anyone who presents the facts. Garrett Hardin writes:
To really get to the heart of the matter [population growth], we must ignore statistical arguments and opt for the common sense approach.
As the logician…Quine has said, ‘Science itself is a continuation of common sense. Therefore, this essay will avoid statistics. The opaqueness of statistical arguments makes it easy for analysts to “get away with murder”. Though often wonderfully useful, statistics can also serve as a substitute for thought…empirical studies …can be so selected and arranged as to seem to support faith in perpetual growth, the religion of the most powerful actors in a commercial society…
…On the one hand, a legion of economists say, “Why worry? An increase in people doesn’t matter…On the other hand, a brigade of environmentalists assert that shortages are real and ultimately decisive”.
In a debate with Dennis Meadows, every time someone asked him how he squared his Malthusian theory with the data I showed, he answered: “Simon looks at the past; I look at the future”. Scientific data, of course, necessarily refer to past.
In an article on the bet, columnist Jessica Mathews said that my views on “finiteness” are “palpable nonsense.” The word “palpable” means felt. No doubt Mathews feels that what I have to say is nonsense; it is indeed not common sense. But feeling is not a scientific argument. And assertions and policy conclusions drawn from feeling – as is often the case in these matters – are likely to mislead us when they run counter to the scientific evidence.
(Typically, Mathews attempts to marginalize me by referring to my ideas as “extreme”. She notes that Ehrlich lost the bet, I won, and then she asserts that the truth apparently is somewhere in between these extremes. Because she derided the bet as being on the wrong matters, I wrote to her: “Would you like to bet on any of these matters? If you can propose a measure or measures for worldwide pollution…I would probably be happy to wager on that one, also.” But she prudently did not respond.)”
Sound familiar to anyone?
“Asimov permitted himself to be bewildered…”I don’t understand this'”
Poor Asimov became “bewildered” because he confined himself to gazing at his own Navel. You’d think that these Great Seers would have recognized this pit-fall.
The mistake is simple. Asimov accepted the premise of Malthus that human beings are in general a net loss on the productivity ledger. In fact, they are in statist / socialist regimes such as the soviet union. However, in free societies, human beings are in general net profitable. By this I mean, that they produce more than they consume. Therefore, having more of them helps rather than hurts.
Probably there will be a defense of this chocolate paper in the MSM now that it has made the rounds. Once in print it is irradicable. Incidentally, not only is bad science published and echoed and re-issued endlessly but good scientific articles that go against the meme are not only ignored, but search engines like Yahoo lose them for you. I recalled a report by Dr. Adussamatov of the Russian Academy of Sciences from some years ago that noted that Mars’s ice caps were shrinking at the same time as those of earth!! This killer of CAGW got the Alinsky treatment. I tried to find it using Yahoo and got stuff on seasonal changes etc. I had to go to DuckDuckGo to get the link!! So not only does the MSM not report “contentious” issues on the subject, but bloody search engines do the same thing!!!!
http://www.canada.com/nationalpost/story.html?id=edae9952-3c3e-47ba-913f-7359a5c7f723
“NASA’s findings in space come as no surprise to Dr. Habibullo Abdussamatov at Saint Petersburg’s Pulkovo Astronomical Observatory….
Dr. Abdussamatov, one of the world’s chief critics of the theory that man-made carbon dioxide emissions create a greenhouse effect, leading to global warming.
“Mars has global warming, but without a greenhouse and without the participation of Martians,” he told me. “These parallel global warmings — observed simultaneously on Mars and on Earth — can only be a straightline consequence of the effect of the one same factor: a long-time change in solar irradiance.”
Please tell me why we are not interested in this?
I think that it was brought up several times on CA. It contradicts even what the luke-warmers say, sot it’s usually ignored and/or shouted down. Mention that the sun is not that constant and Leif will come out of the woodwork to assure everyone that it certainly is.
Interestingly I originally stumbled across this article in Yahoo, who had it splashed on their “current and trending” crawl on their main page.
while reading the original article at io9.com – i came across a link to another article with analogs to Alarmist behavior – it regards the pseudonymous “Food Babe” – who hates to be doubted – http://gawker.com/the-food-babe-blogger-is-full-of-shit-1694902226 – don’t be confused – the article’s author calls herself the “Science Babe” – but she has credentials
I posted this yesterday on “Tips and Notes”
Interesting example of how challenges to peer reviewed papers can be discouraged http://nymag.com/scienceofus/2015/05/how-a-grad-student-uncovered-a-huge-fraud.html
Anthony Watts. Astounding. What makes You ticking. Great.
Regards – Hans