A scientist / journalist shares his story of a sting operation on the scientific publishing process with frightening results.
Howard Booth writes:
An interesting sting was done on scientific publishing to show how remarkably easy it is to get published despite doing a poor research project (in this case on purpose). John Bohannon, a PhD chemist and science journalist, was contracted to do a documentary for German television on the poor research in nutrition that produces outlandish claims about diet and dietary supplements. They conspired to create a research study intended to produce fantastic results, and see how often their poor research would get called out through the entire publishing process.
The study had a terrible sample size, poor quality control, and zero peer review. They managed to produce factually correct results that made an outlandish claim, write it up, get it published (with their choice of venues as many offered to pick it up), and get it splashed into the media with virtually no challenge to their methodology. It was picked up by several noteworthy outlets and on track to get picked up by Men’s Health magazine, but no doubt John’s confession of the sting will forestall that.
I know that WUWT is primarily concerned with CAGW, but I thought this was an interesting peek into the terrible state of the scientific publishing process, and the media’s inability to hold scientists accountable.
The full article can be read over at io9
Sweet…
The correlation with “Diet Science” and CAGW is highly appropriate.. Although at least in diet quackery you get to see some before and after.pics of cute chicks
Reminiscent of the Sokal Affair.
But the msm will look at this and say ‘how stupid could they be?’ with no recognition that they are as much at fault.
“fee-charging open access journals”
I recall a story about global warming causing earthquakes published in one of those that was picked up by MSNBC.com.
As he said
“We needed to get our study published pronto, but since it was such bad science, we needed to skip peer review altogether. Conveniently, there are lists of fake journal publishers. “
And he found one. But whose responsibility is it that fake journals exist? And what should be done about it?
Were any scientists involved in this story at all?
the LaCour affair is interesting too
Just say its a consensus and it MUST be true
To be fair such awful research practices are not only accepted but an honoured part of climate ‘science , therefore this type of story would have no impact .
Thanks for the heads up. Expect a cake of bitter chocolate to feature prominently on the cover of the next IPCC report.
😀
Oh, I’ll just bet their lawyer put in a fudge factor.
(Like: Results not guaranteed. Failure to follow precisely all 35 steps on page 91 (Appendix II) of “Directions for Use” render all implied or express warranties null and void.)
BEFORE — no make-up; depressed facial expression; dumb hairdo; standing face on to camera; harsh lighting
AFTER — MAKE-UP (big time); BIG SMILE; great hair; 3/4 to camera; flattering lighting
Oh — brother! Who — believes — these — ads — anyway??
The fact is that journalism deals with the regularization of gossip.
The fact is that a news report that includes details about the methodology of a scientific/academic study will not lead readers to a magical understanding of the assumptions and limitations at play in a given field of inquiry. This understanding can only be achieved by systematic study over many years and, even then, it will still prove elusive for many.
The fact is that journalists often pimp their offerings with hysterical attention grabbers and, consequently, the readership gradually becomes unresponsive.
It is often said (around here) that the “liberals” control education and the media. If this is so, it only demonstrates their utter failure at indoctrination. The electorate keeps showing time and again that their understanding of current events is not securely bound by these alleged gatekeepers. For instance, there is no guarantee whatsoever that the Democrats will win the next US presidential election.
Stephen McIntyre has commented on the nonsense in reports of medical research.
I see over-hyping and extrapolation, some of by scare-mongers (especially environmentalists) and some of it by persons promoting their own product. I’ve seen what I suspect are malicious attacks – e.g. by an olive oil nut against Canola oil, a much lower priced alternative.
Some of you may have read me telling of the idiot cousin who claimed that Canola oil is toxic because:
– a grandfather used it to kill aphids (likely it smothered them)
– Canola is in the mustard family and mustard gas was used as poison in WWI. (The idiot or his source he didn’t question failed to understand that “mustard gas” meant smell and colour of a manufactured chemical. He didn’t think that mustard condiment is in the mustard family, let alone that people he knew well ate many turnips when he was a child.)
Scary thing is such people have a vote.
Well, according to a 2011 study, chocolate is one the things “threatened” by climate change. Yes indeed, the children aren’t going to know what chocolate is.
No more going cuckoo for cocoa puffs. No more s’mores. So sad.
For more things threatened: http://www.theguardian.com/vital-signs/2014/oct/29/diet-climate-maple-syrup-coffee-global-warming
With 85.2% of the rated capacity of Germany’s 25,000 bird blenders doing nada the entire year, it is nothing short of extraordinary that these exceptional scientists were able to pull off their fine research. And in only a couple of weeks. Truly, there are no barriers to the rigorous expansion of human knowledge. /chocolate
http://notrickszone.com/2015/02/07/germany-2014-report-card-is-in-its-25000-wind-turbines-get-an-f-averaged-only-14-8-of-rated-capacity/#sthash.AQdV3SsA.dpbs
Jo Nova is on the case with regard to Cholesterol …
http://joannenova.com.au/2015/05/cholesterol-how-the-web-and-books-are-years-ahead-of-consensus-and-govt/
But the research showing red wine is good for you is solid, untouchable, settled science.
And don’t try telling me otherwise.
We are paying a very high price for the inability of the media to report on science. Or anything else that is a tad complicated. The Fourth Estate is profoundly broken.
The problem is with (i) predatory open access journals, (ii) scientifically illiterate journalists and public, and (iii) the corporate media echo chamber.
My aunt used to tell my mother back when I was a little one, “Leave him with me, Margaret, and I’ll fatten him up.” Alas, my mother never did.
I have a real soft spot for Reese’s Peanut Butter Cups. And, I’ll swear on a stack of Bibles that the following is true. My gastroenterologist told me just the other day, “As your doctor I advise you to eat Reese’s Peanut Butter Cups.” Life has mysteries science can’t answer.
But… But… this is happening in SCIENCE!
And it always has. Humans are involved, human behavior and failure apply.
This is why skepticism is such a good idea.
Amen.
Love chocolate. Too bad all the chocolate trees are melting due to global warming. 🙂
Waiting for the study about the benefits of beer.
This just shows that it is important to check the sources of your information. Unfortunately, many news sources are eager to just copy/paste and publish, because the speed with which the news is brought has become very important. Also there are journals that are happy to make a quick buck and not care about a decent peer-review process. Looks as though at the moment a lot of the media are not very well-equipped to report on scientific issues, which basically means misinformation of the public.