From Stanford University:
Stanford researchers uncover patterns in how scientists lie about their data
When scientists falsify data, they try to cover it up by writing differently in their published works. A pair of Stanford researchers have devised a way of identifying these written clues.
Stanford communication scholars have devised an ‘obfuscation index’ that can help catch falsified scientific research before it is published.
Even the best poker players have “tells” that give away when they’re bluffing with a weak hand. Scientists who commit fraud have similar, but even more subtle, tells, and a pair of Stanford researchers have cracked the writing patterns of scientists who attempt to pass along falsified data.
The work, published in the Journal of Language and Social Psychology, could eventually help scientists identify falsified research before it is published.
There is a fair amount of research dedicated to understanding the ways liars lie. Studies have shown that liars generally tend to express more negative emotion terms and use fewer first-person pronouns. Fraudulent financial reports typically display higher levels of linguistic obfuscation – phrasing that is meant to distract from or conceal the fake data – than accurate reports.
To see if similar patterns exist in scientific academia, Jeff Hancock, a professor of communication at Stanford, and graduate student David Markowitz searched the archives of PubMed, a database of life sciences journals, from 1973 to 2013 for retracted papers. They identified 253, primarily from biomedical journals, that were retracted for documented fraud and compared the writing in these to unretracted papers from the same journals and publication years, and covering the same topics.
They then rated the level of fraud of each paper using a customized “obfuscation index,” which rated the degree to which the authors attempted to mask their false results. This was achieved through a summary score of causal terms, abstract language, jargon, positive emotion terms and a standardized ease of reading score.
“We believe the underlying idea behind obfuscation is to muddle the truth,” said Markowitz, the lead author on the paper. “Scientists faking data know that they are committing a misconduct and do not want to get caught. Therefore, one strategy to evade this may be to obscure parts of the paper. We suggest that language can be one of many variables to differentiate between fraudulent and genuine science.”
The results showed that fraudulent retracted papers scored significantly higher on the obfuscation index than papers retracted for other reasons. For example, fraudulent papers contained approximately 1.5 percent more jargon than unretracted papers.
“Fradulent papers had about 60 more jargon-like words per paper compared to unretracted papers,” Markowitz said. “This is a non-trivial amount.”
The researchers say that scientists might commit data fraud for a variety of reasons. Previous research points to a “publish or perish” mentality that may motivate researchers to manipulate their findings or fake studies altogether. But the change the researchers found in the writing, however, is directly related to the author’s goals of covering up lies through the manipulation of language. For instance, a fraudulent author may use fewer positive emotion terms to curb praise for the data, for fear of triggering inquiry.
In the future, a computerized system based on this work might be able to flag a submitted paper so that editors could give it a more critical review before publication, depending on the journal’s threshold for obfuscated language. But the authors warn that this approach isn’t currently feasible given the false-positive rate.
“Science fraud is of increasing concern in academia, and automatic tools for identifying fraud might be useful,” Hancock said. “But much more research is needed before considering this kind of approach. Obviously, there is a very high error rate that would need to be improved, but also science is based on trust, and introducing a ‘fraud detection’ tool into the publication process might undermine that trust.”
###
It’s easy to spot fraudulent papers. Just look for the terms “global warming”, “climate change”, “climate disruption”, and/or “unprecedented”.
Opening sentence to wapo article:
“The Obama administration is continuing to resist efforts (not obfuscate, but stop Republicans politically motivated prying) by a top House Republican to gain access to the internal deliberations(How dare Repubs look at INTERNAL government deliberations) of federal scientists who authored a groundbreaking (IT’S GROUNDBREAKING! No bias there) global warming study the lawmaker is investigating.”
Same paper that published two leaked pieces of info from a CIA report. Bush then declassified the document that showed how out of context the info had been presented. Then wapo demanded that Bush declassify all the intelligence reports that went into producing the CIA report.
Then when the woman who had leaked the info got caught by failing a lie detector test the wapo published a front page story on how lie detectors aren’t accurate.
Bah. Доверяй, но проверяй (“Trust, but verify”).
Even now, I’ll hazard that as news of this paper reporting this work has gotten out, there are “white hat” hackers who are devising software to automate the identification and assessment of fraud indicators in all sorts of publications.
Betcha we’ll soon be hearing “There’s an app for that!”
Try to count the number of times Michelle Wie says, “Um” in this vid:
At least she doesn’t say “you know” over and over. Which she does about 15 times here:
Then realize she has a degree in communication from Stanford.
My expectation of Stanford communication scholars is low.
Lying, obfuscation, and distortion has a strong origin in politics, but soon spread to law, media, advertising and then to science, medicine and meteorology. It remains rare in engineering and architecture as it is obvious when something falls down on people.
Missing from the entire discussion is the one and only way to fight bad or fraudulent science both effectively and consistently, and that is through the forced imposition of very high standards of transparency and reproducibility, in all cases, no excuses.
Let the public see every single thing you did, every step of the way, to arrive at your conclusions. Ease the way as much as possible for others to replicate your work. Provide every bit of data, every bit of software, everything. If you say that key bits are proprietary or private, fine. Just don’t expect anyone to give much weight to your conclusions – that is the tradeoff you make.
This horse isn’t quite dead yet, so allow me another whack or two.
If you want to strike a serious blow against crock science, then devise simple metrics to score both transparency and reproducibility of each and every published paper. This is a very doable project. Don’t be shy – set the standards high. If a paper doesn’t score high on both measures, then it should be dismissed then and there, as the authors themselves have demonstrated no confidence in it. If it does score high, others should have little trouble confirming or rejecting it.
Anything effectively tolerant of low transparency and reproducibility simply will not get the job done. Low transparency = high dishonesty, and to believe otherwise is to live in la-la-land.
(PS: Yes, I understand some studies are inherently difficult to replicate. Their numbers are not at all a deal-breaker. Don’t let the perfect be the enemy of the good.)
Lie is a big word, be careful when you use it.
lie.
K
Yeah, the little squiggle has 3 meanings: uttered falsehood, to be in a recumbent position and the distribution of humps and hollows and slopes in the land (golfers ‘lie of the land’ or lay of the land where the ball is and figuratively ‘the facts of the situation’
It’s possible to do perfectly valid research and come to misleading conclusions by picking your subject carefully. I read a news article where a study was done by growing poison ivy, oak and sumac under increased CO2. They grew better, implying increased CO2 was bad. But they didn’t try growing any beneficial plants or food plants under the same conditions!
I am become Immediately suspicious of any paper which uses the term “robust” in any context other than culinary.
As I understand it, bullsh*t-o-meters already are patented and sold through most farm supplies stores. Alternatively, peer reviewers might just want to hire a high school English teacher used to grading “research papers”.
“For example, fraudulent papers contained approximately 1.5 percent more jargon than unretracted papers.
“Fradulent papers had about 60 more jargon-like words per paper compared to unretracted papers,” Markowitz said. “This is a non-trivial amount.”” By “non-trivial” I assume they mean significant.
That means there are 4000 jargon-like words per paper.
The authors used papers published at about the same time and on the same subjects as the withdrawn ones. However, this small difference is very unlikely to persist across papers published on different subjects and at different times.
You would never be able to spot a fraud by comparing a climate research paper with a medical or physics research paper, or even one from the same field from a different time.
The trouble these researchers/scientists create when they falsify facts just to get press coverage and justify their grants is that they can have devastating effect on the businesses of those associated with the subject of the research. In the UK recently there was extensive coverage of a paper purporting to show a ‘possible’ link to cancer from eating bacon and sausages. The words ‘might be’ or ‘could be’ were used in the press release. Three weeks later it was reported that sales of those products in the UK had crashed by more than 15 per cent. This is a tragedy for the British pig industry which has suffered considerably from a downturn in sales over the past ten or more years.
Unfounded statements that can have such profound impact on segments of business and peoples livelyhoods coming from these idiots, should be banned unless the point made is conclusively proven by totally independent peer revue, or a simple analysis of the research documents to support the press release. If then it is found that the facts have been falsified, or created just for newsworthiness, then those responsible should be punished severely as a warning to others. Industry should not be left to the mercy of these selfish people. .
Bacon Sausages link to cancer:This was the World Health Organization recommendations. ie UN need I say more. The reason I became involved in climate change is because, based on their science and their recommendation I became Diabetic, Obesity, High Blood Pressure Serious Heart Problems and Sever Arthritis all based on unproven science. I was on a whopping 6 meds a day I am drug free for 18 month. Please view you will see how the fix is in re Paris. https://www.youtube.com/watch?v=vRe9z32NZHY you can start at the 26 min mark which will inspire you to view all.
Russell, your comments are interesting. I had an elderly lady friend who was taking 14 different tablets a day ostensibly to ease her severe arthritis problem. As she felt worse over time she was prescribed more different drugs to ease her pain. She got so depressed by the whole business that she decided that she would not take any more tablets, and would you believe, she felt considerably better for ten years prior to her recent death of old age.
I myself have been anti tablets/drugs for over 50 years, and would only be persuaded to take them in a life or death situation.
Writes George Lawson:
The effects are far more invidious than this.
Because “peer-reviewed literature” is received as “drop-dead reliable” by almost all those working in the various disciplines of science (if one submits a manuscript to an academic journal, one is expected not only to support each factual assertion therein with the citation of what is considered a trustworthy source with a good reputation – for example, in my case [I’m a physician] a report of investigatory research published in a “high-impact” journal like The Lancet or Oncology or NEJM – but also to provide either a hard copy of the cited reference with attention drawn to the specific passage “with circles and arrows and a paragraph on the back of each one explaining what each one was” or a PDF of said element – similarly marked – as an email attachment).
ANYTHING “these researchers/scientists create when they falsify facts to get press coverage and justify their grants” – even if it’s later retracted by the journal which had published such charlatanry – rings down throughout the sciences like a tolling cathedral bell through the streets of the town, and distracts many, many more published “peer-reviewed” articles and textbooks from adherence to factual reality.
The effects of duplicity (and failures – deliberate or inadvertent – in the error-checking function of peer review) are not only pernicious as regards “the businesses of those associated with the subject of the research” but in fact afflict and impair everyone working in the sciences, from Chief of Department on a university faculty to the technicians sweating down in the bilges of a commercial laboratory in Teaneck, New Jersey.
The pathology multiplies and spreads.
Damnable.
Any wonder that when I was reading through the Climategate emails this time in November 2009, I wanted personally to get my hands around the throats of every lying bastard among “the consensus in client” and squeeze until the particular perpetrator of this malicious mischief had become suitable for the embalmer’s art?
I don’t understand auto-moderation. For example, the word “fra*d” will send your comment to moderation even if you are quoting the post itself! Or if the word is embedded in a link. And yet, the topic of discussion here at wuwt is often the “f-word”. It is often about deceit, scam, chicanery, duplicity, and hoax. What is wrong with using the F-Word to describe Fra*d?
markstoval justifiably complains:
Bah. I’m on “permanent double-secret probation,” so everything I post gets automatically “moderated.” It’s a mockery, but what can be done? There are entirely too many fainting goats masquerading as human beings infesting the blogosphere, and everything must apparently be reduced to bland pap in order to prevent these hircine specimens from going into screaming hissy-fits at the least little provocation.
It’s like forking manure. One hopes the composting process will find some acceleration, but the stench must be borne regardless.
Sorry to hear you are on double-secret probation. I have had those sorts of troubles myself in times past. I suppose I should have guessed since I often find myself in agreement with your comments when I see them.
I find that those of us (just taking about myself here, not necessarily you) who are convinced by the laws of physics and thermodynamics that CO2 can not do what the IPCC, James Hansen, and even the lukewarmers think it does are often not as welcome here as even the alarmists. (because the alarmists are so obviously wrong and real skeptics have the laws of science on our side? who know?)
Anyway, hang in there. Many see your posts when they are paroled.
Oh, and great quote there. It sounds like Orwell could have said it.
http://www.theburningplatform.com/2015/03/27/congressman-knows-regular-lobbyists-order-without-even-having-to-be-told/
Not your first rodeo, Doc.
The wonderfully maladroit dp links to a draw from The Onion and then adds as a remonstrance to me:
Wonderful. This schlemiel really, truly, and sincerely is as gullible and cement-headed as impressions lead all and sundry to surmise.
Or perhaps he refers to the scrambling desperate assholery of that particular Web site’s self-invested “we” in seeking to draw an apology for my action on a purely charitable impulse when it was readily presumed that one of them had assumed that said bit from The Onion was a genuine news item.
Tsk. dp doesn’t know how to handle Social Justice Warriors at all, does he?
If the moderators see the context of the stop word is acceptable the comment will be posted. The stop words list is a limitation of WordPress. It is a simple pattern match and has no intelligence. Use your own.
Writes dp:
Wrong. There is an undeniable “intelligence” behind it, whether “The stop words list” is a consequence of a pervasive and Gadarene litigiousness in these United States or simply moral cowardice writ into policy by the proprietors of WordPress (their property; their right, but either purposefully “politically correct” or simply contemptibly craven nonetheless).
Such a “pattern match” had been deliberately written into the operating software, and what had been written in can be written (perhaps opted?) out. Not all Web logs cybernetically “moderate” their comments to suit either Mrs. Grundy or the Social Justice Warriors.
But the f-word does not send comments to moderation at other WordPress blogs. Why is that?
Tucci – I’m beginning to agree with you and suspect your claims regarding Anthony’s policies have merit. My only regret is he hasn’t banned you entirely. One can still hope.
In response to my contention that “…whether “The stop words list” is a consequence of a pervasive and Gadarene litigiousness in these United States or simply moral cowardice writ into policy by the proprietors of WordPress (their property; their right, but either purposefully “politically correct” or simply contemptibly craven nonetheless).
“Such a “pattern match” had been deliberately written into the operating software, and what had been written in can be written (perhaps opted?) out. Not all Web logs cybernetically “moderate” their comments to suit either Mrs. Grundy or the Social Justice Warriors” we now have dp expressing his scrabbling filthy little inner censor-wannabe with his opinion that my:
Now, out of what mephetidic recesses of (envious?) obliterative viciousness did that little spurt of venom emerge?
[??? .mod]
For people who wish to understand the problem of running an open access blog go have a look at wp-shield, a WordPress plugin that uses stop words as well as heuristic methods to scan comments as part of the approval process. It is not the only plugin that does this and does not appear to be the one used here, but is typical of what these auto-screeners do. They are a necessary tool because comment bots are large in number, economical, and work faster than moderators can scrub undesired comments and do so with an acceptable error rate. And comment spam bots are not the the only concern. There are people who through ignorance or cunning cross the line that says liable be found beyond here and not dealing with the problem preemptively can cost a blogger his livelihood or worse.
And be careful of Tuccii78 – he’s been reading the big dictionary again.
Confirming that sensitivity about his illiteracy gives him to hate and snipe at those with greater facility of expression, we’ve got dp proving that the Green Monster is more than an architectural phenomenon, whining:
Tsk. Insofar as “the big dictionary” pertains, that’s Stedman’s Medical, and I got through that before I’d finished third year. I’d swallowed the largest edition of Webster’s in the library by the 8th grade, when I was my parochial school’s great hope for a victory in the big regional spelling bee.
Who bothers going back, except on the principle of “доверяй, но проверяй”?
“science is based on trust” This statement is very revealing. Trust is a synonym of faith which is what I thought was the ‘enemy’ of science.
Sure; I trust that I can replicate your published results, using your published data and methods.
I trust myself to be the first person I am likely to fool.
I trust my ability to be mistaken.For my pattern seeking mind will invent patterns at need.
See scientific method is based on trust.
If this tool were perfected it would become another tool for the writer, not the publisher, used like a spell checker, grammar checker, or reading level checker. The purpose would be to better conceal fraud. Can you imagine who the customers of such a tool would be?
Consider this: 1) Can you imagine a science editor self-admitting they need such a tool? Wouldn’t that be a kind of career death to admit you can’t spot fraud? Isn’t that actually a requirement of being an editor? 2) Can you imagine a science writer claiming they believe they can conceal fraud even though they have no training to do so? (Note to self – that last part may be false.)
To eliminate fraud you have to put fraudsters and their co-conspirators in jail and make a big deal about how their betrayal to science destroyed their families and careers (It also makes more sense than giving them better jobs with more responsibility as is the norm today). They may not care about their families, but a career death hurts to the brittle bones of these egoists. And that is why there will never be an effort to clean up their act.
Case in point: Why is Peter Gleick still called a scientist? How can he have anyone’s trust? Why hasn’t the scientific community punted him out of it? Because the rule of golden rules is whoever owns the gold makes the rules. The scientific community has no interest in self-improvement.
Perspicaciously, dp writes:
Ouch. Nice pick-up. In this light, I can readily foresee a market for this kind of software among high school, college, and especially grad school matriculants all over the planet.
I’ve found the easiest way to tell when a scientist is lying is that they use absolute numbers when they are high (11,944 abstracts/12,465 papers) but they switch to percentages (97%)when the numbers are low (75 of 77 actual scientists) without citing the absolute numbers that the percentage was derived from. Another ‘tell’ is when they use backwards logic to say things like, “0.7% of the papers explicitly denied a human cause, therefore 99.3% must agree that there is only a human cause”. Which of course negates the possibility that there are other answers, or that sometimes we don’t know the answers.
Some wonderful lecturer would be scheduled to give a lecture, at great expense to the university, we students knew was mostly BS. Before the event, buzzword checksheets would be printed up and smuggled to the attending students throughout the auditorium, beyond if it were broadcast. As the wonderful lecturer would say each buzzword, you could see intense students “making notes”, which really impressed the university brass who thought we were very attentive. As the lecture continued, suddenly, someone would shout, “BINGO!” and stand up all excited. The rest of the crowd would groan and the lecturer, still too naive to figure out what was going on would drone on and on, ad nauseum, as more and more attendees fell asleep, their dreams of winning a night with a co-ed, shattered….
This wasn’t our idea. Its roots are in the halls of the finest engineering school on the planet, MIT and its Model Railroad Club Hackers….
How can you tell when a climatologist is lying to you? The strings on his mouth parts are moving.
About scientists ‘sex up’ their data… it reminds me of the resignation letter (October 6, 2010) to the American Physical Society, of the distinguished professor of Physics, Hal Lewis, and the following extract from that letter:-
When I first joined the American Physical Society sixty-seven years ago it was much smaller, much gentler, and as yet uncorrupted by the money flood (a threat against which Dwight Eisenhower warned a half-century ago).
How different it is now. The giants no longer walk the earth, and the money flood has become the raison d’être of much physics research, the vital sustenance of much more, and it provides the support for untold numbers of professional jobs. For reasons that will soon become clear my former pride at being an APS Fellow all these years has been turned into shame, and I am forced, with no pleasure at all, to offer you my resignation from the Society.
It is of course, the global warming scam, with the (literally) trillions of dollars driving it, that has corrupted so many scientists, and has carried APS before it like a rogue wave. It is the greatest and most successful pseudoscientific fraud I have seen in my long life as a physicist. Anyone who has the faintest doubt that this is so should force himself to read the ClimateGate documents, which lay it bare. (Montford’s book organizes the facts very well.) I don’t believe that any real physicist, nay scientist, can read that stuff without revulsion. I would almost make that revulsion a definition of the word scientist.
I was talking to someone yesterday about all the “lies” that (sic) “scientists” have thrust on us over the years, ie, eggs, whole milk, beef fat and CO2 were a real no-no for years but now we find that this research wasn’t correct and those type of fats aren’t necessarily bad for us, and neither is CO2?
I don’t know about the rest of you, but I have a very low opinion right about now where (sic) “scientists” in general are concerned. I believe the MAJOR problem is government funded studies which push certain outcomes and if that isn’t achieved then the MONEY stops!
So lets face it, in the end it IS all about the MONEY and government needs to GET OUT of the science business!
“Lone Gunman
November 27, 2015 at 5:45 am”
In terms of fat in diets, we have been led down a path of lies. Look up “French Paradox” in terms of diet and rich fatty foods. They have some of the best health records in the EU zone.
Some wise words of caution quoted here-
http://blogs.news.com.au/heraldsun/andrewbolt/index.php/heraldsun/comments/another_climate_scare_debunked_professor_urges_calm/
Not all scientists are in a tizz over this new kid on the block theory and the emotional hyping that goes with it.
Meanwhile back at warming HQ-
‘Researchers have found long-term warming in the Pacific and Indian oceans played a role in making Queensland’s 2011 floods far worse than they otherwise would have been.
The research, published in the American Geophysical Union’s Geophysical Research Letters, found the “warmer background state” in the water around Australia increased the likelihood of the extreme rainfall….
“Many past studies have found a global warming link to heat extremes,” she said.
“This study is one of the first to show how ocean warming can impact a heavy rainfall event.
“As we come into climate change talks in Paris, this research offers yet another incentive for countries around the world to take action to forestall global warming.”‘
http://www.smh.com.au/environment/climate-change/ocean-warming-worsened-2011-floods-researchers-20151127-gla6zb.html
But but our esteemed ex Climate Change Commissioner Tim Flannery told us global warming would reduce rainfall so much so that what rain did fall wouldn’t create enough runoff to fill our dams and then the floods came.
You need to get your heads together and get your stories straight guys.
The Sokal affair comes to mind. ‘Transgressing the
Boundaries Towards a Transformative Hermeneutics
of Quantum Gravity’ Alan Sokal’s hoax article that was
published in a post modern journal.