From the “everything is robust” department.
We’ve long noted at WUWT that the word “robust” has seen a significant rise in usage in climate science papers, becoming a favorite word to use when statistical Spackle has been applied to climate data. Now there’s evidence from a new study suggesting that observation is spot-on.
From Nature:
‘Novel, amazing, innovative’: positive words on the rise in science papers
Analysis suggests an increasing tendency to exaggerate and polarize results.
Philip Ball
Scientists have become more upbeat in describing their research, an analysis of papers in the PubMed database suggests.
Researchers at the University Medical Center Utrecht in the Netherlands say that the frequency of positive-sounding words such as ‘novel’, ‘amazing’, ‘innovative’ and ‘unprecedented’ has increased almost nine-fold in the titles and abstracts of papers published between 1974 and 2014. There has also been a smaller — yet still statistically significant — rise in the frequency of negative words, such as ‘disappointing’ and ‘pessimistic’.
Source: Ref. 1
Psychiatrist Christiaan Vinkers and his colleagues searched papers on PubMed for 25 ‘positive’ words and 25 ‘negative’ words (which the authors selected by manually analysing papers and consulting thesaurus listings). The number of papers containing any of the positive words in their title or abstract rose from an average of 2% in 1974–80 to 17.5% in 2014. Use of the 25 negative words rose from 1.3% to 2.4% over the same period, according to the study, published in the British Medical Journal on 14 December1.
Rising hype
The most obvious interpretation of the results is that they reflect an increase in hype and exaggeration, rather than a real improvement in the incidence or quality of discoveries, says Vinkers. The findings “fit our own observations that in order to get published, you need to emphasize what is special and unique about your study,” he says. Researchers may be tempted to make their findings stand out from thousands of others — a tendency that might also explain the more modest rise in usage of negative words.
The word ‘novel’ now appears in more than 7% of PubMed paper titles and abstracts, and the researchers jokingly extrapolate that, on the basis of its past rise, it is set to appear in every paper by the year 2123.
…
But Vinkers and his colleagues think that the trend highlights a problem. “If everything is ‘robust’ and ‘novel’”, says Vinkers, then there is no distinction between the qualities of findings. “In that case, words used to describe scientific results are no longer driven by the content but by marketability.”
A BBC story here says the use of the word “robust” has gone up 15000% They write:
Despite working with facts, figures and empirical evidence, the world of science appears to have a growing addiction to hyperbole. Researchers at the University of Utrecht in the Netherlands looked at four decades worth of medical and scientific publications, and found a significant upwards trend of positive words. We’ve all heard of those ”ground-breaking” studies or ”innovative” research projects. Dr Christiaan Vinkers – a psychiatrist at the Rudolf Magnus brain centre – was the main author of another ”very robust” report.
This tool used to analyse words, when selected for academic use, shows that indeed, “robust” is a favorite word of science:
Source: http://www.wordandphrase.info/academic/frequencyList.asp
And, this Ngram suggests that at least through 2008, the word “robust” has become vastly more popular in books. It’s almost like a hockey stick of robustness:
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
My first attempt at writing for a journal submission got me a session (in fact several sessions) with one of the professor-ed co-authors that lambasted my use of color commentary (IE “very good”, etc). I scrubbed and scrubbed for days and days till it met his standards for technical writing devoid of such “female” prose.
Yes I know it sounds sexist in today’s atmosphere, but I finally got the point. Alas, it seems we have birthed a generation of prose writers in science positions, or at the very least, behavioral scientists who have been hired to study climate and weather observations and data but have not been schooled in how to write about such observations and data, let alone how to statistically analyze such things.
My experience in writing for government publications is similar, in that we weren’t allowed to use words like “abnormal”, but terms like “statistically atypical” were allowed.
Which government was that ?
Duh, uk(us)-look at his name….government of Rome obviously. 😛
Right, Pamela. Warmunists think they just need to communicate better.
I was happy to see this story appear so quickly. It was covered on last night’s As It Happens (CBC Radio 1) and I was going to send a heads-up letter to WUWT if WUWT hadn’t covered the story already (and well).
Ian M
The problem is that people are using the word “robust” in a way that turns the original concept on its head.
Originally, “robust results” meant that they tracked reality very well even when the truth model was subjected to revision for various disturbing inputs.
Today, they mean they can find a way to revise various disturbing inputs to make the results track reality.
Since this can always be done, any result becomes robust, and the description tautological and meaningless.
Malleus Maleficarum 1486:
“However, it has been found that witches have freely confessed that they have done such things, and there are various instances of it, which could be mentioned, in addition to what has already been said. Therefore it is reasonable to conclude that, just as easily as they raise hailstorms, so can they cause lightning and storms at sea; and so no doubt at all remains on these points. ”
“And so no doubt at all remains on these points” would now be translated into, “these findings are unequivocal and robust. A consensus exists and the science is therefore settled”.
There is nothing new under the sun. Only old wine in new bottles.
That is so true. Every generation indulges itself in the conceit that it is uniquely enlightened compared to its benighted forbears. Or, maybe every other generation, and the ones in-between get to clean up the mess created by the hubris of that immediately preceding. I think that more or less explains why great wars and periods of turmoil tend to be separated by a generational turnover period of about 20-30 years.
People get the idea that they are immune to the fallacies that plagued earlier peoples because they know in hindsight where those people went wrong – kind of like saying you know how to win the game on Sunday because you know how you fumbled the ball last Sunday. And, they think they are scientific because they consume the technologies that came about from the study of science, e.g., because they have iPhones. They don’t know how it works, but it’s really cool, and they have one.
Witches cause climate change?????:)
All the witches flew their CO2 brooms to Paris this last month to complain about how flying brooms is causing the planet to heat up then they all flew home again and flew off onto vacations to very warm places where they played golf.
Golf is a devilish game!
Thank you for this post about the Malleus Maleficarum, aka the “Witch Hammer”, first published in 1486 and used by the Roman Catholic Church as a tool of the Inquisition, to torture and murder hundreds of thousands of innocents.
Nowadays, we have the modern equivalent of the Witch Hammer: the phrase “The science is settled”.
“The science is settled” is used by scoundrels and imbeciles to dismiss scientific reality – that we still do not know enough about climate science to even agree on what drives what (for example, warmists “KNOW” that atmospheric CO2 primarily drives global temperatures, but the data shows that atmospheric CO2 LAGS temperatures at all measured time scales – the warmists are in effect alleging that the future is primarily driving the past).
For clarity in this context, scoundrels are warmists who know that global warming alarmism is a fraud, and imbeciles believe it is real.
The list of academics dismissed from their posts for speaking out against the falsehoods of global warming alarmism is growing, and the number of people compromised by this new Witch Hammer number in the millions.
Global warming alarmist mania will run its course, but it will take years to do so, and society will continue to squander trillions of dollars in scarce global resources in this new false alarm against alleged catastrophic manmade global warming, in a cooling world.
Regards and Happy New Year, Allan
http://www.malleusmaleficarum.org/downloads/MalleusAcrobat.pdf
[excerpts]
THE
MALLEUS MALEFICARUM
of Heinrich Kramer and James Sprenger
Unabridged online republication of the 1928 edition. Introduction to the 1948 edition is also included.
Translation, notes, and two introductions by The Reverend Montague Summers. A Bull of Innocent VIII.
The Malleus Maleficarum (The Witch Hammer), first published in 1486, is arguably one of the most infamous books ever written, due primarily to its position and regard during the Middle Ages. It served as a guidebook for Inquisitors during the Inquisition, and was designed to aid them in the identification, prosecution, and dispatching of Witches.
…
At the time of the writing of The Malleus Maleficarum, there were many voices within the Christian community (scholars and theologians) who doubted the existence of witches and largely regarded such belief as mere superstition. The authors of the
Malleus addressed those voices in no uncertain terms, stating: “Whether the Belief that there are such Beings as Witches is so Essential a Part of the Catholic Faith that Obstinacy to maintain the Opposite Opinion manifestly savours of Heresy.” The immediate, and lasting, popularity of the Malleus essentially silenced those voices. It made very real the threat of one being branded a heretic, simply by virtue of one’s questioning of the existence of witches and, thus, the validity of the Inquisition.
…
It must be noted that during the Inquisition, few, if any, real, verifiable, witches were ever discovered or tried. Often the very accusation was enough to see one branded a witch, tried by the Inquisitors’ Court, and burned alive at the stake. Estimates of the death toll during the Inquisition worldwide range from 600,000 to as high as 9,000,000 (over its 250 year long course); either is a chilling number when one realizes that nearly all of the accused were women, and consisted primarily of outcasts and other suspicious persons. Old women. Midwives. Jews. Poets. Gypsies. Anyone who did not fit within the contemporary view of pious Christians were suspect, and easily branded “Witch”. Usually to devastating effect.
It must also be noted that the crime of Witchcraft was not the only crime of which one could be accused during the Inquisition. By questioning any part of Catholic belief, one could be branded a heretic. Scientists were branded heretics by virtue of repudiating certain tenets of Christian belief (most notably Galileo, whose theories on the nature of planets and gravitational fields was initially branded heretical). Writers who challenged the Church were arrested for heresy (sometimes formerly accepted writers whose works had become unpopular). Anyone who questioned the validity of any part of Catholic belief did so at their own risk. The Malleus Maleficarum played an important role in bringing such Canonical law into being, as often the charge of heresy carried along with it suspicions of witchcraft.
[end of excerpt]
I would think hyperbole would be sympotomatic of post-normal science, where “facts are uncertain, values in dispute, stakes high and decisions urgent”. Climate “science” is rife with it of course.
Is it certain this isn’t just a reflection of the number of pages available growing over time? I don’t see a “per page, per chapter, per book” or pages per year/decade, etc. Absolute numbers pulled from a time-variable pool are a poor indicator.
Scientists in a given discipline tend not only to read papers relating to that discipline, but also tend to follow the format, styles, and the verbiage of those papers. They are just like everybody else, in fact. So we shouldn’t be surprised if styles and even individual words ‘trend’ as has been described here. I don’t think it adds much to our understanding of what is going on in science.
But yes, modern papers do seem to be increasingly ‘promoted’ by the style of their titles and the content of their abstracts, and that is agreed to be regrettable.
There’s nothing new about this sort of thing. 20 years ago when I was writing gene therapy papers it was well known that you needed certain key words in your publications to increase likelihood of publication. These were actually ‘hot’, ‘sexy’, ‘cool’ words in biomedicine at the time. So things like: ‘angiogenesis’, ‘p53’, ‘tumour suppressor gene’ always helped make your paper more ‘topical’. Of course, you couldn’t bring them in if they weren’t relevant to your research, but if they were, you can bet your bottom dollar that those writing papers with the option of including them did so.
Thing was, this helped you get published in ‘high impact publications’ which were the equivalent of your research scientist ‘credit score’: the higher the impact factor, the ‘better’ your research was (the fact that that was somewhat contentious is neither here nor there). More to the point was the fact that powerful Professors were on the editorial boards of ‘high impact publications’ and papers from those labs had a much higher chance of being accepted for publication. Nepotisjm? Corruption? The way of the world?? You take your pick.
But if you want to create a ‘magic circle’ you do this: you create ‘high impact factor publications’ with the magic circle on the editorial board. Then you make ‘high impact factor publications’ key in tenure track position selection panels, you make them key for the awarding of major grants from govt funding bodies.
Once you’ve done that, you’ve created a great little cartel for 20 years.
That’s the way science works.
The very best scientists always get to the top. But for the league below them, the absolutely most important thing is to get politically connected.
Reminds me of the climate science racket.
They declare that only those recognized as a “climate scientist” has a right to express an opinion on climate science.
How do you become a “climate scientist” one might ask.
Those who are already recognized as “climate scientists” declare that you are one of course, and obviously, anyone who rocks the grant money gravy train will not be recognized as one of the club.
Headlines that will not get you ‘research’ funding , everything is ok, there are no changes , things could not be better , and finally the one you will never see , oddly especially in ‘settled science ‘ , no further research is required .
Add to that the fact that within climate ‘science’ it is clear that it is ‘impact’ of the paper not its ‘validity’ that makes the big difference in both personal and professional fronts. And you can see why your getting so much hyperbole.
In most organisations or political parties its ‘culture’ is as much has anything subject to the ‘culture ‘ of its leaders , now consider the ‘culture’ of people like Mann etc and you understand how an area already subject the above problems can become so rotten.
Seeming as the authors of this paper themselves would have had to quote the words they were studying the increase of, did this increase the frequency of those words in the papers in general? I’m guessing it did so by 0.3% with a margin of error of +/- 3% (tip of my hat to the Greenland ice post that I read before this one)
I’ll bet the papers of Newton, Einstein, Planck, et al had no hyperbole and were simply matter of fact in their presentation, possibly they were even humble. Someone here has probably read several of the classics and can enlighten us on this.
One thing’s for sure, most of the words under the graph in the article have absolutely no place in a dissertation that is classified as scientific. Re-read them and see what I mean. They look picked out of high school conversations.
Well, I did recently read Newton’s Opticks, which is much less “formal” and math oriented than Principia, but even so, I can’t recall any hyperbole at all. The focus was the experimentation, any excitement I sensed was about the joy of discovering things. He engages in speculation only when clearly labeled so, and then in bulk, essentially. Reasoning for or against this or that idea seemed moire a display of great depth and breadth of consideration, than advocacy of any sort, to me.
(And it was a great read, I felt)
Gregor Mendel flat out lied.
Hmm . . From the wiki –
“However, reproduction of the experiments has demonstrated the validity of the results.[45][not in citation given] In 2007, Daniel L. Hartl and Daniel J. Fairbanks suggested that Fisher incorrectly interpreted these experiments. They found it likely that Mendel scored more than 10 progeny, and that the results matched the expectation. They conclude, “Fisher’s allegation of deliberate falsification can finally be put to rest, because on closer analysis it has proved to be unsupported by convincing evidence.”[41][46] In 2008 Hartl and Fairbanks (with Allan Franklin and AWF Edwards) wrote a comprehensive book in which they concluded that there were no reasons to assert Mendel fabricated his results, nor that Fisher deliberately tried to diminish Mendel’s legacy.[47] Reassessment of the statistical analyses also disprove the notion of confirmation bias in Mendel’s results.[48][49]”
Not buying it, JK. I was taught 45 years ago that Mendel’s results were perfect. For example, he’d get 75 of one genotype, and 25 of the other. Which he obviously didn’t. We called it the “Mendelian Fudge Factor.” Trying to convince people of his theory would have been impossible if he reported an actual 77-23, for example. He’d have to teach people (unknown) statistics as well as genetics.
I’m not selling it, but I’m not buying what you’re selling either.
PS ~ To me, as statement like this is effectively hyperbolic;
“Gregor Mendel flat out lied.”
And this is not,
“I think Gregor Mendel flat out lied.”
To me, the basic problem this article addresses is far more widespread than just the realm of scientific papers. There is an epidemic of what might be called “error bar free” speaking, which involves “juicing” arguments/propositions through habitual “absolutist” language use, as I “hear” the world.
Study:
hyperbolehyperbowl is increasing in scienceI’ve disliked the word “robust” ever since it entered the computer science lexicon as the follow-on to “structured programming” around 1970. In this case it means software that handles error conditions gracefully and can keep running through “novel” situations. I’m not certain why I accepted structured programming, possibly because it was around when I started college, and “robust” just kind of showed up as an unbidden bandwagon and lots of people lept aboard. Perhaps my problem was that they must not had been writing robust code before being shown The Way.
Really Over Blown Underhanded Science Tactic= ROBUST.
Ric Werme wrote “I’ve disliked the word “robust” ever since it entered the computer science lexicon as the follow-on to “structured programming” around 1970…Perhaps my problem was that they must not had been writing robust code before being shown The Way.”
I still write structured programming. The alternative is unstructured — spaghetti code. It is remarkably difficult to structure FORTRAN, a bit easier to structure COBOL. Either of them with “goto” can make a program amazingly difficult to follow, debug, maintain.
Even modern languages can still usually be “structured”. Start with a design that itself incorporates the problem to be solved and the method to solve it. All these new methods with “scrums” and so on practically guarantee bugs, and they know and accept it.
The “C” language incorporates the idea nicely; “top down programming” — start with a do nothing shell or container that theoretically does *everything*: int main (){}; There’s your entire program right there. Then you start writing top level functions to do the Big Things. Then you take each big thing and write functions to do the pieces of the Big Thing.
There’s not a lot of code re-use except at the bottom and if there’s a significant design change it can produce a huge ripple effect; but regardless of those problems a well-structured program is a beauty and can be maintained ten years later. It probably has a range of optimum size.
I’ve always used robust to describe a program that when presented with a unique error condition, is able to shut down gracefully and leave you enough of a log so that you can figure out what went wrong.
I did Fortran and Cobol for 30 years. I told people I was a “systems archaeologist.” I spent my time trying to figure out what people were thinking when they wrote the code.
“systems archaeologist.”
I love it!
“I spent my time trying to figure out what people were thinking when they wrote the code.”
I still do that especially with programs I am obliged to maintain written in PHP and ASP.
C allows untyped variables and type casting which is a bit ugly, and global vars. C++, Ada and Pascal do away with these to varying degrees – Ada better than most. Most climatologists probably limit them selves to Excel Macro Language. REXX and Perl are the original bungie languages. You can wrap anything in it. Procedural vs object oriented is probably as hotly contested as the two views of climate reality. Object Perl comes close to lipstick on a pig.
“C allows untyped variables”
Nonsense. Every variable has a well defined type in C.
“type casting which is a bit ugly,”
Type casts are just explicit type conversion, and essential tool.
“C++, Ada and Pascal do away with these to varying degrees”
Explicit type conversions exist in all these languages, and are very important.
C++ has not one but five ways to perform explicit type conversions: C-style, C++ style (syntactically similar to object construction, but semantically different), static_cast, const_cast, reinterpret_cast.
Only ML type languages cannot have such type conversions, because of its type system.
“Procedural vs object oriented is probably as hotly contested as the two views of climate reality”
The OOP cult and associated nonsense is as progressive and Californian as climate “science”.
OOP is as well defined as “the climate”, some people say there are as much definitions of OOP as there are OOP programmers.
Try using the word “novel” in a submission to Science or their related journals. You will get upbraided by their very good editors. Don’t try any other loaded adjectives either.
“for the first time” still gets through.
How about this for NASA hype
http://www.nasa.gov/feature/a-still-growing-el-nino-set-to-bear-down-on-us
The current strong El Niño brewing in the Pacific Ocean shows no signs of waning, as seen in the latest satellite image from the U.S./European Ocean Surface Topography Mission (OSTM)/Jason-2 mission.
El Niño 2015 has already created weather chaos around the world. Over the next few months, forecasters expect the United States to feel its impacts as well.
The latest Jason-2 image bears a striking resemblance to one from December 1997, by Jason-2’s predecessor, the NASA/Centre National d’Etudes Spatiales (CNES) Topex/Poseidon mission, during the last large El Niño event. Both reflect the classic pattern of a fully developed El Niño. The images can be viewed at:…
“he new Jason-2 image shows that the amount of extra-warm surface water from the current El Niño (depicted in red and white shades) has continuously increased, especially in the eastern Pacific within 10 degrees latitude north and south of the equator. In the western Pacific, the area of low sea level (blue and purple) has decreased somewhat from late October. The white and red areas indicate unusual patterns of heat storage. In the white areas, the sea surface is between 6 and 10 inches (15 to 25 centimeters) above normal, while in the red areas, it is about 4 inches (10 centimeters) above normal. The green areas indicate normal conditions. The height of the ocean water relates, in part, to its temperature, and is an indicator of the amount of heat stored in the ocean below.”
It’s unprecedented how many times the media now uses the word unprecedented.
That’s easy to do when you don’t understand the topic and disallow your eyes to land on opposing views.
Word inversion. Common words now have the opposite meaning to the traditional one. So robust now equals flaky, liberal now means illiberal or intolerant etc.
So, does a novel paper like this count in their survey?
Reworded: Study-hyperbole-is-at-greatest-level-eva!
OK, so science uses more hyperbole than other fields.
I’d be interested to hear which science specialty uses the most. My money would be on Climate “Science.”
“In that case, words used to describe scientific results are no longer driven by the content but by marketability.”
That has been apparent for some time, noted in the papers on marketability of the climate change narrative. Many papers are more interested in how to sell the idea—especially those written by psychologists.
It seems to me the problem is not merely that the use is rising, but that these words are used at all. Most of them–disappointing, amazing, unacceptable, etc.– are completely inappropriate in scientific writing.
When I read anything from a scientist that amazed, surprised, or disappointed them, my first thought is….so, you got a result you didn’t anticipate…that tells me that you don’t fully understand the subject of your analysis.
“The number of papers containing any of the positive words in their title or abstract rose from an average of 2% in 1974–80 to 17.5% in 2014. Use of the 25 negative words rose from 1.3% to 2.4% over the same period, according to the study.”
Whoever wrote this should not be trusted as a science writer, and should be given a scholarship to a Numeracy Seminar before they harm my scientific sensibilities any more.
Rahmstorf and the robustness of sea level rise predictions…
https://climatesanity.wordpress.com/2012/09/29/rahmstorf-2011-robust-or-just-busted-part-5-why-a-paper-about-robustness/
I wonder how often the phrase “One Study Wonder” shows up?