Study: hyperbole is increasing in science

From the “everything is robust” department.

We’ve long noted at WUWT that the word “robust” has seen a significant rise in usage in climate science papers, becoming a favorite word to use when statistical Spackle has been applied to climate data. Now there’s evidence from a new study suggesting that observation is spot-on.

From Nature:

‘Novel, amazing, innovative’: positive words on the rise in science papers

Philip Ball

Scientists have become more upbeat in describing their research, an analysis of papers in the PubMed database suggests.

Researchers at the University Medical Center Utrecht in the Netherlands say that the frequency of positive-sounding words such as ‘novel’, ‘amazing’, ‘innovative’ and ‘unprecedented’ has increased almost nine-fold in the titles and abstracts of papers published between 1974 and 2014. There has also been a smaller — yet still statistically significant — rise in the frequency of negative words, such as ‘disappointing’ and ‘pessimistic’.

Source: Ref. 1

Expand

Psychiatrist Christiaan Vinkers and his colleagues searched papers on PubMed for 25 ‘positive’ words and 25 ‘negative’ words (which the authors selected by manually analysing papers and consulting thesaurus listings). The number of papers containing any of the positive words in their title or abstract rose from an average of 2% in 1974–80 to 17.5% in 2014. Use of the 25 negative words rose from 1.3% to 2.4% over the same period, according to the study, published in the British Medical Journal on 14 December¹.

Rising hype

The most obvious interpretation of the results is that they reflect an increase in hype and exaggeration, rather than a real improvement in the incidence or quality of discoveries, says Vinkers. The findings “fit our own observations that in order to get published, you need to emphasize what is special and unique about your study,” he says. Researchers may be tempted to make their findings stand out from thousands of others — a tendency that might also explain the more modest rise in usage of negative words.

The word ‘novel’ now appears in more than 7% of PubMed paper titles and abstracts, and the researchers jokingly extrapolate that, on the basis of its past rise, it is set to appear in every paper by the year 2123.

…

But Vinkers and his colleagues think that the trend highlights a problem. “If everything is ‘robust’ and ‘novel’”, says Vinkers, then there is no distinction between the qualities of findings. “In that case, words used to describe scientific results are no longer driven by the content but by marketability.”

A BBC story here says the use of the word “robust” has gone up 15000% They write:

Despite working with facts, figures and empirical evidence, the world of science appears to have a growing addiction to hyperbole. Researchers at the University of Utrecht in the Netherlands looked at four decades worth of medical and scientific publications, and found a significant upwards trend of positive words. We’ve all heard of those ”ground-breaking” studies or ”innovative” research projects. Dr Christiaan Vinkers – a psychiatrist at the Rudolf Magnus brain centre – was the main author of another ”very robust” report.

This tool used to analyse words, when selected for academic use, shows that indeed, “robust” is a favorite word of science:

Source: http://www.wordandphrase.info/academic/frequencyList.asp

And, this Ngram suggests that at least through 2008, the word “robust” has become vastly more popular in books. It’s almost like a hockey stick of robustness:

Source: https://books.google.com/ngrams/graph?content=robust&case_insensitive=on&year_start=1800&year_end=2014&corpus=15&smoothing=3&share=&direct_url=t4%3B%2Crobust%3B%2Cc0%3B%2Cs0%3B%3Brobust%3B%2Cc0%3B%3BRobust%3B%2Cc0