Each year, biomedical scientists pump out about 1 million new papers, but a troubling truth hides in plain sight: much of this work can’t be replicated. Far from a small glitch, this is a colossal crisis—squandering billions, eroding faith in science, and stalling genuine breakthroughs. In an interview with Chemical & Engineering News, pharmacologist Csaba Szabo, a professor at the University of Fribourg, Switzerland, confronts this chaos head-on, previewing his recently published book, Unreliable. His verdict? The scientific system is fractured beyond repair, and Band-Aid fixes won’t cut it. Nothing short of a revolution will do.
Szabo’s journey into this quagmire began casually—over beers with colleagues in New York during a sabbatical. The question that kept surfacing was simple yet haunting: “Why is it that nobody can reproduce anybody else’s findings?” It’s a problem scientists have grumbled about for years, but Szabo decided to do something about it. The result is Unreliable, a deep dive into the causes of irreproducibility—ranging from hypercompetition and sloppy errors to statistical trickery and outright fraud. His conclusions are as sobering as they are provocative.
The Scale of the Problem
Szabo’s findings are jaw-dropping. After sifting through the global scientific literature—not just the polished papers on PubMed, but everything published anywhere—he estimates that 90% of it fails the reproducibility test. Even worse, he believes 20-30% is entirely fabricated. “I didn’t expect the numbers to be that high going in,” he admits. “It’s just absurd.” The financial toll is staggering. Paper mills—fraudulent outfits that churn out fake research for profit—rake in billions annually. This isn’t a cottage industry; it’s a industrial-scale scam.
What’s more shocking is who’s left to clean up the mess. It’s not the grant agencies, universities, journals, or governments stepping up. Instead, it’s a ragtag crew of private investigators—working unpaid, late into the night, while dodging lawsuits from the very fraudsters they’re exposing. “What kind of a system is this?” Szabo asks. It’s a fair question.
No Quick Fixes
Szabo doesn’t mince words: the half-measures tried so far—workshops, checklists, and the like—have flopped. “The things that we’ve tried are not working,” he says. “I’m trying to suggest something different.” His fix? A top-to-bottom overhaul of how science is taught, funded, and published. It’s an ecosystem, he argues, and you can’t fix one part without tackling the whole.
On education, Szabo proposes splitting scientific training into two tracks: one for discovery and another for integrity. The latter would train a new breed of professionals in experimental design, statistical rigor, data management, and independent review—essentially, watchdogs embedded in the system. It’s a bold idea, but it could professionalize the policing of science.
For funding, he takes aim at the current grant lottery, where scientists spend endless hours writing proposals, only for the same big institutions to pocket the cash year after year. His radical suggestion: give institutions lump sums tied to strict reproducibility and integrity benchmarks. Let them figure out how to spend it. The catch? It demands visionary leadership—something in short supply. Still, it could free researchers from grant-writing drudgery and focus them on actual science.
On the publication front, Szabo wants top journals to demand replication supplements—independent labs verifying findings before papers go to print. It’s a practical step that could boost confidence in high-profile claims. But he doesn’t stop there. He also floats shrinking the scientific workforce and installing cameras and keystroke monitors in labs. “There are cameras in cockpits and behind the barista at Starbucks,” he notes. With so much money—and ultimately human lives—at stake, why should labs be exempt?
Controversy and Hope
These ideas won’t go down easy. Shrinking the workforce and adding surveillance will rile plenty of researchers. Szabo knows that. “I’m not saying that my ideas are brilliant,” he says. “I’d love to see more out-of-the-box ideas.” He’s throwing darts at the board, hoping to spark a broader debate.
One glimmer of hope comes from an unexpected corner: Jay Bhattacharya, Stanford professor and President Trump’s pick to lead the National Institutes of Health (NIH). Bhattacharya has publicly backed more funding for replication studies—a move Szabo cheers. “Amen to that tweet,” he says.
If Bhattacharya takes the helm, he could wield the NIH’s clout to push reform. But Szabo remains cautious. Recent NIH moves—like halting meetings and purchases—feel more punitive than progressive. Real change, he insists, must come from the top.
I don’t know Jay, but if he gets the job, I don’t think he wants to go down in history as the person who destroyed American biomedical science. He wants to go down in history as the person who reformed American biomedical science. Maybe I am still naive and hope for too much from our politicians or governments, but this reform has to come from the top, from whoever holds and controls the money. The American system, and the NIH in particular, has a lot of influence. They have more power than even they perhaps realize, if they really wanted to do something.
https://cen.acs.org/policy/publishing/nobody-reproduce-anybody-elses-findings/103/web/2025/02?sc=250305_news_eng_cennews_cen_Member
Dark Humor, Stark Truth
Szabo punctuates Unreliable with biting cartoons that cut deeper than his prose. His favorite? A recruitment committee choosing between a plagiarist, a cheat, and a harasser—only to pick the one with the most grant money.
I think one of the most offensive ones is the scientists at the “publication workshop” [four scientists in a restaurant, ordering up a paper on herbal nanoparticles in cancer cells for publication in a journal with good impact factor]. Another one is of a recruitment committee looking at three candidates for a job. One is a plagiarist, one is a cheat, and the third is a sexual harasser, and they decide to go with the one with the most grant money. But it’s not a joke—I have seen this kind of thing.
https://cen.acs.org/policy/publishing/nobody-reproduce-anybody-elses-findings/103/web/2025/02?sc=250305_news_eng_cennews_cen_Member
In the end, Szabo’s message is clear: science is drowning in waste and fakery, and it’s time to stop pretending otherwise. His solutions may not be perfect, but they’re a starting point. If we want science we can trust, we’ll need to rethink everything—from the classroom to the lab bench to the journal page. The clock’s ticking.
Unreliable by Csaba Szabo hit the shelves this month. It’s a book no one wanted to write—but one we all need to read.
H/T Michael E M (no, not Mann)
________________________________________________________________________
“Statistical Trickery” fits Sir John Houghton’s Global Warming Potential numbers perfectly.
Most often there is a least one variable that is ignored or at least uncontrolled, sometimes intentionally.
In the case of Sir John’s GWP numbers, it’s the final answer that is ignored.
Methane is 86 times more powerful at trapping heat,
and it will run-up global temperatures by so many
degrees by such and such date.
Climate science never says, and neither policy makers or the media ever ask.
Best guess is around a 20th of a degree by 2100.
Any time you care to check retraction watch, the vast majority are medical science – one way or another.
It’s the one field where replication is somewhat routine. Pharmaceutical companies troll the literature looking for research findings that can be developed into drugs. In his book Rigor Mortis, Richard Harris points out that Bayer and Amgen couldn’t replicate as many as 90% of the research findings they investigated. In many cases, the scientists involved couldn’t even duplicate their own experiments,
In medicine, the corrupting influence is the possibility of big money if one’s research findings lead to a profitable drug. In other fields like climate, or transgender studies, the corrupting influence is ideology.
Defund the universities.
“Each year, biomedical scientists pump out about 1 million new papers, but a troubling truth hides in plain sight: much of this work can’t be replicated.”
It’s certainly not just that field with a replication crisis.
Contemplating the garbage that’s produced promoting global warming is not for the faint of heart.
This has been known for a long time. This paper was published 20 years ago..
https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
Ancel Keys is the posterchild for scientific fraud.
Rockefeller, one of the original robber barons wrecked all of medicine, first in the USA, then in the rest of the western world.
Even the USA food pyramid is literally killing people who follow that nonsens. The producers of the food pyramid knew that but dollars speak louder than lives.
Even good old Max Planck was saddened by the state of science: “Science progresses one funeral at a time”.
I’m afraid the birthrate of idiots exceeds their deathrate, however that is possible given they are so confused about gender.
The climate fraud is a mirror image of the Ancel Keys fat fraud.
I recently read that a number of Tufts faculty were suing the school. The school had reduce their salaries because they failed bring in research grants equal to half their annual salary.
Don’t doubt it since universities are dinosaurs, many got plenty but want more. I worry that cuts will not be very sensible like libraries and save the administrators. Not just medicine. Todd, P. A., et al.,. 2010. One in four citations in marine biology journals is inappropriate. Mar. Ecol. Prog. Ser. 408:299-303. https://doi.org/10.3354/meps08597
Suspect that they are just being kind, appropriate or not is new ad hoc buzz word.
Seems a vicious cycle…
All Journalists should read this so they stop saying “scientists say” or “experts say”
In the old days, Canada’s NSERC funded individuals on the basis of short proposals, enough to run a small lab and one student with a bit of help from other small sources. If the idea did not pan out, one was free to switch horses in midstream. You were, of course, punished by the university for not being BIG. However, what I produced on a shoe string is still relevant in that 14 years after retirement is still running 1-2 citations a week.
Then the government got involved. It wanted “formal collaborations”, “projects with commercial potential”, and big projects that had to span more than one university. Projects grew with these into ways to split a big grant around.
Funnily, we always had collaborations that grew organically linking us with other labs with other technologies and their own funding to put into a project.
Much of the issue with reproducibility is a lack of training in measurement uncertainty. Most reproducibility studies are measuring something different. E.g. if a drug reproducibility study does not use the EXACT same strain of mice right down to the DNA, the study is measuring a different thing. What should be done is enough reproducibility studies on different things to produce a proper distribution. The drug effects then become the average with a measurement uncertainty determined by the standard deviation. Too many scientists today take multiple samples of different things and then use the SEM as the uncertainty instead of the standard deviation of the samples. The SEM only tells you how accurately you have calculated the average value, it doesn’t tell you the measurement uncertainty associated with that average.
Perhaps withhold 50% of the grant until the paper has been independently reproduced successfully?
“After sifting through the global scientific literature—not just the polished papers on PubMed, but everything published anywhere—he estimates that 90% of it fails the reproducibility test. Even worse, he believes 20-30% is entirely fabricated.”
Doesn’t sound very scientific. And reproducible?
Have you read the book or are you relying on the post being accurate?
“are you relying on the post being accurate?”
Where have we come to if we can’t rely on WUWT posts being accurate?
Has anyone here read the book?
I haven’t because it is a book simply discussing a problem and one persons take on it. Your website is somewhat similar only you try to claim to be more “scientific” but yet you seem to fall into every error possible.
Yeah, not nearly as sciency as the peer reviewed papers claiming “97% consensus of ALL climate scientists” that agw is real, human caused, and DANGEROUS.
Can’t get any more ‘scientific’ than that pearl of replicability hey Nick?
(unless of course, you select a few of the participating “consensus” “scientists” to publish confirming studies)
This, in the world of ordinary folks, is usually referred to as a “circle-jerk”.
He didn’t claim it to be scientific unlike climate science when they do this sort of junk.
Nick parodies himself….
When science gets its knickers in a twist over gender, isn’t that a sign that all intelligence has been lost if not already then by the many in high places? We need a proper purge across the board especially in political circles and the media. An echo chamber has always been cheap, nasty.and very unintelligent.
Good post about an interesting book. I will read it. One root cause not mentioned is ‘publish or perish’ in the academic race for tenure. Apparently quantity not quality is what counts.
Stanford’s Ioannidis has been onto this for a long time, and Stanford Medical School has a number of formal programs working on reproducibility there. Worth reviewing them.
The problem is not limited to biomedical research. And some of it is due to outright fraud per Retraction Watch.
Not impressed. His solutions are top-down central planning.
The endless taxpayer spigot is the problem. My fixes:
“Get rid of all government control and funding of education. No public schools or colleges, no student loans, no research grants, nothing.”
What a terrible solution. How many children of financially poor parents would lack education as a result?
Just about the only education worth having starts with how to use pliers, crescent wrench, and screwdriver. (beep) fuzzy studies.
You missed one.
End all ‘gain-of-function’ research.
Our citizens are sick enough already.
Yep, brilliant proposal. The US has a huge lack of uneducated people, this will fix that in a hurry.
And we really need to reduce the spending in all those pesky areas that are not of interest to the wealthy. I mean, everyone knows that it is best to stay young, healthy and rich. And then just educate people to do that, which will fix most problems we have.
I guess you took your own advice on this one, given the list you made?
Peer review science- 60% of the time, it works every time.
In science today, your income is based on your outcome. That’s why everything is a “breakthrough”. and why so many areas in science are fundmentally broken and non-reproducable.
This was highlighted by Ben Goldacre in his book Bad Science first published in 2008 but nothing at all has changed.
I rather lost faith in Goldacre when he went all-in on the CAGW fraud.
Agree but his book still stands.
I know nothing about scientific studies or publishing. What I do know is lying is unacceptable. There have been numerous articles here at WUWT that have thoroughly dismantled published scientific reports.
I don’t see a one size fits all for the simple reason that not all research is the same. The easiest solution is if the research can’t be reproduced the grant money must be returned. Journals that publish research that can easily be dismantled should also be held accountable. Outfits providing the money should be held accountable.
I can see where this wouldn’t be appropriate for some research. The only thing I can think of is things like General Relativity. Einstein was researching things few even considered his research was more like opening the discussion for something others should consider and see if it makes sense. I don’t see how we could demand reproducibility in a case like that. We would expect others to take a look at it and find weaknesses or strengths in what his research claims.
All other research must be reproducible, research like climate change or the efficacy of medicine or medical procedures and on and on. If no one can reproduce your research give us our money back. If you publish research that can’t be reproduced you will be punished. I don’t know how publishers are punished but some clearly need it. If you give grant money to people whose research can’t be reproduced your grant making authority and funds will be taken away from you. Remember that money is not yours it is ours, the American taxpayer.
“All other research must be reproducible, research like climate change or the efficacy of medicine or medical procedures and on and on. If no one can reproduce your research give us our money back.”
I posted this in another message. Not all experiments are perfectly reproducible. The experiments need to be done multiple times to get a proper measurement uncertainty interval for application to different test subjects. Most re[lated experiments actually have different measurands, like different DNA strains of mice as the subjects in repeated experiments.
What needs to be done is to start training scientists in proper measurement protocols, including measurement uncertainty for when the same measurand is measured multiple times and for when multiple measurands are measured a single time.
“while dodging lawsuits from the very fraudsters they’re exposing”
Why did I immediately think of Mickey Mann, I wonder?
To add to this discussion, consider the large volume of “alternative medicine” or “herbalist” entities that you probably have in the US and UK, similar to here in Australia.
These organizations peddle cures that the medical researchers have largely rejected as either ineffectual or worse than alternatives. Some have succeeded in taking in enough dollars to buy and own large pharmacy chains.
Now, imagine a more ideal world here in Australia, where the marginal alt people were ordered to put their effort into manufacturing proven medications for export, not packaged grass and herbs for ignorant locals. The people employed in the alt factories could, with a minor change to products, become part of a small industrial powerhouse adding wealth to the nation.
I have no idea, except the usual greed, why these people exist doing next to no good, when they could do a lot of good.
Geoff S
Renowned analyste Stephen McIntyre opined that the health field was worse than the climate field for low quality of research.
My daughter has worked for Elsevier for many years and she’s adamant that “peer review” is the gold standard. We had a bit of a go-around over the subject — and of course I’m the old, ignorant, fuddy-duddy father, who doesn’t know what he’s talking about.
He’s right, a complete overhaul is the ONLY way
There are those that try to fight the system from within, papering over cracks, sticking plasters on amputated limbs
Some of them have noble intentions, they really care about science
Some are parasites, they spot an opportunity to gain financially or otherwise – sites like retractionwatch I put in this category
No matter what, we’ve had a billion different initiatives and none worked
Quelle surprise. Not really. It’s fundamental rot, and until you fix this nothing will change
Governments have to realise around world that science needs some degree of honesty in the way it is taught and conducted in order to provide the best results – with minimal wastage – and to discourage the sort of mediocre people managers who have captured and currently control it
True scholars find it difficult to survive, because most are not naturally conformist. Modern academia requires swearing an oath to a million different conformist ideologies, and bullsh1tting your way to the top, pedominantly through convincing funding agencies that your daft ideas are worth funding, and that a repacking of the same idea should be funded at least once more every subsequent year
This won’t be easy, because it’s tied up with massification, “democratisation” and commercialisation of tertiary education
Universities serve more of a function as degree factories than research as far as governments are concerned
Universities, politicians, parents, school leaders and others need to be weaned off the idea that university is some rite of passage