Peer-Reviewing Peer Review

Guest Post by Willis Eschenbach (@weschenbach on X, blog at Skating Under The Ice)

We’re told, endlessly, that science is a self-correcting machine. A pristine engine of truth where bad ideas are discarded, and good ones rise to the top like cream. We are told to “Trust The Science™“ because it has passed the magical, mystical trial known as Peer Review.

Bad news. The machine is broken, the cream is curdled milk, and the gatekeepers are asleep at the switch—or worse, they’re selling tickets to the vandals.

A new study out of Northwestern University, entitled Organized scientific fraud is growing at an alarming rate, study uncovers, has just pulled back the curtain on what many have said for years. It turns out that “organized scientific fraud” isn’t just a few rogue grad students fudging a data point. No. It is a global, industrial-scale operation.

According to the study, we are now dealing with “sophisticated global networks” that function essentially as criminal organizations. They aren’t just faking results; they are manufacturing entire fake scientific careers. They sell authorship slots on bogus papers like they’re selling condos in Florida. 

You want to be a “First Author” on a groundbreaking physics paper? That’ll be $5,000. You want to be a co-author? We have a discount on aisle three.

The study notes that this fraud is “outpacing the growth rate of legitimate scientific publications.” Think about that. The cancer is growing faster than the host.

YIKES!

And the peer-review system, that vaunted shield that is supposed to protect us from error? It’s acting less like a shield and more like a sieve.

But wait. Before we blame this all on shadowy “criminal networks” and nameless paper mills overseas, let’s look a little closer to home. Because the rot isn’t just coming from outside the house. It’s coming from the basement.

I’ve seen this movie before. I’ve lived it.

Years ago, I wrote about my peer-review experiences with Dr. Michael Mann, author of the infamous “Hockey Stick.” I called him a “Smooth Operator“, and I meant it. In the climate world, “peer review” has too often morphed into “pal review.” It’s a cozy club where friends rubber-stamp friends’ papers and, more importantly, block the publication of a study from anyone like me who dares to question the “Consensus.” 

And as I detailed in “Freedom of Information, My Okole“, I’ve spent years asking for the data and code behind these taxpayer-funded studies. And what do I get? Stonewalling. Refusals. As Phil Jones told Warwick Hughes, “Why should I show you my data when you only want to find something wrong with it?”

That is not science. That is a priesthood protecting its dogma.

The current peer-review system is a black box. An editor sends a paper to two or three anonymous reviewers. If those reviewers are the author’s pals, the paper gets a pass. If the author is an outsider, or a skeptic, the reviewers can kill the paper in secret, with no accountability, for reasons that have nothing to do with the science and everything to do with protecting their turf.

Of course, only an extremely rare, perfectly honest reviewer is going to allow the publication of any study that demolishes the very foundations of the work that he’s spent his life building and expounding. As Upton Sinclair famously explained, “It is difficult to get a man to understand something, when his salary depends upon his not understanding it”. I call that the “Sinclair Trap”, and it’s far too easy to fall into.

Unfortunately, the Sinclair Trap is even worse for scientists because it’s not just money. I’ve said before that “Science is a blood sport”. What I meant was that any new scientific discovery or understanding has the possibility of being very costly, not just to the salary, but to the prized professional reputation of the holders of the previous view. 

It doesn’t have to be costly if the scientist whose prior work is discredited is honest, open about it, and willing to move forward and embrace and further the new understanding.

But that’s not every scientist. 

And now, we see the result. A system so opaque and unaccountable that it can be gamed by criminal syndicates on one end and ideological gatekeepers on the other.

So, what do we do? Do we just throw up our hands and say “Science is hard”?

No. Just no.

We need a total overhaul. A complete tear-down of the secrecy that allows this problem to thrive in the dark.

I’ve proposed a solution before, and I’ll propose it again. I call it Peer Review Plus. 

Here’s how it works. It’s simple, it’s cheap, and it would solve 90% of these problems overnight.

First, you keep the traditional peer review. But here is the kicker: You publish everything.

When a paper is published, you don’t just publish the paper. You publish the entire correspondence between the authors and the reviewers. You publish the reviewers’ and editors’ names. You publish their objections, and the authors’ rebuttals.

Let the world see how the sausage was made. If a reviewer gave a paper a pass because they’re pals, it will be obvious. If a reviewer blocked a paper because they didn’t like the conclusion, that will be obvious too.

But I’d go further.

We should also publish any valuable rejected papers.

Science proceeds by falsification. When a paper is rejected, it’s usually because a reviewer claims to have found a flaw. That falsification, valid or not, is a valuable piece of scientific data. But right now, it’s thrown in the trash bin of history.

If a reviewer rejects my paper, I want that rejection—and my answer to it—on the public record. Let the community decide if the rejection was valid or if it was just gatekeeping.

We also need an “Open Review” system where publishing the data and code are mandatory. As my friend Mosh used to say “No code, no data, no science”. This should result in “No publication”. Period.

And then, once it’s published, open the floor. Allow moderated, non-anonymous comments on the web from the larger scientific community and the public. The truth is that this public peer review by the larger world is the only truly valuable review.

The Northwestern study says we need to “police ourselves better.” They’re half right. We don’t need more police. We need more lights and more transparency. Sunlight is the best disinfectant. Right now, the scientific establishment is operating in a windowless room, and the mold is taking over.

We need to stop treating “Peer Reviewed” as a stamp of divine truth and start treating it for what it is: a fallible, human, and currently deeply corruptible process.

Until we pry the doors open and let the public see who is paying whom, who is reviewing whom, what the objections to the study were, and where the data actually is, this fraud will continue. The paper mills will keep churning, the pals will keep reviewing, and the public trust in science will continue its long, slow slide into the gutter.

Both the public and the scientists deserve better.

w.

PS—After writing the above, I thought I’d use perplexity to educate me on the history of peer-review. As always, there’s more for me to learn. This is a good question for AI, since the history is not a highly debated subject. I asked:

What is the history of the peer-review process in scientific publication? Who pushed for it? Who objected to it?

I included my standard instructions about not guessing, using primary sources, and the like. Here’s the response:

===

Peer review in scientific publishing developed gradually from informal editorial consultation in the seventeenth century to standardized external refereeing after the Second World War, with strong institutional support from major learned societies and leading journals, and persistent criticism from scientists and methodologists who argued that it is conservative, biased, and often unreliable. No single individual “invented” modern peer review, and surviving historical documentation does not support attributing it to one specific person; rather, it emerged through incremental practices at journals such as Philosophical Transactions of the Royal Society and later formal policies at journals such as Nature. [1] [2] [3] [4] [5] [6] [7]

 Early editorial review (17th–18th centuries)

• In 1665, Henry Oldenburg began editing Philosophical Transactions of the Royal Society and sought advice from knowledgeable colleagues to decide which submissions to print, which historians regard as a precursor to formal refereeing, although it was not systematic external peer review in the modern sense. [2] [8] [7]

• In the eighteenth century, the Royal Society created a Committee of Papers that could accept, postpone, reject, or refer manuscripts to designated members for expert evaluation, embedding internal scientific judgment into the journal’s constitution but still without standardized, anonymous external review. [3] [8] [7]

 Nineteenth‑century institutionalization

• During the nineteenth century, Philosophical Transactions and other society journals increasingly used written reports from selected referees, often Fellows of the society, to guide publication decisions, making the process more formal and collective than Oldenburg’s largely personal editorial control. [7] [3]

• Historians of science emphasize that practices varied widely among journals and disciplines, and that even where referee reports existed, they were not yet a universal or uniform requirement for all scientific papers. [4] [2]

 Twentieth‑century “modern” peer review

• After the Second World War, rapid growth in the number of manuscripts, expansion of specialized fields, and rising journal prestige led many journals to adopt routine external refereeing by domain experts, which is often identified as the birth of “modern” peer review. [9] [1] [4]

• By the late 1960s and early 1970s, leading journals such as Nature had formal peer review systems, and the term “peer review” itself came into widespread use, supported by citation-tracking infrastructures that rewarded publication in peer‑reviewed venues. [10] [4] [9]

 Who pushed for peer review?

• Documentary evidence shows that officers and councils of learned societies, particularly the Royal Society of London and similar bodies in Europe and North America, pushed for more systematic expert evaluation to protect scientific reputation, manage limited printing budgets, and filter the growing volume of submissions. [2] [3] [7]

• Editors and publishers of major journals in medicine and the natural sciences in the mid‑twentieth century promoted external peer review as a quality‑control and gatekeeping mechanism, and later governmental and funding agencies reinforced its use by tying research assessment and career advancement to publication in peer‑reviewed outlets. [6] [1] [4]

 Who objected to peer review?

• Historical and contemporary analyses document objections from prominent scientists, including Albert Einstein, who was notably resistant when one of his papers was subjected to anonymous external review, as well as from later critics who argue that peer review can suppress innovative or unconventional work. [5] [11]

• Methodological critiques in the biomedical and social‑science literature have highlighted problems such as low inter‑reviewer agreement, bias, conservative preference for established paradigms, and limited ability to detect errors or fraud, and some scholars have called for major reforms or even abolition of traditional pre‑publication peer review in favor of alternative models. [12] [13] [1] [6]

Limitations:  The surviving primary records (editorial minutes, correspondence, referee reports) are incomplete and uneven across journals, so attributions of “first use” and lists of specific individuals who pushed or objected are necessarily partial, and no primary source provides a complete, definitive chronology for all disciplines. [14] [7] [2]

 [1](https://pmc.ncbi.nlm.nih.gov/articles/PMC11215012/)

 [2](https://ethos.lps.library.cmu.edu/article/id/19/)

 [3](https://arts.st-andrews.ac.uk/philosophicaltransactions/brief-history-of-phil-trans/phil-trans-in-19th-century/)

 [4](https://blog.f1000.com/2020/01/31/a-brief-history-of-peer-review/)

 [5](https://theconversation.com/hate-the-peer-review-process-einstein-did-too-27405)

 [6](https://pmc.ncbi.nlm.nih.gov/articles/PMC4975196/)

 [7](https://pmc.ncbi.nlm.nih.gov/articles/PMC4360128/)

 [8](https://arts.st-andrews.ac.uk/philosophicaltransactions/brief-history-of-phil-trans/phil-trans-in-18th-century/)

 [9](https://mitcommlab.mit.edu/broad/commkit/peer-review-a-historical-perspective/)

 [10](https://www.youtube.com/watch?v=RGuaua10oRE)

 [11](https://www.experimental-history.com/p/the-rise-and-fall-of-peer-review)

 [12](https://pubmed.ncbi.nlm.nih.gov/12127284/)

 [13](https://mediatheoryjournal.org/2018/09/03/mieke-bal-lets-abolish-the-peer-review-system/)

 [14](https://www.nature.com/articles/d41586-024-03287-4)