Guest Post by Willis Eschenbach
A Modest Proposal For Improving Peer Review
Abstract.
A proposal is made for the design of a specific type of post-publication peer review.
Background
In 2006, the Journal of the Royal Society of Medicine published a widely cited paper (960 citations) by Richard Smith entitled “Peer review: a flawed process at the heart of science and journals”. In it he noted the following problems with the peer-review system.
• There is no clear definition of “peer-review”, nor any standardization of protocols.
He described this lack of a definition as follows:
“What is clear is that the forms of peer review are protean. Probably the systems of every journal and every grant giving body are different in at least some detail; and some systems are very different. There may even be some journals using the following classic system.
The editor looks at the title of the paper and sends it to two friends whom the editor thinks know something about the subject. If both advise publication the editor sends it to the printers. If both advise against publication the editor rejects the paper. If the reviewers disagree the editor sends it to a third reviewer and does whatever he or she advises.
This pastiche—which is not far from systems I have seen used—is little better than tossing a coin, because the level of agreement between reviewers on whether a paper should be published is little better than you’d expect by chance.”
Other problems with peer review pointed out in his study are:
• It is slow and expensive
• It is inconsistent
• Reviewers often have biases
• It allows reviewers to block publication of ideas they simply disagree with, or to prevent their scientific opponents from publishing their ideas.
• It sometimes functions to inhibit innovation.
These findings by Richard Smith were borne out by a number of other studies. In particular, John Ioannidis’s study of research findings entitled “Why Most Published Research Findings Are False” [2] highlighted the size of the problem. Further issues, along with examples, were identified in work published by the BMJ. [3] And chance has been shown to play a big part in whether something is approved. [4]
As a result, it’s clear that the peer review process that we have isn’t working.
Proposed Additions
Here is how we can improve peer review.
Start out with standard double-blind peer review, where neither the authors nor the reviewers are identified. Then when the journal article is published, the journal would also publishe online a single web page containing a transcript of the complete correspondence between the authors and the reviewers. The reviewers would be identified by their names and credentials.
Below that would be the usual setup for a thread of comments and questions.
The reviewers, the authors, other scientists in the field, and the public would be invited to address the issues. It would be lightly moderated, no ad-hominems, stick to the science.
This would have a number of benefits for all participants, and would solve a number of the problems with traditional peer review:
Benefits For The Reviewers:
• It would allow their specific, detailed views and comments on the subject to be made known. In many cases the reviewers will know more about some aspect of the paper’s subject matter than the author. This would give their views on that aspect of the subject matter and on other related issues a much wider audience.
• It would allow for the publication of the minority views among the peer reviewers.
• It would give the reviewers a chance to go on record that although they recommended publication, they still disagreed with or had issues with certain parts of the study.
• It would reveal the process by which they shaped the paper for publication.
• It would place on permanent record their contributions to and their positions regarding both the paper and that area of study.
• In science, as with many occupations, all publicity is good publicity.
For The Authors
• All of the benefits for the Reviewers apply to the authors as well.
• It would tend to discourage bad behavior by reviewers. Scientists are humans, and sometimes do things for less than the finest motives. Making the process transparent and visible to all will help encourage reviewers to act out of scientific motives.
For Both Authors And Reviewers
• It would encourage both sides to be clear, collegiate, and dispassionate in their interactions with each other during the review process, knowing that their words would eventually be published.
• It would provide a public forum where the discussion that went on during the review could be continued, with both reviewers and authors having a larger time and place to explain and defend their ideas.
For Other Scientists In The Field
• It would provide much deeper insights into both the areas of agreement and the areas of disagreement between the reviewers, the authors, and other scientists studying the subject in question.
• It would allow other scientists to see just exactly why the paper was recommended for publication.
• It would provide other scientists a forum to ask questions of the reviewers, the authors, or other scientists commenting on the subject matter.
• It would offer other scientists a forum where they might be instrumental in resolving any unresolved questions or possible misunderstandings between the authors and reviewers, by contributing their own insights, theories, questions, and understandings.
For The Public
• It would give the general public a look “behind the scenes” at what goes into the process of peer review, by both authors and reviewers.
• As with the scientists, it would give the general public the chance to improve the scientific process.
• Science lives or dies by transparency. At present, the process is very opaque to the general public.
For The Journals
• Inviting new subscribers. People will be more willing to subscribe to a journal that has an understandable process behind what they are reading.
• Access to volunteer reviewers. Scientists are just like anyone else, looking for recognition. This will give them a chance to get their ideas published and to defend their ideas under their own names. And in turn, for many, this will make them more likely to volunteer their time.
• It will give all scientists confidence that your peer review process is solid, and thus your journal is reputable.
For Science In General
We desperately need a substitute for peer review. While this proposal may need fine-tuning, it goes a long way towards solving the problems with the current peer-review process.
• It makes scientific findings and the scientific process accessible and understandable, thus improving the acceptance of scientific findings.
• It offers, on a paper by paper basis, the opportunity for scientific ideas to be tested, updated, refined, or falsified. Over time, the ReviewPlus page for any given paper will provide a record of just where those ideas fit in the current scientific understanding.
• It would provide a place where even sometime later, as new understandings of the subject come to light, they can be made publicly available to either support or falsify the conclusions of the paper. This would have the effect of keeping the status of the various ideas in the paper up to date. Have the claims been falsified over time, or are they still valid?
• It would provide a place where scientists who disagree with all or some part of the study can have their objections and counter-arguments placed on permanent record.
• It would give all of us a clear idea of the complexities, areas of agreement and disagreement, support for and arguments against, and a much more comprehensive view of the state of current science regarding that study.
Implementation
For clarity, let me call the single page where the discussion occurs the “ReviewPlus” page for that specific study. For this to work, the ReviewPlus discussion web page needs to be accessible, interesting, and most of all, scientific. So, on the ReviewPlus page:
• Identify the authors’ and reviewers’ comments with some special symbols like “A:” and “R:“, with no special symbols for everyone else.
• Allow several levels of indenting so conversations can diverge to discuss details.
• Moderate all the comments.
• No anonymous posting. Make people take responsibility for their words. Ensure that they have a valid email address.
• Have few but clear site rules, and enforce them, eg:
No personal attacks
Stick to the scientific subject of this study
Keep it polite.
Something like that. And then enforce them strongly.
• Don’t censor comments in private. Replace them with something that says “[SNIPPED: Please stick to the scientific subject.]” or “[SNIPPED: No personal attacks.]” in bold. This lets other folks know what’s going on. In addition, you can snip just that part of an otherwise interesting comment that breaks site rules, and leave the rest.
• Ban repeat offenders, but again not in private. Simply replace their comment with something that says [BANNED: Regrettably, your repeated transgressions of site rules have earned you a ban. Please email the editors if you wish to be reinstated.]
ReviewPlus Page Proposed Layout
At the top of each individual ReviewPlus page would be what is sometimes called the “money graphic”, the one graphic from the study that best exemplifies and embodies the ideas.
Below that money graphic we’d have:
The title of the study.
The names of the authors
The names of the reviewers, with their relevant credentials
Then, the “lead-in”, viz:
This page hosts a public discussion of the ideas presented in the above study. The discussion involves some or all of the authors, the reviewers, other scientists in the field, and interested lay people. It is a place to strive for understandings, to request clarification of unclear parts of the study, to provide additional links and information that tends to either support or falsify the study, and to ask questions of the authors, the reviewers, and other participants.
That example above is just a very rough first cut at what is a critical part of the page. For newcomers it sets the tone and describes the direction and requested behavior. In some ways it’s the most important part of the page. It should be brief, clear, and interesting.
Below the lead-in would be the Abstract of the Study.
Below that would be a standard set of links to:
The Study (whether paywalled or not)
Supplementary Online Information
Data As Used
Code As Used
Next would come the Review Transcript. The Review Transcript would be the complete and exact record, warts and all, in chronological order, of the questions, the to and fro, the discussions, and the changes and exchanges between the reviewers and the authors during the review process.
Then, after the Review Transcript and above the “Comments” text box, would be the ask, e.g.
We invite you to contribute to the ongoing scientific discussion of the issues arising from this study. If you choose to do so, please follow these few simple site rules. Comments that are over the line will be snipped to maintain decorum.
• To avoid misunderstandings, please quote exactly or link to what you are discussing. This avoids endless misunderstandings.
• No personal attacks. We repeat. No personal attacks. We repeat.
• Stick to the scientific subject of this study.
• Please keep it polite, friendly, and collegiate. We are not adversaries. We are working together towards greater and clearer scientific understanding.
The ReviewPlus page will also need a checkbox or other way to subscribe to further comments on that particular study.
Rejected Papers
Each journal should publish papers that have been through the full peer-review process but that have been rejected, in electronic form only, and allow free public access to them. And each rejected paper should have its accompanying ReviewPlus page.
In a way, this is more important than publishing the accepted papers. Science proceeds by falsification. But we have hidden away the most important falsification in the entire process, the falsification done by the reviewers.
These provisionally falsified claims are very important. If the reviewers’ rejections hold up, it will provide the ideas and logic needed to assess future repetitions of the same claim. If an eminent statistician peer-reviewing my work has convincingly refuted my argument, that should be in the public record.
Then the next time the argument comes up, someone could just say nope, someone tried that, here’s a link t why it doesn’t work.
It would also encourage people to be reviewers, since their eminently scientific work of falsification would not be hidden away forever … and where’s the fun in that?
Discussion
Let me emphasize that Peer Review Plus is not something different from traditional peer review.
It is something in addition to traditional peer review … that’s the “Plus” part.
Peer review will go on in exactly the same way as before, unchanged … the only difference is that at the end of the usual process the complete correspondence between the authors and reviewers, their ideas, objections, compromises, clarifications, and all, will be published on the given study’s ReviewPlus page and people will be invited to contribute.
Peer Review Plus is a bolt-on addition to traditional peer review, not a replacement. And daring to dream big, the fact that it is a bolt-on to peer review, and not a replacement for peer review, will make the global transition from peer review to Peer Review Plus much simpler. It will be able to take place gradually, one journal at a time, without any disruption, changing, or weakening of the current system.
=============
Looking to the future, there should be a standard template for the ReviewPlus web pages, such that any journal can simply download the template, fill in the blanks and the links, and they’re in business. This is worth keeping in mind as the first pages are designed. This standardization is important for several reasons.
First, it will make it easy for any journal to opt in to adding Peer Review Plus to their traditional peer review.
Second, it will let the reader know where to find specific info on any journal’s ReviewPlus page. People should find the format familiar even if they’ve never been to the particular journal’s website.
And third, it will help with searching. If all the fields (authors names, review transcript, links to code, abstract, etc) are the same and in the same order, it will be possible for someone like Google Scholar to search the global set of ReviewPlus pages curated by all journals by field, or title.
References
[1] Smith, Richard. “Peer review: a flawed process at the heart of science and journals.” Journal of the Royal Society of Medicine vol. 99,4 (2006): 178-82. doi:10.1258/jrsm.99.4.178
[2] Ioannidis JPA (2005) “Why Most Published Research Findings Are False.” PLoS Med 2(8): e124. https://doi.org/10.1371/journal.pmed.0020124
[3] Henderson, Mark. “Problems with peer review.” BMJ 340 (2010).
[4] Cole, Stephen, Jonathan R. Cole, and Gary A. Simon. “Chance and consensus in peer review.” Science 214, no. 4523 (1981): 881-886.
Pal review. Waste of time.
Incomplete comment. Waste of time.
This looks like an obviously sensible proposal, which is why it will not be done. Some people like the role of gatekeeper, and will oppose any dimunition of their power.
I’m not sure that that is valid. Surely it’s up to the individual journal to decide how it wants to do peer review? Eventually there’s a “race to the top”, and others will have comply. It’s the old “publish or perish” thing again.
As usual, an excellent article by Willis.
There is no “race to the top.” Every journal races to profitability consistent with its ideology.
The worldwide internet doesn’t need journals to filter information. People are not stupid.
When journals and the UN IPCC pump out “Hockey Sticks” we know climate science is post-normal.
When the UN IPCC pumps out CMIP climate models we know climate science is post-normal.
Thankfully, lies are eventually exposed. Sadly, sometimes it is after great cost.
I fully agree with you. Politics do not want a transparent process, or to “strengthen” peer review. On the contrary, they’re happy to see science being brought down to the partisan, religious, dogmatic level where censorship is omnipresent. These people hate even the idea of debate, and love to make their politically motivated decisions citing “their” science and cancelling everything else.
Important, here, to draw a distinction between academia and raw science. The peer review obsession can be viewed as a hindrance and not a mode of advancement. Although, it has its place. The greatest cutting edge works can never be peer reviewed. But, as you say, they should still be posted.
For instance, the optical radiation physicists have now monopolized peer review of climate, as self declared purveyors of truth. And yet, they have not demonstrated an ability to understand that their perspective only represents one small factor, of many, within climate dynamics. It’s an embarrassment to ‘science’; academic method gone awry. Why must we pass through their guarded gates to proceed? Why must we seek their approval?
The reason liquid water exists on the surface of Earth is due to the process of DEEP CONVECTION. Ultimately it is the reason present lifeforms can exist on Earth.
The process is poorly understood. It is stuck between aviation risk and meteorology but does not get considered as the fundamental process for Earth’s energy uptake because there is an absurd fixation on atmospheric spectral absorption. Climate is the new religion. The Climate church supports a make-believe world with no basis in physically verifiable beliefs.
There is no requirement to rewrite the entire basis of the greenhouse effect to highlight problems with the current policy approach, but you are welcome to do so.
What the current consensus model of science in the peer review process lacks is the critical principle of True Scientific Controversy. This involves disagreements over how data should be interpreted, over which ideas are best supported by the available evidence, and over which ideas are worth investigating further. True Scientific Controversy involves competing scientific ideas that are evaluated according to the standards of science such as: fitting the evidence, generating accurate expectations, and offering satisfying explanations. This often requires judgement. The gatekeepers of climate science have abused the scientific method by suppressing True Scientific Controversy. Modern academia has demonstrated that it is unable to support True Scientific Controversy due to financial interests, ego, and other conflicts of interest. Academics actively suppress opposing views, which is contrary to scientific advancement.
In climate science, to find a 2-4 w m-2 extra energy at the surface can be described by numerous explanations. This is not disputable, and the evidence exists.
Professional academics tend to have a strong sense of self importance, and they tend to be high achievers. They are praised with awards and respect. Their entire sense of self rests on always being “right”. This psychology is in direct opposition to science.
Perhaps their institutions should offer an honor or award for individuals calling into question widely held beliefs within their own disciplines as a motivating factor. There must be a mechanism to restore integrity within these departments. There must be a mechanism to encourage challenging their peers. It seems quite cushy when you’re surrounded by like-minded individuals. This way they can still get their praise and save face. If there is no debate within departments they might as well pack up and go home. Additionally, and critically, their institutions must protect them from lawsuits brought by the green energy sector and governments which have made guarantees to their investors. There are trillions USD at stake if someone sticks their neck out. It’s a dangerous game.
Thank you for your blessing to present facts. Why do you think open ocean surface temperature is limited to 30C? This has been frequently observed and occurs across all tropical oceans. It remains the case for the past and present without significant variation in the atmospheric mass.
Ya I got it. You post the same thing every day and I’ve discussed this with you. Are you waiting for someone else to put your ideas into action? Why not go for it?
I have gone as far as I care to go:
http://www.bomwatch.com.au/wp-content/uploads/2021/08/Bomwatch-Willoughby-Main-article-FINAL.pdf
This paper is freely available for anyone to review and understand the process of deep convection. Knowledgable people will study it and understand the process. Climate clowns will continue their struggle to grasp real world processes while they get deeper into their muddled, modelled mess.
I am not paying to have my work on climate published. Those who can grasp basic physics will latch on to it over time.
There are only three key points:
It is simple and undeniable. The process results in explosive mixing of moist air above the level of free convection all the way to 14,000m. The CAPE that drives deep convection is ubiquitous across the tropical oceans:
https://earth.nullschool.net/#current/wind/surface/level/overlay=cape/orthographic=-196.50,-3.38,356/loc=175.188,-17.964
This page is loaded with talented empiricists such as yourself, with ideas about what is not causing perceived changes to climate. It is a useful endeavor. At some stage though, in my opinion, it is worth considering describing the factors that are increasing people’s exposure risk to weather related hazards. This approach has many practical applications, and it can be quite persuasive.
If you are successful in overturning current policy with this idea, it will be highly beneficial for my work in environmental management. My budgets have been slashed repeatedly as increasing investment goes towards technology scams. My goal here is to increase awareness of land connections to climate. And I welcome anyone here wishing to implement a solution to get people out of the current CO2 delusion.
Water retained on land in tropical to sub-tropical regions is vital to ensuring deep convection occurs over land. You can see the way mid level water diverges from the Arabian Sea to India here:
https://earth.nullschool.net/#current/wind/isobaric/500hPa/overlay=relative_humidity/orthographic=-288.82,20.85,659/loc=64.571,23.680
With mid level RH at only 17%, there will be no level of free convection in the position shown. However the nullschool model is showing high CAPE:
https://earth.nullschool.net/#current/wind/isobaric/500hPa/overlay=cape/orthographic=-288.82,20.85,659/loc=64.571,23.680
4215J/kg. Deep convection is poorly understood and this shows the dilemma. The nullschool CAPE value is wrong – but they are trying.
There needs to be at least 30mm of TPW to get an LFC and cyclic deep convection requires at least 45mm:
https://earth.nullschool.net/#current/wind/isobaric/500hPa/overlay=total_precipitable_water/orthographic=-280.90,21.21,659/loc=78.609,11.080
Only the southern part of India has enough atmospheric water over the land to support deep convection. Once instability occurs. mid level moisture is pulled into the convecting zone.
South Africa east coast currently very wet with enough water to create cyclic deep convection over land:
https://earth.nullschool.net/#2022/04/16/1300Z/wind/surface/level/overlay=total_precipitable_water/orthographic=-327.34,-17.30,491/loc=33.473,-24.193
The myopic focus on CO2 has reached a point of criminal negligence. Getting the broad population to believe they can alter the climate by not burning fossil fuels is akin to a belief that the Earth is flat – it is nonsense. It misdirects effort from the things that should be getting done to ensure the land masses retain surface moisture sufficient to support deep convection when the sun warms the surface.
Convergence to the convecting column is also a significant factor. The surface in ocean warm pools gets around 15mm/day rainfall. That is twice what the evaporation so moisture converges from non-convecting zones. The divergence zones axperience about 3mm/day precipitation , lees than half of what they evaporate. Instability of the air column is more likely to occur over land because the air flow is more disturbed. So once the water column is established over land to start the cycle, it will draw moisture from the adjacent oceans because it is more likely to become unstable before the air mass over the ocean. Ocean water close to land can exceed 30C because the mid level moisture is drawn off and no LFC can form. This is observed in the Persian Gulf in August. The mid level moisture converges to the convecting zone in the Arabian Sea and is replaced with very dry air from the deserts to the north. Persian Gulf has never sustained a cyclone despite the surface temperature reaching 34C in August.
The optical radiation perspective represents only a small factor. Happer and Wijngaarden have also demonstrated beyond doubt, that CO2 or any other GHG will not have any important effects in the future with respect to climate. The advantage was, that the system is quantifiable (pure light absorption physics) and verifiable (by direct comparison with satellite data). It doesn’t describe climate, but it allows to rule out GHGs as “critical factors” which are “extracted” from questionable accordingly tuned climate models. The authors had a difficult time to publish, and their work isn’t cited as much as it would deserve for political reasons. Peer review is unfortunately not about science, it’s about control of information. I doubt that a process that suits the people in power particularly well will be easy to change.
In my opinion, H&W are the only show in town. All the others, the modelling mafia, are just imposters.
Right. H&W wish to demonstrate that their area of expertise is largely irrelevant for describing climate changes. It’s an awkward place to be, to render your discipline as having no practical application for climate change studies. It is people like H&W that got us into this rut in the first place, and so it’s prudent for them to get us out. It is an important lesson so that in the future it is not the optical radiation physicists we should be asking about climate and environmental policy.
H&W calculate 2.97 degrees warming per doubling of CO2 in a clear sky, so they aren’t the zero-warming saviours some hope for. Their methodology uses entirely sound radiative gas physics that doesn’t seem to contain any errors. One can make a good argument that low level clouds in the real world would result in a number lower than 2.97…..
The models come up with 3.7 and higher, yet claim to include additional aerosol cooling….something is definitely wrong with the models by a factor of one and one-half-ish….
See below Fig 3 https://arxiv.org/pdf/2103.16465.pdf if you think H&W are saying something else…
Compared to alarmists claiming 3°C to 9°C warming for a 50-80ppm rise in atmospheric CO₂, it is almost zero warming.
All of it beneficial.
Whether you want to start at 380ppm or the current 415ppm, a doubling isn’t reached until 760ppm to 830ppm.
I’m afraid you are misrepresenting the work. Try this one which includes the effect of relevant feedbacks in a clear sky scenario. The CO2 effects are practically indiscernible in real atmosphere. That is the point. So now we should be moving on from this overly complex conceptualization of climate. It is only relevant for theoretical interest. You have also critically confused the units. None of us should seek optical physics as a savior, only for them to undo the mess they have created so we can move on. https://wvanwijngaarden.info.yorku.ca/files/2020/12/WThermal-Radiationf.pdf?x45936
It is striking to me the number of people who have not understood these works. It could be why it has failed to be of much influence as of yet. They spell it all out for everyone to see. It must be frustrating for the authors.
I see other figures: The theoretical temperature increases were 1.7°C in the Sahara, 1.3°C at Mid-Latitute and 0.3°C for Antarctica for an “instant doubling of CO2” (Their paper from 2019). So no big deal there.
In H&Ws paper of 2020, there are values between 1.4-2.3°C depending on lapse rate and fixed or variable humidity.
Thomas Gold wrote of the ‘Herd Instinct’ as it relates to peer review, decades ago.
His writing is as true now as it ever was.
Journal03-03.cdr (jpands.org)
Japan us French mox plutonium caused Triple meltdowns 20.000 more years & chernolble hells ways to boil water + fake 1919 oil law betz claims no new wind inventions possible brainwash science is fact ignored while free wind and tides not used .by hoover dams size automatic feathering flat blade radial windmills science,?
Steven, that AI comment robot is really coming on. Well done.
I disagree, Jay. It’s OK for a version 2.4.1, but that Chinese-to-English being used is only good for technical instruction manuals and not general commenting.
? I think I just developed aphasia reading that.
I thought my dyslexia went out of control.
???????
I take issue with claiming “plutonium cause triple meltdowns 20,000 more years”. If this was Pu-238 the half-life is only about 85 years… However, I agree with “brainwash science is fact ignored while free wind and tides not used by hoover” That part is entirely sensible. Please correct the first line and you are ready to publish! ;-p
I think he forgot to reference “Einstine” though
Is this word salad Big Oil Bob in disguise?
I don’t do facebook or twitter.
Maybe somebody should copy that onto them and see if it passes “fact-check”?
Then let us know. 😎
a bot, and a crappy one
For publishing ruddy poetry you really have to go elsewhere.
Sensible proposal, internet exposure peer review bolt on.
Why it is unlikely to be adopted by main stream journals. But the grave peer review problems already exposed in medicine and climate mean those journals are now also much less trusted than before. Self inflicted wounds, when a layman like myself can expose multiple instances of outright academic misconduct in peer reviewed papers in journals like Science (Marcott) and Nature (Fabricius).
May not be necessary either, as the internet itself provides an alternative means of publication, as here at WUWT. And there are lots of people who have become fairly adept at sorting wheat from chaff, also as here.
What we have to understand is that what is commonly called “peer review” is actually ‘Gate Keeping’ for the benefit of the journals. That is, the journals are more concerned about their reputations than they are actually advancing science. Academic library subscriptions are their Bread and Butter. They sometimes fail at that, as you point out, however. But, basically, the journals are most concerned about excluding articles about ‘perpetual motion machines’ and controversial attacks on the prevailing paradigms. They want to play it safe. However, in doing so, they frequently fail.
Yes. “peer review” is mostly not what many people imagine. It is largely just to sort out the wheat from the obvious chaff and dross. It’s a crude time saver for people to avoid even starting to read articles by authors who can’t even operate a spell-checker.
Unfortunately, semi-competent frauds easily overcome this barrier.
This covers much in some disciplines.
I could easily have fabricated results in my very limited academic publishing and no one would know. Very little published science is ever repeated/replicated by other scientists. Why would they? Who would pay them to do so? Trust is still paramount, but is abused in some fields more than others.
Where the discussion rests on GW I can’t find the scientific importance of peer reviewed given that the scientific consensus is that the science is settled.
I’d say ‘peer reviewed’ is then in equal portion save the ahem, science, and save our tax funded paychecks. The collective opinion and the collective paycheck must survive. Truth be damned. CAGW policy Depends on it. Depends is capitalized because it’s a brand name.
There are two costs that need to be addressed if one wants to have an article published in a top journal: the cost to the author and the cost to the reviewer. I would suggest that if anyone wants an article of say 10 pages published that it be at no cost but that the author be required to review two ten page articles with a half a page of comments – positive and negative. Those who review without publishing much should receive some other appropriate remuneration, perhaps technical books. Journals can negotiate with publishers for this scheme to encourage good reviews which are time consuming. Both would benefit.
That’s an interesting idea for author’s reviewer contribution, might be less bureaucratic than “double blind.” Reviewers lost out from lack of reward to publication with good old “Impact Factors” along with political incorrectness. “…double-blind peer review..” might work, sounds good, but in some areas I examine papers have a “signature,” especially obvious with the multiple authors situation.
One of the last to succumb to models are oyster biologists, been too much immersed with those lately. They seem to be similar to climate and similar, but don’t end with usual pleas except some do mention warming that some biologists seem to have more of a lack of homework consensus than more physical-chemical types. They may catch up (devolve?) as boilerplate introductions, sometimes including questionable papers, and too verbose with repetitions too common however. One did say that they need to “…end the debate…” whatever that meant.
Graph in this paper shows how biologists have joined the exponential rise of easy equation solving. Ganju, N. K., and 13 other authors. 2016. Progress and challenges in coupled hydrodynamic- ecological estuarine modeling. Estuaries Coasts 39(2):311–332. https://doi.org/10.1007/s12237-015-0011-y Open Access
Now defunct Tulane Studies in Zoology used to publish reviewer’s names. Nice requirement. Another casualty of the system along with many more local publications.
“Each journal should publish papers that have been through the full peer-review process but that have been rejected, in electronic form only, and allow free public access to them. And each rejected paper should have its accompanying ReviewPlus page.”
Usually a rejected paper is submitted to another journal. The proposed approach would preclude that as at the time of submission it is necessary to affirm that the paper is not currently submitted elsewhere.
Publishing the “rejects” would clog the system with a lot of really bad stuff – I speak as a regular reviewer for 40 years. Any papers that represented real innovative ideas would just be buried.
On the other hand, I fully agree that reviewer comments and reviewer identity should be public once a paper is published, and reviewers should get academic recognition for their service.
The notion of double blind review would only mask identity of the small players. Topic, technology, viewpoint and citations can usually identify the lab a paper comes out of.
I could see resistance to the publishing of rejected manuscripts, as usually the authors just resubmitt elsewhere. It’s a numbers game, if the peer reviews are no better than tossing a coin, it becomes a relatively simple binomial probability distribution.
So on average for a great paper (I say great – but what I mean is non-controversial and including some nice fact that is appealling to the general public: Frogs recognise people! or We’ve only got 12 years to save the world! etc.) – it would take 1 to 4 efforts maximum and you’d be very unlucky to go for more, whereas for a difficult paper it may take 15-20 attempts. Even then, it may be worth it, if the alternative is a rejected permanent link.
I’m not saying it is a bad idea on this basis, only that I think it will be unattractive for this reason.
Who does the “Peer review” on all the climate BS that the UN is pumping out?
18 years ago the UN said this about winter sports venues.
UN Winter Sports Forecast | Real Climate Science
And since then the ski resorts in the Rockies have been having a lot of great years. Here is the snow report from the resort I was at a few years ago in the Tetons.
Snow Report – Grand Targhee Resort
Meanwhile, it is April 16th and here at my central Indiana home we have a hard freeze warning for tonight.
Don’t give me that crap about it being just “weather”. This weather was not supposed to be happening!
No kidding Rah. My kin In ND barely were able to feed their cattle in the blizzard. I thought we weren’t supposed to know what snow is anymore…maybe next year 🤔
I”d leave the “weather proving anything” to the Greens & we only respond to the nonsense with data to
show other times that it was ~ same or worse. Weather’s a lot more variable than we realize.
2012- many MN lakes set early ice out records. 2013- many set records for the latest- 6-8wks apart.
Earliest Dad started planting grain- ~March 25, 1945. In 2008, we had lows in the teens the second week
in May. (We had no planting weather before then which would have to have been replanted. 1945- global
cooling, 2008- GW). Many MN towns set record cold temps in 1996- GW. The winter of 1936 was very,
very cold & snowy, the summer had record heat. 1988- the hottest year since then- a drought yr when
Yellowstone & the Black Hills burned- ~June 15- 101F, within 36 hrs corn froze as it turned slightly black
2 days later. The list is ~ endless where weather can change a lot. Go figure!
Sure it can change a lot, I’ve Berg Heiled the three highest peaks in the Alps and several other high ones. Weather changes faster at altitude than anywhere else in my experience.
But any way one cuts it the weather is not what has been predicted by the alarmists now, just as sea levels are not what was predicted. It is long past time for all of those so called experts that made those alarmists predictions to be held accountable.
But they are still out there pumping their same messages of doom, and many, even more stridently now than ever before.
I passed beyond the saturation point a long time ago and look upon those adults of middle age or older that continue to buy into the lie that CO2 is the thermostat with disdain.
BTW New England, and the lower great lakes in the east are going to be plastered with snow on Monday. And it’s looking like more than 1/2 of the US is going to see much colder than average temperature right up through the middle of May.
If we had the same weather, including the snow in the middle of June would you say that it’s just weather? It is often noted here and else where how it snowed in July in New England after the 1815 Tambora eruption.
Could not agree more Willis, but I think the establishment will view it as a turd in their punch bowl.
Willis, you have my vote for amateur scientist of the era. Your suggestions are a triumph of reason.
Whenever someone interjects the question if someone else’s ideas were peer reviewed, I ask them who it was that peer reviewed Newton and Einstein.
That’s when the slack jaw expression overcomes their face right before they get angry about being asked the question.
As I recall Einstein’s Annus Mirabilis papers were reviewed by either Max Planck or Wilhelm Wien.
Willis – what a thoughtful article. I do agree that something needs to change. I suggest that yours would be an excellent guideline for major journals to use, but that it should not be the only method. There are already journals breaking the mould in their own ways and creating progress.
In science, a study which fails to find the hoped-for or indeed any conclusion can be valuable and should be published. In the same vein, I really like your suggestion that rejected papers must be published too: “Each journal should publish papers that have been through the full peer-review process but that have been rejected, in electronic form only, and allow free public access to them. And each rejected paper should have its accompanying ReviewPlus page.“. However, I suspect that your “in electronic form only” recognises that the improved system would encourage too many low-quality papers to be submitted because the authors would know that the papers would get published no matter what the reviewers said.
In the end, I suspect that editors do need to be able to use discretion, but your proposal could still lead to a major overall improvement.
There are a lot of journals out there, devoted to many different subjects of course. A very real issue that many struggle with is getting even three people to review a submission. Reviewers get nothing for their time, which has to be taken from whatever tasks they are on the hook to perform at wherever they are employed. So the incentives to do them are quite small, and of necessity reviews have to be done as quickly as possible.
Journals are a commercial product. They exist to make a profit. No-one publishes anything as a charitable endeavour. Journals make money.
At the moment the prestigious journals in each field have a captive audience. Every major university or research centre in their area needs to buy them.
How does this proposal make any money?
I think that a more rigorous peer review selection process is needed.
First of all, there has to be an adequate reward for the time and effort spent on a good review. The reviewers need to go through a certification process where the tenants of a good and fair review are taught – it obviously is not happening in the classrooms. once certified, they can participate in peer reviews – obviously this means an adequate reward for all this trouble.
One can lose one’s certification given lazy or abusive behavior. Peer review should be a status enhancing process.
When a paper is reviewed, the feedback must remain on factual aspects, not opinions. If there is a difference in opinion, then this can be mentioned but not grounds for rejection. If anthropologist #1 says the skull fragment looks to be of Homo Alpha and reviewer #1 says “no way, its of a Homo Beta” then that can be put into a comment box with the publication, but it cannot be used as a reason to reject the paper – they are opinions. If anthropologist #1 says it is the skull fragment of an alien then that can be rejected as there is no formal evidence of aliens ever existing. The author would have to provide some proof of the existence of said alien other than a bone fragment that could just as well be human.
The rest of what you propose all sounds reasonable.
Willis–
A fine well-thought-out proposal. Some journals are already doing this. Case in point: last week a journal called Sensors published a paper of mine on particle monitors.
https://www.mdpi.com/1424-8220/22/7/2755
There were 4 reviewers of the original submission, with two rather negative and two positive.
On the second round, the two negative reviewers recommended rejection. Only one reviewer recommended acceptance. However, the journal editor opened up a third round. At this point, all four reviewers recommended acceptance, although continuing to indicate some disagreement with parts of the article.
If you go to the link, you can go a sublink called Review Reports and there you will see every word of all reviews and my responses.
Peer review was invented so one could publish without showing their work. The scientific method requires you release all of your research so that others may duplicate your work and prove or disprove your work.
Remember cold fusion. I am not sure that what was published was intentional fraud but seeing nobody was able to duplicate the work, something was wrong with the premise of the experiment. Being able to independently duplicate the work is an absolute requirement of science. Without it, you have an open invitation to fraud and we all know that when big money is involved, all people in science will not be influenced by the money. /scr
I’ve written in detail before about a malicious reviewer and sycophantic subeditor. He asserted that the paper didn’t consider something important. When I pointed out that there was a whole section devoted to it, with an obvious heading (and maybe the subeditor should ignore this expert), she said that she would send it to a third who didn’t try to hide that it was the same malicious reviewer. This time, it was rejected for such poor grammar as to be unreadable. Strangely, not an issue the first time or the reviewer who liked the paper.
I would have loved it if this had been made public so we could all have a good laugh at their expense. They clearly were use to behaving this way, often.
A similar occurrence for me was a proposal to NASA for advanced laser equipment in a program which was explicitly designed to provide university researchers with the ability to duplicate the advanced experiments that were being done at NASA. When I got the response it was rejected with virtually no comments except for the remark that the work would merely repeat work being done at NASA! Needless to say I wasn’t happy and called the person in charge of the program who was very embarrassed and apologized. Next year I resubmitted the proposal with the added letter that requested that the previous reviewer not be used. That time I got the award, just a year’s delay.
Both Richard Smith’s article and John Ioannidis’ focus on medicine. Medical results are almost exclusively evaluated with statistics. The method is not science.
I have both published and reviewed science articles. Reviews in Chemistry, where I have direct knowledge, are often good. In my experience, editors of Chemistry and Chemical Physics journals have allowed my controversial papers after strict but fair reviews.
The further one gets from the core sciences, Physics, Chemistry, Biology, the more statistics comes into central play and the further from scientific methodology — falsifiable theory and mortally threatening result — is the field practice.
Financial interest is the big problem in Medicine. Billions of dollars, first in expense then in profits, can depend on drug tests. The pressure to prune, over-interpret, or tendentiously extrapolate data can be intense.
Tess Lawrie’s conversation with Andrew Hill about Ivermectin is a perfect example. They had been writing up a major meta analysis showing Ivermectin’s strong efficacy against Covid-19, when Dr. Hill preempted their paper with his own. In it, he reported equivocal results.
The video shows their conversation. Dr. Lawrie is direct, passionate, and outspoken. “The data are clear,” she said. “We can save people’s lives right now!”
Dr. Hill is shifty and avoids direct answers. He looks away. Eventually it becomes clear that UNITAID made a $40 million donation to Dr. Hill’s research dependent on a poor recommendation for Ivermectin. UNITAID’s people actually wrote the conclusion to the paper.
Afterward discussion here. It was all about money. UNITAID is Bill Gates money.
In full view there is the problem that has been rife and hidden behind medical research probably for the last 50 years.
So, the problem is not with peer review. The problem is with people who put their financial or political interests ahead of their professional integrity and justifiable personal ethics.
No matter the revised method of review, it will not survive the abandonment of integrity and ethics that plagues peer review in medicine.
Other fields where the data are murky suffer from advocacy peer review, most notably cultural studies and sociology. Most of published material there is never cited and does not rise above garbage. Unfortunately, policy-makers take that stuff seriously.
‘Unfortunately, policy-makers take that stuff seriously.’
Case in point is the 2021 Nobel prize in economics, awarded to the author of a econometric paper that purportedly showed that increasing the minimum wage had no impact on unemployment. This, of course, violates the ‘law of demand’ (quantity demanded varies inversely with price), which most folks would probably consider to be obvious.
Multiple studies continue to show that Ivermectin is worse than useless. Have a look at:
https://www.cidrap.umn.edu/news-perspective/2022/02/ivermectin-futile-mild-moderate-covid-19-study-finds
Believing and promoting quack cures results in people dying needlessly. Vaccines work and reduce the risks of dying by almost an order of magnitude. Furthermore there are new and effective anti-viral drugs that have been proven to work. The fact that Republicans in the US appear to prefer dying to taking a vaccine is incomprehensible to most of the rest of the world.
Izaak — your linked article: “There were no significant differences between the two groups in … 28-day in-hospital death,..”
The paper: “The 28-day in-hospital mortality rate was similar for the ivermectin and control groups (3 [1.2%] vs 10 [4.0%]”
The paper says the control group suffered 3.3 times more deaths than the Ivermectin group. Your evidentiary article says there was no significant difference.
Article: “There were no significant differences between the two groups in … rates of mechanical ventilation,…”
Paper: “Mechanical ventilation occurred in 4 patients (1.7%) in the ivermectin group vs 10 (4.0%) in the control group”
The paper says the control group needed ventilation 2.5 times more than the Ivermectin group. Your evidentiary article says there was no significant difference.
Was article author Mary van Beusekom lying? Or just incompetent?
And how about you? You didn’t read the actual study, did you. Just blindly touted the article, didn’t you.
The study included people who already exhibited “mild to moderate illness (Malaysian COVID-19 clinical severity stage 2 or 3; WHO clinical progression scale 2-4) within 7 days from symptom onset.”
Ivermectin is most effective when started immediately upon first appearance of symptoms. Not days later.
The study patients were already past the initial infection, when the virus had already spread into the tissues and lungs.
The study also excluded people who tested positive for Covid-19 but were asymptomatic — the very people most likely to benefit from Ivermectin.
As usual, Izaak, you’re negligent and wrong. Your opinions followed widely have caused mass death. Your willing cooperation in that criminal enterprise stains you with the crime.
Ivermectin for Covid: 82 studies, 129,808 patients, 64% risk reduction.
VAERS deaths and injuries disprove your fatuous remark about so-called vaccination. The incomprehension of the rest of the world merely proves them fools. And your remark about Republicans is just stupid.
Quack-quack-Izaak.
The differences are insignificant given the size of the trial. Also note the levels of adverse effects report:
“Five serious adverse events—four in the ivermectin group—were reported. Two patients had a heart attack, one had severe anemia, and one went into shock owing to fluid loss from severe diarrhea, while one in the control group had arterial bleeding in the abdomen. Adverse events led six patients to stop taking ivermectin and three to withdraw from the study. “
So combining the negative results from treating COVID with Ivermectin with the adverse effects means that it is worse than other treatments. As all other properly conducted trials have shown.
I do not have a drum to beat here. It would be great if Ivermectin worked and indeed the initial trials were promising. But that hasn’t panned out and it is time to stop promoting it and instead move onto proven treatments. Even your claim about a 64% risk reduction is small compared to the effects of the vaccine in preventing death and hospitalisations. So why take something that isn’t proven, has serious adverse side effects over a vaccine which has been proven in multiple countries to be safe and effective?
And here are the results of another randomized trial in Brazil:
https://www.nejm.org/doi/full/10.1056/NEJMoa2115869
again there was no effect found.
The opening statement immediately (Page 1, col. 1) renders the integrity suspect, given the uncritical acceptance of mRNA injection as a vaccine, and given the implicit acceptance of mRNA injection safety and efficacy, despite evidence to the contrary (pdf). Also here.
No test for which Corona virus variant has infected and is the cause of illness.
Once again, patients admitted to the study had, “acute clinical condition consistent with Covid-19 within 7 days after symptom onset;…”
Once again, IVM is most useful administered immediately with onset of symptoms. Not days after symptoms begin, allowing the virus to spread into lungs and tissues.
Once again patients received IVM 400 ugm/kg — twice the recommended dose.
Recruitment began about 1 year prior to the trial. Had any of the participants gotten ill with Covid-19 over the intervening year? The researchers do not say.
Patients were “randomly assigned to receive either Ivermectin or placebo between March 23, 2021, and August 6, 2021.”
Brazil transitioned from the gamma variant in March 2021 to the delta variant by August 2021. See Campos 2022, Figure 2.
So the study likely included patients ill with different Corona variants with perhaps different susceptibility to drugs.
People who had already been treated with mRNA were allowed into the study group. Presence or absence was not tracked. The study groups were not controlled for mRNA injection status — either in exposure or number of injections or manufacturer.
These problems are easily severe enough to set aside the study. One might almost infer from them that the study was designed to fail.
Further comments from Twitter:
Chris “Early Treatment” Martenson, PhD @chrismartenson
“/2 The 1st and most damaging issue that the trial was conducted in an area where IVM use was already exceptionally high. (my bold)
“They **DID NOT EXCLUDE** for prior IVM use. Only “known hypersensitivity” to tested drugs. Case closed. This trial cannot deliver interpretable results.
“3a The randomization of the TOGETHER trial was **NOT RANDOM.**
The trial protocol states: “Participants are randomly assigned with equal allocation (…) Allocation of participants to treatment arms is uniform across all concurrent interventions as well as placebo.”
3b
Did the TOGETHER randomization process equally and randomly assign patients to various arms in equal measures at equal times?
No.
Case closed. This trial didn’t even follow its own protocols.
“4: Finally, as an obvious nail in the coffin, if patients were randomly assigned, and the placebo was inert, and they had no reason to suspect they weren’t getting the treatment they wished, there wouldn’t be this ENORMOUS drop-out gap with only 40% adhering to placebo.”
Martinson notes that Table 2 shows that 58% of the Placebo group uniquely dropped out. Note final per-protocol versus initial intention to treat populations. Did they know they were getting a placebo?
@CarlingtonRacki
Mar 31
“Did you notice the 100% protocol adherence for IVM was the same in Table 2 & Table 3 (624), but for the placebo group this changes from 288 in T2 to 547 in T3.
“In T3, the 547 is footnoted as saying duration of placebo was 1, 3, 10, or 14 days, but doesn’t give a breakout.
“Why would the placebo adherence numbers be different in the tables and why were there multiple durations of placebo use? And how many people only took a placebo for 1 day and then took something else after?”
Oh my, you really do swallow the party line, hook, line and sinker.
“The differences are insignificant given the size of the trial.”
There isn’t a decent P-value in any of the primary or secondary outcomes, Izaak.
Nevertheless, you chose that study as definitive.
So live with your own choice: 3.3 times fewer deaths and 2.5 times fewer ventilations with Ivermectin.
“Five serious adverse events…” All the patients had at least one comorbidity. The list is impressive.
Hypertension, Diabetes mellitus, Dyslipidemia, Obesity
Chronic diseases: Kidney, Cardiac, Pulmonary, Active smoker, Cerebrovascular disease, Malignant neoplasm, Gout, Thyroid disease.
Chronic disorders: Neurological, Liver, Autoimmune disease, Immunosuppressive therapy.
But you know the five adverse events were due to Ivermectin, anyway. Which has an excellent safety profile in humans.
The usual dose for humans is 150-200 ugm/kg. Your study administered 400 ugm/kg.
That’s 27 mg per 68 kg male — twice therapeutic dose and more than twice the 12 mg dose recommended for Covid.
Scandal of the suppressed case for ivermectin
Your mRNA jab has killed in excess of 1000 times more people than any single vaccine, ever. The list of permanent injuries is far greater. Immunity lasts a few months and disappears.
This, you call safe and effective. You’re living in fantasyland, Izaak.
I don’t wish you ill. But if you’ve taken all the jabs, have a care for your health.
Is this the give them the medicine right before they die? See, it doesn’t work 😉
This is again off-topic but very worth knowing. Naomi Wolf interviewed hedge-fund data guy Edward Dowd — on Rumble where it won’t be censored.
Dowd is the guy who blew the whistle that insurance companies were reporting a 40% increase in death rate among the 18-40 year age range. The usual year-to-year variation in death rate in this group is about ±1%. So, we’re talking a larger than 6σ event. All due to mRNA jab injuries.
Wolf and Dowd talk about the evidence from released documents that Pfizer and Moderna knew the mRNA jabs were unsafe and ineffective by December 2020. They talk about the hidden data on deaths and injuries. Dowd surmises we will be looking at a huge problem of breadwinners unable to work because of injection spike-protein injuries.
Apparently released documents show that from the very beginning Pfizer was planning for six – count’em 6 — injections. And varying doses to see the effects on children.
It’s an hour-long conversation with an explosive revelation every 5 minutes.
Dr. Mengele isn’t dead. He has metastasized throughout the world.
Thanks Pat
Izaak is right.
Ivermecrin really is worse than useless.
Unless you’re a horse with intestinal worms.
Shocking medical fraud explodes Ivermectin | The Mallen Baker Show – YouTube
Isn’t that the one Fauci approved?
Izaak is wrong and so are you, Phil.
The IVM efficacy database does not include the Elgazzar study that is Mallen Baker’s focus.
The database authors go further, and note, “Notably, removal of Elgazzar, Samaha, and Niaee improve the treatment delay-efficacy and dose-response relationships and may further increase confidence when considering all information.” (emphasis added)
Well I just recovered from covid-omicron with nothing but Lemsip;
although my gut feels like its still full of horse-worms.
“Start out with standard double-blind peer review, where neither the authors nor the reviewers are identified.”
You didn’t discuss the most fundamental aspect of your proposal. In order to enable double blind peer review, the process starts with understanding the paper sufficiently to randomly and blindly assign reviewers who must have been pre registered and be associated to the field. And do you add statisticians to a biology paper or not? How about a chemist or physicist?
And to truly make it double blind, the number of reviewers for any given topic must be quite large which, I imagine, would itself be difficult for niche fields. And presumably reviewers are going to be unavailable at times.
So a major part of this proposed process is the construction and maintenance of a database of reviewers, their availability and their fields. And understanding what kinds of reviewers to use and do that all blind.
Jesus Christ, people: Just put all papers and their data and methods on the internet at no charge and let everybody have a whack at verifying, disproving or clarifying them. They are all funded by governmental entities. With modern communications journals are not needed anymore! Who needs gatekeepers?
CliSciFi is the premier example of the failure of scientific disciplines to monitor and correct internal aberrations. External controls are required. But the research cowboys will resist. F-’em.
Well said. You are right. The world has changed with the internet. You can publish on the net anything now. Each individual scientist decides what’s important and what is not and what is actually dishonest when we search. Peer-review does not matter and some journals like Nature have become political anyway. Just read their briefs. Publication only really only matters for promotion in Universities not for science. It’s tough for the young that reviewed papers do not matter but increasingly their funding capacity and teaching counts more than review in Universities.
I have published 383 papers but stopped in 2015 and just deal with my sponsors and communicate with whom might find it useful and put non-commercial stuff on Researchgate. It works fine.
Mick, from your handle I assume you are or were involved in the university game. It also appears you have an ongoing business and communicate with clients and potential clients and, on the side, publish general information (for your specialty) freely.
Am I correct? If so, is your approach since 2015 a developing trend in academia or with independent researchers? I’d really like to learn about your related experiences; they might also be of interest to a larger audience and other professionals such as yourself.
Acid test, get hockey stick people to agree to it?
I wouldn’t mind a small yearly fee for the general public to comment on these papers. 50$ a year or something. First, it keeps out any bots and most of the troll posting. Second, it could help cover any costs of publishing and reviewing and moderating. I believe it would help focus the discussion on the science of the article.
When I was a kid I read about a meticulous double blind experiment that gave significant positive results (something like five sigma) demonstating ESP. The study was submitted to a scientific journal. The editor of the journal wrote the authors and said they would not publish the paper because, (you just can’t make stuff like this up), their readers “do not believe in ESP”.
From then on I believe nothing anywhere in science until it appears in the kitchen.
Willis,
I’ve done peer reviews for several journals for many years. SOME journals allow the reviewer to score the paper as publishable with changes or rejected outright. In the case where changes are needed, I’ve seen papers incorporate so many changes that the final version looks almost nothing like the initial version. I’ve also rejected papers outright. There’s two problems I see with your suggestions, Willis.
1) The history of comments will often make no sense when looking at only the final version of the paper. The history of comments is, generally, of little use to the reader of the final version.
2) More than once, I’ve seen a paper I rejected outright appear without modifications in another journal. Not all journals adhere to the same review standards and authors shop around for a journal that will publish them. There’s lots of junk journals. It seems to matter very little to the press where the paper got published – if it supports the press’s narrative they’ll run a story about the paper anyway.
meab, no disrespect but your second item contradicts the import of the first: How are people, especially the press, to evaluate the faulty paper that does appear in a journal? All one hears is the “peer reviewed science” mantra.
There is sufficient evidence that major journals such as Science, Nature & etc. are hopelessly corrupt. How is one to decide if a journal is reputable? I’m referencing journals that publish CliSciFi, not other fields in which I have limited knowledge.
Thak you Willis, for sharing your ideas.
Very good points, but:
Here is how we can improve peer review
Definitely I agree with double blind peer review with transparent posting of the (anonymised)) referee correspondence.
Some journals e.g. open access ones already do this and go further with naming both authors and referees.
Also I agree with similar access to rejected papers and their correspondence.
Paradoxically this might increase the potential “punishment” aspect of a rejected paper – it’s rejection will be there for all to see.
Rejection won’t always be by political chicanery. Sometimes a paper really is bad.
Interesting suggestion, but it doesn’t address the common situation, in my experience, of having a number of reviewers recommending publication and a number voicing objections. The editor then decides what to do. Often rejection. That decision and its rationale should be public as well, but there may be great reluctance to do so, as it directly may affect the editor’s job security.
Again, the solution is to bypass the whole journal thing. Let the admittedly messy internet-assisted free-market of ideas sort the goats from the sheep. I know that this is anathema to control freaks of all sorts but truth will out only sans gatekeepers.
1) A massive issue with published science is that results are not even reproducible. It’s called the ‘replication crisis’. This WUWT article does not contain any mention of ‘replication‘ or ‘reproducible‘! Is it time WUWT introduced peer review?
2) Help is on it’s way with reproducibility
See: “‘Robot scientist’ Eve finds that less than one third of scientific results are reproducible.” press release | paper
The purpose of peer review is to give an article a ‘scientific spell-check‘ or grammar review = to see whether the article makes scientific sense.
Peer review is unable to figure out whether a article is true in the scientific sense. Far more than peer-review is needed for that. All possible empirical tests must be made. Taking both the stances of validation and falsification into mind. The results must be reproducible. The ideas or hypotheses must be consistent with all other accepted theories and laws
and a million quid must be deposited in some billionaires bank account other fact-checker will cancel it (just kidding)It’s obvious that peer-review hasn’t even been doing its job since a vast amount of junk science continues to be published. Just ditch peer-review.
I have trained 25,000 people in peer review over the last 20 years in the aerospace/defence sector. The approach uses experts to provide ‘free consultancy’ to othe projects, verifying their approach, methods and processes for optimum outcome.
During this time I worked with some of the sharpest minds on the planet (eg engineering next gen nuclear subs).
What I learned from them, and they from me, was very simple.
For peer review to be effective requires proper expertise (‘wisdom’ in the subject matter) AND appropriate independence (the greater the project consequence / risk / value / complexity – the more independence required).
Compromise either of these factors and you’re wasting everyone’s time, going through the motions.
Nice idea. I doubt it will happen for two reasons.
First, one of the purposes of peer review is to suppress ideas contrary to the existing dominant paradigm…. gatekeeping to block different views is a desired feature of the process, not a bug. If review is open, honest, and publicly recorded, gatekeeping becomes much more difficult, and the gatekeepers subject to critique.
Second, what you describe will consume far more time than most potential reviewers are willing to commit to. Most reviews are quick and dirty, not in depth. The holders of the dominant paradigm just want an easy thumbs-down option; they don’t much care what is in a paper, so long as it supports the dominant paradigm they are heavily invested in.
.
Scientists, like most people, are not saints, and suffer all the flaws the rest of us do. Trying to hold them to saintly standards in peer review will get a lot of pushback.
In health care and medicine, “peer review” is NOT done the way protrayed. The editor does not hand it to a couple people he or she knows and ask for their opinion.
You would have to be involved in the process to know how it actually works.
The reviewers comments and public discussion for each article would be a fantastic source of inspiration for a researcher looking for a niche to explore, that alone would be a huge boost to science.
Very reasonable, but you have omitted this key sections:
Downsides to the current status quo and to the winners of the existing system.