What the pandemic revealed about scientific publishing
Here are a few selected excerpts from this excellent article from elemental
Mark Humphries Jun 3 · 7 min read
I was reading my umpteenth news story about Covid-19 science, a story about the latest research into how to make indoor spaces safe from infection, about whether cleaning surfaces or changing the air was more important. And it was bothering me. Not because it was dull (which, of course, it was: there are precious few ways to make air filtration and air pumps edge-of-the-seat stuff). But because of the way it treated the science.
You see, much of the research it reported was in the form of pre-prints, papers shared by researchers on the internet before they are submitted to a scientific journal. And every mention of one of these pre-prints was immediately followed by the disclaimer that it had not yet been peer reviewed. As though to convey to the reader that the research therein, the research plastered all over the story, was somehow of less worth, less value, less meaning than the research in a published paper, a paper that had passed peer review.
Imagine reading about the discovery of the structure of DNA with that same reticence we use today: “In a recent Letter to the journal Nature, Cambridge University scientists James Watson and Francis Crick proposed a new structure for DNA (not yet peer reviewed). They claim their “double helix” model, a spiral of two strands of bases, both explains decades of experimental work, and provides a clear mechanism for copying genes. Their proposal drew heavily on data contained in Letters in the same issue of Nature from the teams of Rosalind Franklin (not yet peer reviewed) and Maurice Wilkins (not yet peer reviewed).”
Or consider this modern take on a certain scientist’s annus mirabilis:
“The past year of 1905 has been a remarkable for one Herr Einstein, who proposed no less than four theories new to modern physics in a series of papers. His first was on a much-anticipated explanation of the photoelectric effect (not yet peer reviewed), the second on how Brownian motion arises from the collision of invisible particles (not yet peer reviewed), the third on the equivalence between mass and energy (not yet peer reviewed), and the final paper updates Newton’s mechanics to be more accurate for objects moving close to the speed of light (not yet peer reviewed).”These imagined reports are both eye-wateringly ridiculous.
Continuing…
Pandemics don’t conveniently hang around waiting for that slow, grinding peer review process to judge science. Science in the time of Covid-19 has had to be nimble, quick on its feet, has had to show its findings to the world without the layers of review-and-revise. We’ve needed rapid research into models of transmission, into immunity and reinfection, into public health messaging and effective interventions, into drugs to treat symptoms, machines to support the severely ill, and the development of radically new types of vaccine. So pre-prints, those manuscripts put on the internet for all to read before peer review, became the weapon of choice. Long common in physics, pre-prints in biology and especially medicine exploded in number during the pandemic.
And the media have dealt with this explosion by consistently pointing out when research has not yet been peer reviewed. Presumably they do this to warn the reader that the research lacks the safeguards that peer review brings. The problem with that warning is peer review guards against nothing.
Does it catch fraud or manipulations of data? No, patently not: peer reviewers are not omniscient, so they cannot divine made-up data, nor can they check all the outputs of a lab to see when they’ve simply copy-and-pasted data between papers. If they were, we wouldn’t have the website PubPeer stuffed to the gills with people flagging potentially serious misdemeanors in published papers, nor Retraction Watch’s endless reporting of papers so dodgy they’re expunged from the literature.
And finally concluding. This is an excellent piece and should be read in its entirety at the source.
This then is why I was so bothered about how Covid-19 research is reported: peer review is no guard, is no gold standard, has little role beyond gate-keeping. It is noisy, biased, fickle. So pointing out that some piece of research has not been peer reviewed is meaningless: peer review has played no role in deciding what research was meaningful in the deep history of science; and played little role in deciding what research was meaningful in the ongoing story of Covid-19. The mere fact that news stories were written about the research decided it was meaningful: because it needed to be done. Viral genomes needed sequencing; vaccines needed developing; epidemiological models needed simulating. The reporting of Covid-19 research has shown us just how badly peer review needs peer reviewing. But, hey, you’ll have to take my word for it because, sorry, this essay is (not yet peer reviewed).
Mark Humphries researches computational neuroscience at the University of Nottingham, U.K., and is the author of “The Spike: An Epic Journey Through the Brain in 2.1 Seconds” (Princeton University Press) out now.
Peers. Dodgy review. Like a teenager proposing to go to a party: “OK, I see your classmates all say it will be fine. Have a good time.” Said no parent, ever.
Funnily enough, the most blatant examples of “pal review” seem to be from climate skeptics:
Example #1 : https://en.wikipedia.org/wiki/Pattern_Recognition_in_Physics
Example #2 : https://www.sciencedirect.com/science/article/pii/S0921818117306586
(non-paywall copy here: https://boris.unibe.ch/114178/7/Grosjean%20et%20al%202018%20%28Global%20and%20Planetary%20Change%29_accepted.pdf)
I have some experience in this, and I know for certain the problem infects the side you appear to take too.
The only side I take is that of science. What I want is to avoid policy being based on misinformation (for instance obviously flawed arguments, such as that the rise in CO2 is natural or barrycentric theories with no plausible mechanism).
Scientists are human beings, we *all* have our flaws and weaknesses, so expecting scientists to behave perfectly at all times is unreasonable. There are a *lot* of mainstream climatologists and very few climate skeptics, so to have two such extreme cases is somewhat surprising.
At the end of the day, if you can’t refute the reviewers criticisms, the paper shouldn’t get published. Take it on the chin, improve your science, improve your writing and try again. My record is three journals rejecting one of my papers before it was accepted in the fourth. Reading the paper, an experienced scientists might be able to detect which sections I had added to answer the reviewers criticisms from previous rounds! ;o)
I had a paper rejected by three journals also, and each time the paper improved, but in one review someone from outside the formal review process injected himself to demand the paper not be published, and on another the person actually said “this paper conflicts with currrent doctrine.” It was finally accepted by journal four, but who knows if it got a thorough review that time…
Great claims require great evidence. If you are overturning a paradigm, you will have difficulty getting it past the reviewers (I was doing something counter-cultural in the paper I mentioned as well). Can you give a link to the paper, is it climate related?
“Great claims require great evidence” is a ridiculous statement. Groupthink sheeple constantly recite it like an incantation, but it carries all the gravitas of a slogan in a commercial.
Great claims once arrived in Europe of an animal so tall it ate leaves at the tops of trees. Did it take great evidence to establish the claim? Of course not. All it took was to pull a giraffe around in a cart. Not since a week or two after the first cart was invented, has it been “great” or extraordinary to cart an animal about.
My experience was similar to Kevin’s, except it took 6 years, 13 submissions and 30 reviews to get published. And it’s climate-related.
The analysis is straight-forward: propagation of error through a calculation. The problem was that my reviewers were almost exclusively climate modelers, and climate modelers aren’t scientists. They know nothing of physical error analysis.
Apart from my last two reviewers, I have encountered no consensus climatologists who understand physical error analysis. They’re apparently not trained to evaluate the physical reliability of their own data and models.
I think the reviewers were correct. Figure 6 gives error bars containing values the the models could not plausibly produce. That indicates that you have not propagated the errors correctly. I seem to recall we have discussed this before.
Here is something one of the reviewers said about the paper:
I am listed as a reviewer, but that should not be interpreted as an endorsement of the paper. In the version that I finally agreed to, there were some interesting and useful descriptions of the behavior of climate models run in predictive mode. That is not a justification for concluding the climate signals cannot be detected! In particular, I do not recall the sentence “The unavoidable conclusion is that a temperature signal from anthropogenic CO2 emissions (if any) cannot have been, nor presently can be, evidenced in climate observables.” which I regard as a complete non sequitur and with which I disagree totally.
The published version had numerous additions that did not appear in the last version I saw.
I thought the version I did see raised important questions, rarely discussed, of the presence of both systematic and random walk errors in models run in predictive mode and that some discussion of these issues might be worthwhile.
https://pubpeer.com/publications/391B1C150212A84C6051D7A2A7F119#5
My comments on that thread are there under my real name (Gavin Cawley)
Dikran, “Figure 6 gives error bars containing values the the models could not plausibly produce.”
This is a standard mistake of climate modelers — unable to distinguish between a physical temperature and a statistical uncertainty.
Uncertainty bars are simultaneously positive and negative about a point.
Your interpretation of them as physical temperature implies a model output that occupies two opposed thermal states simultaneously.
Your view includes a physically impossibility, apart from falsely conflating physics with statistics.
I recall our debate well, Gavin. You never came to grips with what was actually done. You insisted on imposing your own mistaken interpretation, well-exemplified by the mistake you exhibited above.
Regarding the reviewer’s comment, the central point is here: “there were some interesting and useful descriptions of the behavior of climate models run in predictive mode.”
That statement endorses the error propagation. The reviewer’s problem is with the conclusion that a climate signal is undetectable.
Given the predictive incapacity of climate models, I would like to know what would constitute climate signal from CO2 emissions.
By the way, Gavin, it would be more forthcoming to comment here under your own name, as so many do.
These guys love hiding behind masks. I remember outing Gavin years ago when he unethically made ridiculous claims under this nom de plume.
‘Nom de plumes” eh “gator”?
Yep! You see Dik, I use a nom de plume to protect my family from crazy leftists, and not to pretend to be a cheerleader for my own work and profit. My use is ethical, and yours is not. Thanks for pointing that out!
show us where you outed him. his identity which is known and unimportant. is widly known.
prove you outed him pat. your peers want to check your claim
Hello steven
I don’t have to show you anything Mosher, just as you never show us proof that man is warming the planet, yet I am still supposed to “believe”. And really, you have much work to do before you could be considered my peer.
If you are thinking this is Gavin Schmidt, I don’t see that being possible as he is in a different country, definitely far from New York.
No, I am Gavin Cawley, I’ve already given my real name on this thread, and it’s never been a secret.
Your part of the hide the decline mob.
You kept it a secret until I outed you on Goddard’s site about nine years ago. I remember your cheering section being quite upset. Telling the truth is not your strong suit.
No mistaken identity, Sunsettommy.
Gavin Cawley and I have a long history at PubPeer, where he repeatedly and insistently demonstrated total ignorance about physical error, calibration, and uncertainty, and their analysis.
Apart from Gavin’s train wreck, most of the rest of the thread is argument from authority and oracular pronouncements.
A guy posting under various genus species pseudonyms made an effort, but one ultimately failing.
Sorry, you are just repeating you misrepresentation of the pub peer thread, and given you dismissive attitude to all technical criticism there, I don’t see a prospect for productive discussion here.
The problem is Dikran, you don’t know what you are talking about. And Pat Frank’s statement about error bars is absolutely correct.
Arguing that’s not the case just makes you look like an idiot.
No, I do know this topic, having published on it. The error propagation is obviously incorrect if you perturbed a real climate model by that amount and it were unable to produce an out put any where near as large. As I pointed out on the pub peer discussion.
“if you perturbed a real climate model by that amount”
Utterly, absolutely, nonsensical comment, Gavin. Uncertainty statistics do not perturb a climate model. Incredible.
What I dont understand is the argument you’re making? Its unrelated to the claim and implies to me at least, you dont understand the claim.
The claim is uncertainty is much larger than the sensible values the model can and does project and implies the model can project any result and will have no actual skill.
The model can be tuned to give whatever “reasonable” outcomes the modeler likes. And that’s exactly what happens. Oh, they dont aim for a specific ECS, they tune until the model is in the expected range.
But be sure, different tweaks will give different results, many of which blow the model up.
Dikran, some basic points which demonstrate the absurdity of your position.
So the only reason the mean of the models looks like Tobs. is because the input forcings look like Tobs. and averaging lots of useless models with random noise in them results in cancelling all the random noise and allows us to see the only common factor – the input forcings. The whole show of climate models is circular nonsense.
“I don’t see a prospect for productive discussion here.” Nor do I Gavin.
You allow no distinction between accuracy and precision. Physical science is impossible in your regime.
@ThinkingScientist: thank-you.
YAWNRTG (yet another who needs to read the GUM).
A mathematical probability of error range is in no way the same as an accurate error bar. It’s merely an indicator of where the author thinks the likely main data points are going to be.
And if the uncertainty width is larger than any possible physical magnitude, the predicted value has no physical meaning.
So it is with the air temperature projections of advanced (all) climate models.
Dikran’s view would preclude the concept of testing the statistical significance of a correlation coefficient.
When the range of error on a correlation coefficient at a given p value includes zero, it is not statistically significant. Same concept.
It is not climate related, but rather a different side research project.
E. Shimron, A. , Shirav, M. , Kilty, K. , Halicz, L. , Pedersen, R. and Furnes, H. (2020) The Geochemistry of Intrusive Sediment Sampled from the 1st Century CE Inscribed Ossuaries of James and the Talpiot Tomb, Jerusalem. Archaeological Discovery, 8, 92-115. doi: 10.4236/ad.2020.81006.
Cheers, I’ll have a look if I have access!
I see you’re more than casually involved, Kevin. 🙂
So who’s buried there?
Pat, a person could write a fascinating historical novel on the things we found out about that particular ossuary and the others in the Talpiot tomb. Lead contamination, metallic fragments in the James ossuary, implied familial relationships, etc. The story has something in it for everyone to hate, thus the seemingly frantic efforts to prevent publication — and to prevent Aryeh from public talks on the topic at certain Ivy League schools.
Most people don’t know how small the community of Jerusalem and its evirons was in first century CE — a small place with an impact on world history utterly out of proportion. Thus, those who take a stance that this family and person “jesus” never existed are fighting the reality of the associated names in that tomb. If one takes the entirely reasonable a priori stance that the family has to be buried somewhere in the Jerusalem area (deceased Jews were buried quickly, not hauled on roads back to hometowns), then the Talpiot tomb is the place almost without doubt (this upsets the Holy Sepulchre crowd). I won’t go into speculation about the metallic contaminants and the lead contamination of bone, or the embarrasments implied for the Israeli Antiquity Authority, the markets for stolen artefacts, and so on. It was a project begun in 2007.
Your story would make a fascinating post here at WUWT, Kevin.
A question: do the scored surfaces of the inscribed letters show the same age and exposure as the plain surface of the ossuaries?
What problem do the Ivy League schools have? It can’t be with evidence that Jews lived in Jerusalem. Can it?
Why were there frantic efforts to prevent publication?
The name Jose is strange. I’ve never run across that name among Israelis, and it doesn’t appear in my Concordance.
There are undoubtedly lots of brothers Jose and Jesus in Mexico, but in 1st Century Jerusalem? 🙂
And congratulations on publication, by the way. 🙂
This has been a long running part of the controversy of the James Ossuary. According to some scholars the second part of the inscription is modern and fake, others say it is ancient and entirely authentic. The seven year-long trial of Golan Oden, the antiquities dealer who owned the ossuary, resulted in a finding of not enough evidence to conclude the inscription is forged. From the way that I constructed my probabilistic argument and statistical analysis, in other words, from the way I conditioned the question, it is sufficient that only the first part of the inscription “James Son of Joseph” be valid — “brother of Jesus” is interesting but superfluous.
Aryeh and Moshe are two very solid geochemists and they were convinced of the authenticity of the patina covering the entire surface. I didn’t look into the matter as I had enough on my plate. My main part of the project was to figure out how to handle, analyze and interpret the major element geochemistry and explain the results in a way many people would understand. The figure of studentized data was my idea.
As you probably know, the IAA at the insistence of the religious authorities does a thorough “scrubbing” of these ossuaries to return human remains to a grave. However, if a person knows what they are doing, and understands the physical processes these objects undergo, it is amazing what evidence persists.
Karen King is one of the main antagonists here. I am unsure about why she was so antagonistic, but she had a planned lecture by Aryeh cancelled at Harvard. She came to some difficulties of her own lately in another controversy.
I am speculating here, but the big name scholars at the divinity schools base a lot of their research on rhetoric rather than physical evidence. I think they are a self-referential community for whom physical evidence just smacks of science invading their classical domain. Horrors! One who is scientific in his approach is James Tabor who was very supportive of our work. At any rate, we present a lot of evidence contradicting what one scholar called “1600 years of church doctrine.”
The name Jose is probably the Iose in Greek of the Gospel of Mark, and Yose in Aramaic. People often presume Yose is a pet name for Yoseph, but in the first century CE it appears to have been a name in its own right. It is unusual enough that it provided quite a bit of evidence about the owners of the tomb.
Since you are interested in this, there is one part to this story that I find fantastically coincidental. The Talpiot tomb had its closing stone rolled away, not by an angel, but by an earthquake in 363CE. The tomb being filled with landslide debris made ebverything in it chemically unique. I am a strictly scientific fellow, Pat, but it does seem that the fates wanted that tomb to be recognized for its contents.
Thanks for answering all my questions, Kevin. I should have realized the ‘J’ in Jose is pronounced ‘Y.’ I knew a Yosi or two during my postdoc at the Weizmann.
Maybe Jose is actually Joseph, the father of Jesus and grandfather of Judah. It makes a certain sense that Mary and Joseph would bookend those names.
Quite the story about Karen King. I read the whole Atlantic article. What a mess. Maybe her opposition is just professional jealousy. Who knows.
It was the big name (Humanities) scholars who opposed Galileo, too, and who oppose science in general today.
No surprise there. They hate most that science restricts their exegetical fantasies. Those darn facts and physical theories just so gum up the arguing of a favored story line. Definitive falsification: a real bugaboo among the Humanati.
An interesting side-light of 363 CE is that the earthquake (May 18,19) occurred just before the June death of Emperor Julian the Classicist. Maybe the rubble avalanche was to preserve evidence for the validity of his view, that the Jesus legend was nonsense.
Back in 2012, I worked on the preserved wood rostrum of the ram of a Roman warship recovered from the floor of the Mediterranean, just off the shore of Messina. The wood carbon dated to 277±83 BCE, just about the time of the first naval battle of the First Punic War.
The wood was preserved because the ram was buried. The burial site became anoxic. Sulfate-reducing bacteria produced H2S, which suppressed woodworms and aerobes, bringing the ram and its wood into the 21st century.
Like you, I’m strictly materialist, Kevin, and so can’t say that the ram was buried because the fates wanted archaeologically important wood to be preserved for later recovery. 🙂
E=MC^2, but 100 scientists disagreed and wrote a scathing book all about it why it was so ridiculous.
Using your logic, you would have gladly signed your name to their book as well.
“There are a *lot* of mainstream climatologists and very few climate skeptics,…”
There are actually very few “Climatologists” with advanced degrees in climatology. Most have their degrees in physics, geology, chemistry, oceanography, meteorology, astronomy, etc. That’s also true of most CAGW skeptics. AFAIK “climatology” as a specific field only emerged since the climate scare activist movement started with the “Ice Age Cometh” hype of the late seventies. Reid Bryson (Meteorologist) and Stephen (‘double ethical bind’) Schneider (Mechanical Engineering & Plasma Physics) were proponents. Bryson went on to acknowledge his error and became a vocal global warming skeptic. Schneider embarrassed himself by endorsing scientists lying to be effective activists.
In fact, advanced degreed climatologists mostly entered the field because they already believed there was a problem and there was a lot of funding flowing into newly created university climate science departments. I’d bet that the vast majority of “Climatology” PhD theses are related to global climate change alarmism.
ad-hominem. My degree and phd are in electronic engineering but I teach computer science and my research is essentially statistics (machine learning). Most research is very interdisciplinary these days and nobody pays much attention to your degrees once your research is established.
And yet any time a scientist makes a remark resembling skepticism, alarmists shout at the top of their lungs: “He’s not even a climate scientist.”
Apparently scientists are allowed to opine if their opinion is climate alarm, but not otherwise.
That’s interesting. My degree is in mathematics (emphasis on probability & statistics) by my career is is Mechanical Engineering with a PE license. So we can agree that one does not need a “Climatology” degree to understand climate. In that case, I can show you a list of over 30,000 scientists and engineers who do not think climate change is a major issue requiring drastic action.
Those who research climate probably have a better grasp of the subject though than those that don’t. As I said the degree doesn’t matter much once you have established your research.
“research climate probably have a better grasp of the subject though than those that don’t.”
Climate modelers do not research the climate. They play video-games decorated with mathematics.
If you mean the Oregon Petition,
> 30,000 is a tiny percentage of the targetted constituency, which includes nurses, vetinarians and chiropracters.
> It was distributed with a tricked up fake research study designed to look like a PNAS paper. The NAS put out a PR disassociating themselves from the fakery.
https://en.wikipedia.org/wiki/Oregon_Petition
“…There are a *lot* of mainstream climatologists and very few climate skeptics…”
Let me fix this statement:
“There are a lot of mainstream climatologists who have given up the scientific process for activism and a lot of mainstream skeptics still practicing science as well…”
There, much better.
Are you saying there are no natural causes for the rise of CO2?
That’s exactly what he said. That means that the evolution of human beings and their associated behaviors are either un-natural or that divine intervention was the cause.
There is no other explanation.
Are you sure it’s not a question of “…the problem infects the side you prefer to take too…”?
All in all, I think this “article” throws the baby out with the bath water. Without peer review, we get unproven, unapproved, experimental genome manipulators masquerading as “vaccines” for diseases we are not sure actually exist, like covidiocy.
There was a time when peer review was a matter of pride; scientists were proud to exhibit their brainchildren before the world, daring anyone to count the toes and find too few. Then we privatised education, turned it into a business model. Like everything else in life, you never get what you paid for, you get what the seller feels he got paid for. In education, science and leadership (which has been outcontracted to ‘advisors’ and ‘think tanks’), we are receiving what we paid for, the half-assed plastic crap made by communists. All new and shiny, but the quality is purposely kept low, apparently throwaway rubbish is what “grows the economy”.
Anyone here going to argue that climastrology/ covidiocy is not moving a hell of a lot of money?
Declaring that the other side isn’t perfect, is not a defense recognized by formal logic.
Neither side is perfect. However pal-review serious enough to shut down a journal or to result in changes to the review process is very unusual. As I said in another post, peer review is only a basic sanity check. It is fairly reliable provided that all parties take it seriously. The alternative (sling everything on Arxiv has its problems as well, for instance how can you identify the good paper in the deluge?).
All scientists are subject to bias and online critique is the best thing that ever happened to science.
Now we can all read the paper, read the critique and make our own assessment of the arguments on both sides. Many, many times activist “scientists” have been outed this way.
People like McIntyre and Lewis are just gold on that front. Activists can complain about them all they like but can they find flaws in their work?
Alarmist scientists refuse to review work that undermines their agenda because they don’t want to be put in the position to accept how wrong they are. Scientists who are skeptical that the IPCC has more legitimacy than the scientific method don’t have this problem because they don’t fear being wrong as legitimate science will figure it out.
.
That doesn’t make a great deal of sense to me. If they don’t review it someone else will, and they may not be expert enough to spot the flaws.
Dikran.
You making an argument,
where and when you know, and I accept it, that it couldn’t be wrong subjectively.
But objectively, as far as I can tell, reality shows that you could be very very wrong.
How does this sounds this far to you?
Does it make any sense?
cheers.
“where and when you know, and I accept it, that it couldn’t be wrong subjectively.”
“Does it make any sense?”
no, sorry, I have no idea what you are trying to say.
If someone wanted to scupper a paper they would actively want to review it so that they could criticise it. The only reason to refuse to review it was because they felt a conflict of interest, which would be to their credit.
Does it make any sense to you, that subjectivily a hamer is a tool, but objectively in reality is a weapon too?
Does it make sense too you that the democracy is all good subjectively, but objectively in reality this is not always the case, especially when considering Democratic Repblics like that of Congo or North Corea?
Does it makes sense to you that ‘status quo’ is an unavoidable condition in society, which subjectively is neither good or bad, but moore like neutral,
but subsequently as reality has shown and keeps showing, it can end up to be more than a “tool”, and weaponized as a mechanism of tyranny and intended oppression?
Where and when societal paradigms become tyrannical and ideological, with very high costs to society.
Does any of this make sense to you?
The status quo even when not much appreciated, still if within the range of overall productivity and positivity objectively, is endurable,
but not the same can be said if it expands to tyranny and ideology, and ends up as a weaponized mechanism to dictate paradigms up to the point of vilifying and openly punishing challenge, with no any regard or remorse.
cheers
Sorry, I still have no idea what you are trying to say.
Ok, that is fine.
Just for a further clarification.
I agree with your attempt of defending peer review as a mechanism or a tool, but only in the
context of subjectivity,
as in reality, objectively the case shows that it has become a weaponized mechanism of suppressing and oppression towards challenges and challengers of the given status quo scientific paradigms.
Peer review is an adjudication process,
which subjectively and objectively should considerably show a considerable degree of just… justice.
Simply hiding behind the veil of fairness does not cut it… or excuse the malfunctioning or the tyranny there…
it simply, that makes it worse.
cheers
In overall, that process, while in concept and “paper” sounds all good,
and also expected to be a helpful tool in application subjectively and objectively,
still lately it has become clear that it has ended up as void of justice,
decaying, highly corrupted and wholly decadent.
cheers
Ah, I think I understand. I don’t agree though. Many climate skeptic scientists seem to have no more problem getting published than I do. There are plenty of obviously wrong skeptic papers (e.g. Douglass et al) in climate journals that would never have made it past review if it were that hostile.
The trouble is that authors are never happy when their papers are rejected.
Fair enough, if one keeps with the agreeing to disagree.
But the point made is not about which side is more whip orientated,
It is about the very core of the process as applicable in reality.
Respect your position taken in the point of our conversation,
but still not accepted as right or just as it stands.
Thank you for your effort.
There is no real enemies unless we invent them.
In my understanding what you trying a defend is a process which has ended up in a way or another keep inventing enemies…. unjustly.
But hey, that is me and my view, which you are not demanded or required to accept.
Thanks again.
cheers
Let me get another line of approach there, in the context or prospect of your education and your expertise.
It is said that when Tesla asked what electricity is,
the answer from Tesla, was something like;
“No body knows, or no one knows.”
So, as you learned, educated in the electronics, what will your point of address can be there.
Was Tesla wrong or right with his offered answer,
where and when both of us know that Tesla no any near or close to either yours or my knowledge or understanding of electronics.
So what is your position there?
cheers
Ok, it is a reply intended to you Dikran.
🙂
cheers
Knowledge isn’t binary. Not knowing everything doesn’t mean you know nothing.
Even inTesla’s time.
In consideration of your education in electronics which Tesla had not.
Technichally from first line of approach Tesla can be considered wrong.
Simple definition of electricity;
“Another form of energy like, thermal or radiative or magnetic.”
Simple as that in the proposition of large print.
But nevertheless in small detailed print is another matter entirely, which in proposition of electronics, it gets paradoxical.
Would you not agree with this one?
cheers
Marsupial,
What about climate science does make sense? It’s broken science supporting an evil Marxist agenda reinforced by a complicit media that repeats lies, fails to report relevant information, makes things up and hyperventilates about meaningless crap. BTW, you can replace science with politics and every word I’ve said is still true which is why politics must keep its nose out of science.
Who was it that emailed to other “scientists” about a new climate paper –
“we’ll keep it out (of publication) even if we have to redefine what peer review is”?
Yes, that is pretty standard British hyperbole-as-humor. Did they keep it out? No.
You must not be very familiar with British humor, in that case. Hyperbole is the stock-in-trade of American humor, understatement is British. I have read the comment in context, and it was a serious plan of action. Really, as they were proposing it act in concert, the very definition of conspiracy.
I am, I am British. British humour uses both hyperbole “I’ll eat my hat and swallow the buckle. Whole” and understatement “well I’ll go to the foot of our stairs”. It is a bit of a trope that British humour is just about understatement.
Would you prefer scientists to include papers they thought were fundamentally wrong in an important document? I wouldn’t.
If they thought a paper was fundamentally wrong, doesn’t scientific rigor require them to publish a counter-case with supporting observations, rather than just using “the system” to black-hole someone else’s findings?
This has been what is fundamentally wrong with “climate science” since the get-go – a cabal of “insiders” who conspire to control the narrative, research and of course the grants $$$s.
“If they thought a paper was fundamentally wrong, doesn’t scientific rigor require them to publish a counter-case with supporting observations, rather than just using “the system” to black-hole someone else’s findings?”
no, the usual fate of bad papers is that they get ignored – nobody cites them. If scientists had to publish counter-cases for every bad paper they saw, senior researchers would have no time to do anything else. Peer review is just a sanity check. Most papers have flaws, which is why you shouldn’t look at just a handful of papers, but consider a wide range.
I’ve written a fair few comments papers, and there is no reward for doing so and it is a lot of work. One of mine was a comment on a paper suggesting the rise in co2 is natural, which is very obviously not the case. So I wrote a comment paper explaining the basics. Has it changed any bodies mind? No. Extreme skeptics are still trying to prove that the rise is natural, even though it is refuted by multiple lines of evidence, and those involved refuse to discuss the arguments in the paper.
one of the other comments allowed me to publish a paper about dinosaurs though, which was nice.
Correct me if I’m wrong (I know you’ll try), but I understood that manmade atmospheric CO2 contribution to total content was ~ 3%.
And that outgassing from the oceans was responsible for most of the CO2 that goes into the atmosphere.
So how did nature take a back seat in all this, and leave it all to mankind?
This is explained in my journal paper
https://pubs.acs.org/doi/abs/10.1021/ef200914u
there is a freely available pre-print here:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.713.6770&rep=rep1&type=pdf
I’m about finished commenting here for a while as the thread on the carbon cycle seem to have fizzled out, but if you have any questions, do feel free to send me an email (the address is on the pre-print). Alternatively, ask Ferdinand Engelbeen, who clearly has an excellent understanding of this topic (ISTR he wrote a series of blog posts about it for this blog, which I would recommend)
Thanks for the links.
I was getting into your paper with interest, right up until page 7 where you jumped the shark by referencing “ocean acidification”
I’m sorry, that nonsense term just shuts down any appreciation for me that I’m reading any kind of serious scientific offering.
If the authors and users of this term can’t even get their heads around the pH Scale, I conclude that any other insights they’re offering are probably tosh (as distinct from the term you use in your paper – “beyond reasonable doubt“)
”I was getting into your paper with interest, right up until page 7 where you jumped the shark by referencing “ocean acidification”
Yes. Any drop in alkalinity observed in the ocean as a result of co2 is transitory at best. All the solid carbonates will first need to be dissolved. Hasn’t happened yet and won’t happen.
https://postimg.cc/4KxwL9hn
”Extreme skeptics are still trying to prove that the rise is natural, even though it is refuted by multiple lines of evidence, and those involved refuse to discuss the arguments in the paper.”
Oh what a load of garbage. Circumstantial ”evidence” is not a refutation.
What we ”extreme” sceptics maintain (without refutation) is that we accept that hypothetically, there may be some increase due to anthropogenic co2 emissions (yet to be demonstrated) and that is a far as your ”evidence” stretches.
“ Would you prefer scientists to include papers they thought were fundamentally wrong in an important document? I wouldn’t.”
Would you allow Michael Mann’s ?
No it wasn’t humor, he was being serious about it.
Sorry Dikran, you are completely wrong here.
I have first hand evidence that Jones and his mates corrupted the peer review process by excluding papers they didn’t like. Please don’t suggest that those comments in the Climategate emails were somehow in humour and not really true.
As I say, I have first hand independent evidence that they were colluding and deciding a priori to prevent publication of papers that contradicted their own published body of work. I have a copy of a paper I was asked to review independently as it was having trouble being published, as I have expertise in geostatistics (which Phil Jones and his cronies do not). This was 2005. The paper was never published because Phil and his cohort did not want a paper published that would contradict their own paper. The exchange turned up in the emails in Climategate some 4 years later – imagine my surprise when I found it being specifically referred to.
It was Phil Jones. As both the author and recipient were well aware that neither actually had the ability to ‘redefine peer-review’, the remark was obviously not meant to be taken literally.
In the event, both of the papers in question, (McKitrick and Michaels (2004) and Kalnay and Cai (2003)) were included in the IPCC report, since then, both pretty much sank without leaving even a ripple.
The end of the statement not being serious doesn’t lessen the seriousness of the statement.
Someone arguing that they’ll “allow school integration over their dead body” is no less serious because they don’t actually intend to die.
The fact scientists were discussing trying to block the publication of works with which they disagreed is absolutely abhorrent, as was their discussion of obstructing FOIA requests.
I find it fascinating that these two are actually still pretending that “climategate” never happened and that Phil Jones et. al. weren’t attempting a conspiracy to further their “cause”. It’s not surprising that at least one of the two is British … and a warmunist.
“The fact scientists were discussing trying to block the publication of works with which they disagreed is absolutely abhorrent”
Nobody was trying to block publication. The studies were already published; they were discussing which papers should be discussed in an IPCC Assessment report, with Jones arguing these two in particular did not deserve inclusion.
In the end they were included, and judging by their trajectories since, Phil Jones was on the money.
John Phillips. Not true. And I have first hand knowledge of a paper that WAS blocked by them because it went against their own published work. And they discuss it in the Climategate emails, something I discovered afterwards. It happened. Suggesting they didn’t do this is is simply absurd.
Not the old “taken out of context” deflection?
Nobody with a modicum of literary comprehension could read the entire volume of the Climategate emails and not conclude these “scientists” were caught in flagrante delicto.
You believe they were exonerated multiple times because the investigators simply failed to understand the material?
Novel interpretation.
“If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him”
-Cardinal Richelieu.
Examine several thousand of any groups’ private emails in bad faith and you will be sure to find something that fits your confirmation bias.
Full disclosure –
I was a disinterested registrant of climate prognostications until I read the entire volumes of the Climategate emails.
In the corporate finance world I worked in, such exchanges as these would have had executives and directors fired and expelled, if not being led away in handcuffs.
And then the ‘investigations’ topped it off for me – obviously the JFK Warren Commission was the template for these.
Both invoked the classic Profumo defence –
“they would say that, wouldn’t they?”
2 of your examples is about the same group and the third example shut down a year after it began.
You are grasping for straws here.
No. There are only two examples. I put a footnote providing a non-paywalled version for the second example.
It was the first example that was shut down a year after it began, and the blatant pal review was the reason for it being shut down, that was the point. The journal involved in the second example is still running AFAIK.
HIlarious, a downvote for pointing out that I gave two examples not three?
The downvoting of factually correct posts correcting someone’s error does Reuther give the impression that I am intruding in an echo chamber where dissenting voices are not welcome. Is that really the impression you want to give? ;o)
Gee, thank you for telling us about your fragile ego.
You have been civil and interesting here, but your comments are not popular to many for various reasons, but they have been civil with you the whole time in their replies.
This place isn’t an echo chamber because there are long episodes of skeptics arguing with each other in many threads.
Well, I’m struggling to understand how come that only human CO2 is rising, and why the rise in CO2 levels now is dangerous compared to the rises in CO2 levels in the past..
Replying to yourself multiple times is a major kook-sign.
You posted THREE links, while I stated this:
You have a problem here with logic.
links and examples are not the same thing. If you want to talk of fragile egos, why not simply accept you made a minor error.
It is not my ego that is reacting to the downvotes by the way; it is my sense of humour – it is genuinely amusing that people are so absurdly partisan here ;o)
You are actually suggesting this is a minor error?
Yes, confusing a link for an example is a minor error, it isn’t a big deal, I don’t see why you are making such a fuss rather than just acknowledge it.
The reason you get so many down votes is that you are generally wrong and supporting an issue well understood to be scientific fraud. As for your “sense of humour”, so far there has been none detected.
Says Michael Mann 😉
You’re going to have to do better than that. Pattern Recognition in Physics has nothing to do with Covid or climate change. Of course, it’s Wikepedia. Wikepedia content is censored by the gatekeepers.
So whatever great insight you think you found, you haven’t shown your point with the links you gave.
Well, I completely misstated that. Yes, Pattern Recognition in Physics dealt with climate change. But you can’t be serious with a Wikepedia article as your source.
Of course you can find cranks and crackpot science opposing the climate change hysteria. Same as the side supporting it — lots of junk science out there.
Kudos for admitting your error.
The Wikipedia thing is a bit weak though as it gives likes to the primary information.
I don’t really like the term junk science, it encourages people not to engage with the content. The two articles on WUWT in the last week or so arguing the rise in CO2 is natural (which is why I popped in in the first place) are every bit as obviously incorrect as anything in Pattern Recognition in Physics.
“The Wikipedia thing is a bit weak though as it gives likes to the primary information.”
Oh that’s freaking beautiful! People like you that use Wikipedia never actually have looked at the bibliography.
References not found in citation, dead links, citations drawn from media sources, and my favorite- citations to books, not available to read online nor in print. This is every article in Wiki. Every, single, one.
Here’s as random an article I can do:
https://en.wikipedia.org/wiki/Ford_small_block_engine#AR02
Citation #3 nothing, #5 nothing #6 nothing #7 a book! can’t read #9 self links the Wiki page #10 dead
6 of 11 citations are dead ends.
I haven’t even bothered checking the 5 citations that actually went somewhere. What may be the odds they do not contain the info they are being referenced in the article? More than likely.
You are a joke! And Wikipedia is an embarrassment for those that use it.
Scientists need to stop conflating being published in a refereed journal with peer review. By that standard none of Einstein’s theories were peer reviewed. The reality is that peer review can take many forms and so in order to break free of the limitations of journals we need to openly recognize that. As long as journals are the only forum that’s spoken of as providing peer review they will continue to have an oversized influence on science.
Publication in a refereed journal is peer review.
I have been on both sides of that fence. It serves a useful editorial purpose, weeds out total crock, maybe checks the maths but that is about all.
It’s open to abuse by gatekeepers. The worst problem though is the assumption that peer review confers scientific rigour on the results, as beloved by politicians, activists and the BBC. That assumption is both absurd and dangerous.
That kind of science, as Feynman said, is the belief in the ignorance of experts.
Well, this is helpful. 100% exactly correct. Saved me the trouble, and said it better than I could.
I think a lot of the consternation surrounding peer review is due to a misunderstanding as to what it is and what it is not.
“It serves a useful editorial purpose, weeds out total crock, maybe checks the maths but that is about all.”
Exactly. Do not expect more than this.
And then this:
“It’s open to abuse by gatekeepers.”
As we have all seen a million times in climate science. But more, even in non-controversial areas, we have situations develop. One example is where a “leading researcher” in an area will shoot down the papers of young up-and-coming researchers until they wise up and pay homage to the “Grand Old Man” of the field.
Petty, petty, petty.
“Publication in a refereed journal is peer review.”
I know, my point is that it is just one form of peer review, it is not the one and only method. Any time a scientist makes their work available to relevant experts who provide feedback that constitutes peer review. For instance, if someone writes a book with their claims and other scientists write responses on blogs, that would constitute peer review. In some ways that’s much better than a journal because it is all very public and transparent, it can involve far more people, and there is often real discussion that digs deep into the relevant issues.
People will respond that this can’t be real peer review because there’s no easy, clear way to know if the responses are mostly positive or negative. But that’s the nature of science. Science is an extremely messy, uncertain process of many different people across the globe trying to work through what claims are right and what wrong, with many, many disagreements that take a lot of time and effort to work through. Using refereed journals seems like a nice, clean way to evaluate claims because it provides a clear thumbs up/thumbs down based on whether the paper was published or not. But such an approach natural leads to the conclusion that whether or not a paper has been published in a journal determines whether it is legitimate or not.
Such a simplistic attitude is not good science and we need to break it. Peer review can not be a pass/fail thing determined by a handful of people at a journal. Refereed journals must be seen as just one part of peer review, not the gold standard or only method. Doing so will mean that it will be harder to declare if something has been truly “peer reviewed”, but that’s a good thing because there will be less of an illusion that any particular claim has been fully vetted.
Going through references can eat up loads and loads of reviewer time (assuming they can all be located).
The entire field of CliSciFi has been corrupted by socialist ideology, politics and crony capitalism. Every special interest group has jumped on the bandwagon. Even the race hustlers are using it.
Any time someone stands on a stage to say that climate change is an existential threat and is not hooted off is proof CliSciFi is driven by massive amounts of propaganda and ginned up political and social pressure. Other than some minor warming and wetting since the end of the Little Ice Age, there has been no change in any climate metric detected on a global basis.
Yes, if I have to read ‘peer review is the gold standard of science’ one more time, then I may have to run screaming down the street. Except, I have a load of papers to review and don’t have the time for running and screaming.
A lot of journals are starved for qualified reviewers, who have to take time away from other more important tasks, without compensation.
It maybe that the best and most qualified are too busy or not interested in peer review, which is not good. Though sometimes new ideas themselves may come out of the review of a paper. Something in the paper may spark a thought in the reviewers mind. I think new ideas often come out of in depth review of other ideas.
Worse, the reviewer sees an idea about to go to print that they have not had time to write up yet so….kill the paper.
Or spot a good idea off the main point of the paper, kill the paper and use the good idea yourself.
That’s why anonymous review is bad. And it does go on.
Free reviews means more profit for the publishing journal. They should pay their way and not expect to be supported by academics who might be motivated by other than money in return for a service.
Including the editors also, I think.
I remember the last couple times I submitted to an IEEE journal: part of the process was requiring the author submit a list of proposed reviewers (who end up in the master list of people kept by the editor(s) to be hit up for future reviews). This always struck me as leaning toward pal-review.
Indeed, it is something journals should not do. The action editors ought to be sufficiently familiar with the sub field to identify reviewers for themselves. This was exactly the issue with the example I gave earlier involving Harde, who listed five potential reviewers, none of which had any expertise on the topic, all of which shared his theory. The journal stopped asking for suggested reviewers as a result of the failure of the review process.
This is bigger than: Does Peer Review work? Come on. Are we children? What country/planet do we live on?
Peer review and science do not ‘work’ (actually they do not exist) in a country that has been ‘Institutionally’ corrupted/taken over.
Obviously…This is not random corruption. This is planned, organized corruption. There is an agenda… That stays/stayed the same.
The CCP wants/wanted our country to do stupid country damaging activities. Their plan is sort of sabotage.
It is a fact that there has been trillions and trillions of dollars spent to finance the take over of our country and its institutions. Large companies are now controlled. The internet is controlled by controlling google, facebook, and so on.
News outlets have become propaganda outlets … Who all push the same agenda.
The content of movies is now controlled by controlling the large companies. Schools are controlled. This is the very end of the process.
The corruption was so effective because it was financed and is part of a cold war. Each year as we lost control… The corruption and organized stupidity (doing country damaging stuff with propaganda to hide the country damaging facts) became deeper. CIA, FBI, CDC, and so on are onside/controlled.
Twenty years ago we could have had a talk about ‘peer review’. That ship has sailed
In China for example …. Scientists, Journal editors, news companies, …. Everyone does as they are told.
CAGW was never justified based on observations, analysis, or logic. The Solution to CAGW does not work. Why is it still pushed?
We are in an economic crisis, at the time of loss of control. And there is zero discussion about the country ending unsustainable overspending. Why are we spending money/destroying jobs which we do not have on a solution that does not work?
It is a fact, that ‘CAGW’ is a sneaky plan to spend ourselves to death and destroy our government.
The weird things that are being done in the US is because of the US – China cold war. US is obvious losing. China now has 53% of the world coal fired power plants and China is modernizing their coal plants to reduce costs.
Our government/the CDC is so stinky corrupt they are trying to hide the fact that covid was manmade.
The Lancet ‘letter’ was written by Gain of Function researchers. The ‘letter’ included multiple false statements to support their incorrect ‘conclusion’ … that covid was natural. Come on. The covid spike has never been seen in nature.
The covid spike is what Gain of Function researchers use .. when they make deadly contagious viruses that are not found in nature.
The head of the Wuhan lab a year before covid was released… Bragged about a deadly bird flu that, the Wuhan lab had created… that did not exist in nature and for which there was of course no vaccine for.
Even so-called professional societies like the IEEE have swallowed the green climate bait hook-line-and-sinker, the 1.5C Paris lie is accepted as truth without question.
And now there is a video out of one of these virus lab clowns talking about positive financial possibilities of prepandemic market investing, back in 2018.
The signatories of that Lancet letter, which included Peter Daszak, claimed to have no conflict of interest. Remarkable.
You don’t meet with your boss at least once a year and defend your “service?”
Not since I hung up the shingle and retired. As a non-hourly government lab rat, I never had to go into this level of time scrutiny.
Peer review is only the first step to acceptance by the research community, it certainly isn’t the last, and never has been. It is a fairly reliable sanity check, nothing more. Scientists generally know that.
Why did the 1998 “hockey stick” paper survive peer review when it was a badly flawed mathematical construct?
Because the supposed flaws were addressed in subsequent papers and found to fall into two categories:
-Those that had some merit but did not significantly affect the conclusions e.g. non-centred PCA
-Those that might have altered the conclusions, but were without merit e.g. proxy selection.
But you’re fighting a two-decades old battle, the ‘hockey stick’ is now established science and has been for the best part of 20 years, that is not going to change.
unless someone has compelling evidence.
Indeed, I was expressing an opinion that it is highly unlikely that such evidence will come to light after two decades. It is not as if people have not been looking. Indeed, I cannot offhand think of a study that has received the same level of scrutiny. Apart from several papers in the literature and exhaustive analysis on the internet, there was an NRC panel set up, the Wegman report and a House Committee hearing. Not exactly usual for a paleoclimatic reconstruction.
All comparable reconstructions published since fall within the error bars of the original MBH studies, not least Mann et al 2008 – and we now have PAGES2K, which between the two of them made use of pretty much every known proxy. If course, it is always possible that new contradictory data might emerge that contradicts this body of evidence, but it is hard to conceive where it might be hiding.
A lie doesn’t make it true. Shame on you for scamming people
If you actually believe what you have written you need to spend more time at Climate Audit reading Mcintyre’s back catalogue.
All the subsequent paleo-reconstructions from The Team suffer essentially from the same problems. Mcintyre has endlessly and tirelessly demonstrated this in methodical detail, but The Team is like the Terminator and just will not stop.
Also I still find it astonishing how otherwise intelligent people can continue to make a defence for uncentred variance calculations as being a valid approach. The very concept offends. Statistics was built on very sound foundations and uncentred variances was not one of them.
“If you actually believe what you have written you need to spend more time at Climate Audit reading Mcintyre’s back catalogue.
All the subsequent paleo-reconstructions from The Team suffer essentially from the same problems. Mcintyre has endlessly and tirelessly demonstrated this in methodical detail, but The Team is like the Terminator and just will not stop.”
I have read enough McIntyre to understand his agenda and methods (exaggeration, errors and missing context mainly). And that claim is not actually true. McIntyre asserted that non-centred PCA biased the original curve. This had some merit, but the effect was negigible (missing context) and did not affect the conclusions. More recent reconstructions use methods other than PCA. McIntyre asserted the hockey stick got its shape from undue reliance on a particular tree ring series. This was never actually true (error), and again, the updated study (Mann et al 2008) found modern warmth is unprecedented whether or not any tree rings are used. And so on and so forth, in short all of the criticisms McIntyre and McKitrick made in the literature were examined and found wanting. If he wants to be taken seriously he needs to leave the safe space of his blog and actually submit something for publication. Until then ….
John, simply repeating the same old whitewash and deflection (“these are not the droids….”) is simply laughable for those of us who can actually read those papers and are able to decide for ourselves.
Here’s my prediction: history will not be kind to Dr Mann and his body of work (except possibly on the AMO – which ironically he now says is wrong!).
Also the idea that McIntyre “has an agenda” is, and always has been, breathtaking misdirection. If any one in this sorry saga has an agenda it’s Dr Mann and his various lawsuits and offensive rhetoric designed to close down discussion.
I just hope Mark Steyn manages to get Dr Mann on the witness stand……
Pages 2k does not have a single example of a valid proxy study from mainland Australia. Some minor, flawed ones offshore from Tasmania, calibrated from defective temperature records, then off to NZ for some dendrochronology etc. Then, some locations from the northern hemisphere were thrown into the sth hemisphere reconstructions. It is weird. Why try to make it seem the hemispheric part of a global study when it means next to nothing for coverage or significance?
Geoff S
Oh, you mean like statistics?
I have an open mind, evidence from any relevant field must be considered.
However, as a statistician I am more convinced by physics than statistics.
Little wonder. If you were to actually look at the statistics showing a result that is largely influenced by a single tree you might be less convinced.
What kind of warm fuzzy feeling does that leave you with as regards the authors? their motives? peer reviewers?
“that is not going to change”
Translation to English- it’s now dogma. Thus, nobody should waste any time looking at it again- it’s scripture. By one of the saints, no less. Nothing to see here- move on.
Laughable. The Hockey Stick is NOT established science – it is contradicted by an overwhelming number of paleo-climate studies done worldwide and is even contradicted by written history.
Repeating a lie doesn’t make it true.
No it was exposed as statistical garbage by the Wegman report and the North report.
It was also just for NORTHERN Hemisphere using a very rare tree in a rare climatological region.
It also contradicts a lot of written history as well.
Refreshing your memory holes:
There He Goes Again: Mann Claims His Hockey Stick was Given “Clean Bill of Health”
Mann has been repeating this arrogant duplicitous spin continuously since Climategate and refuses to acknowledge any problems whatsoever with his infamous doomsday hockey stick graph. Mann always refers to the subtly worded US National Academy of Sciences (NAS) report as his ally because he knows McIntyre & McKitrick, the Wegman Report, Hans von Storch, et al, and now the Head of the Royal Statistical Society have minced no words debunking his hockey stick. But what did the NAS report and the authors actually say about the Mann hockey stick? In fact, the NAS report validated all of the significant criticisms of McIntyre & McKitrick (M&M):
1. The NAS indicated that the hockey stick method systematically underestimated the uncertainties in the data (p. 107).
2. In subtle wording, the NAS agreed with the M&M assertion that the hockey stick had no statistical significance, and was no more informative about the distant past than a table of random numbers. The NAS found that Mann’s methods had no validation (CE) skill significantly different from zero. In the past, however, it has always been claimed that the method has a significant nonzero validation skill. Methods without a validation skill are usually considered useless. Mann’s data set does not have enough information to verify its ‘skill’ at resolving the past, and has such wide uncertainty bounds as to be no better than the simple mean of the data (p. 91). M&M said that the appearance of significance was created by ignoring all but one type of test score, thereby failing to quantify all the relevant uncertainties. The NAS agreed (p. 110), but, again, did so in subtle wording.
3. M&M argued that the hockey stick relied for its shape on the inclusion of a small set of invalid proxy data (called bristlecone, or “strip-bark” records). If they are removed, the conclusion that the 20th century is unusually warm compared to the pre-1450 interval is reversed. Hence the conclusion of unique late 20th century warmth is not robust—in other word it does not hold up under minor variations in data or methods. The NAS panel agreed, saying Mann’s results are “strongly dependent” on the strip-bark data (pp. 106-107), and they went further, warning that strip-bark data should not be used in this type of research (p. 50).
4. The NAS said ” Mann et al. used a type of principal component analysis that tends to bias the shape of the reconstructions”, i.e. produce hockey sticks from baseball statistics, telephone book numbers, and monte carlo random numbers.
5. The NAS said Mann downplayed the “uncertainties of the published reconstructions…Even less confidence can be placed in the original conclusions by Mann et al. (1999) that ‘the 1990s are likely the warmest decade, and 1998 the warmest year, in at least a millennium.’
Mann never mentions that a subsequent House Energy and Commerce Committee report chaired by Edward Wegman totally destroyed the credibility of the ‘hockey stick’ and devastatingly ripped apart Mann’s methodology as ‘bad mathematics’. Furthermore, when Gerald North, the chairman of the NAS panel — which Mann claims ‘vindicated him’ – and panel member Peter Bloomfield who Mann says above came to the opposite conclusions as Prof Hand, were asked at the House Committee hearings whether or not they agreed with Wegman’s harsh criticisms, they said they did:
THE HOCKEY SCHTICK: There He Goes Again: Mann Claims His Hockey Stick was Given “Clean Bill of Health”
======
Did you forget the Wegman and North reports conclusions?
They along with M&M destroyed the absurd paper.
Same tired long-debunked myths. Whoever wrote this is relying on people not actually reading the NAS report.
“The NAS indicated that the hockey stick method systematically underestimated the uncertainties in the data (p. 107).”
That is not on p107 of the pdf I have, however it is the case that the panel found that uncertainties were underestimated in all reconstructions, not just Mann et al. What this author ignores is the impact this has on the conclusions (not much).
“In subtle wording, the NAS agreed with the M&M assertion that the hockey stick had no statistical significance, and was no more informative about the distant past than a table of random numbers.”
So ‘subtle’, I cannot find anything resembling this in the text. I think it may be a reference to the low CE score as reported in Wahl and Amman, while failing to mention their analysis of the relevance of CE (not much).
“M&M argued that the hockey stick relied for its shape on the inclusion of a small set of invalid proxy data (called bristlecone, or “strip-bark” records). If they are removed, the conclusion that the 20th century is unusually warm compared to the pre-1450 interval is reversed. Hence the conclusion of unique late 20th century warmth is not robust—in other word it does not hold up under minor variations in data or methods. The NAS panel agreed, saying Mann’s results are “strongly dependent” on the strip-bark data (pp. 106-107), and they went further, warning that strip-bark data should not be used in this type of research (p. 50).”
The NAS panel did no such thing. In fact the issue was addressed in the Wahl and Ammann (2007) paper referenced above who found that excluding Bristlecones had a minimal effect on the reconstruction, but did reduce the skill…
“The poorer performance of Scenario 3 in relation to Scenario 2, which cannot be attributed to calibration overfitting because the number of proxy regressorsisreduced rather than augmented, suggests that bristlecone/foxtail pine records do possess meaningful climate information at the level of the dominant eigenvector patterns of the global instrumental surface temperature grid.
(Susequent work seems to indicate concern about Bristlecones is unfounded : http://www.pnas.org/content/pnas/early/2009/11/13/0903029106.full.pdf)
And here is the quote from the report’s conclusions that completely refutes the author’s (and Tommy’s) claims…
“The basic conclusion of Mann et al. (1998, 1999) was that the late 20th century warmth in the Northern Hemisphere was unprecedented during at least the last 1,000 years. This conclusion has subsequently been supported by an array of evidence that includes the additional large-scale surface temperature reconstructions and documentation
of the spatial coherence of recent warming described above and also the pronounced changes in a variety of local proxy indicators described in previous chapters”
The NAS said ” Mann et al. used a type of principal component analysis that tends to bias the shape of the reconstructions”, i.e. produce hockey sticks from baseball statistics, telephone book numbers, and monte carlo random numbers.”
This tells you all you need to know about this author’s honesty. Here is the full quote
“As part of their statistical methods, Mann et al. used a type of principal component analysis that tends to bias the shape of the reconstructions. A description of this effect is given in Chapter 9. In practice, this method, though not recommended, does not appear to unduly influence reconstructions of hemispheric mean temperature; reconstructions performed without using principal component analysis are qualitatively similar to the original curves presented by Mann et al”
As I wrote, the PCA criticism has merit but the panel themselves say it does not influence the outcome. In fact whether you use centred PCA, non-centred PCA or no PCA, the result is a hockey stick.
Some highly dishonest selective quotation, some outright lies.
Same old tired myths.
Ignorant comment, Philips. Even the IPCC backed away from the phony Hockey Stick in the forth and subsequent assessments after it was discredited.
No they did not. Mann et al appears alongside other reconstructions in AR4 (Fig 6.10). The criticisms of McIntyre and others are discussed, and, as I stated above, found to be either insiginificant or without basis. (Page 466).
There is only one subsequent IPCC assessment, AR5, which indeed does not mention the MBH98/99 hockey stick study. Not because it was ‘discredited’ but because it was over a decade old and Mann et al had updated it (Mann et al 2008) …
https://www.ipcc.ch/report/ar4/wg1/
https://www.pnas.org/content/105/36/13252
Figure 6.10 shows a number of temperature reconstructions, and the take away is that several newer ones look NOTHING like Mann 1999. There is NO hockey stick in most of those newer reconstructions. What figure 6.10 actually shows is that ALL reconstructions (including MANN) show a MWP, most show a pronounced decline in temperatures during the LIA, and all show modern warming starting in 1850 BEFORE CO2 increased significantly. Where they differ is in the magnitude of the temperature anomalies, as shown by the very large differences in the overlap between the reconstructions in 6.10c.
AR4 did not, in any way, feature the Mann Hockey Stick like AR3 did, in fact, it actually showed that several other temperature reconstructions completely disagree with it. AR4 actually did acknowledge one paper that concluded Mann had errors in his PCA analysis but claimed that the errors were small.
Where in the world do you get that the hockey stick is “established science” out of any of that? Did someone tell you that or did you make it up yourself?
Ah. so you admit the post where you called me ignorant was itself devoid of truth.
Which of the more recent reconstructions falsifies the hockey stick,(ie falls outside the error bounds of that study)? Answer – none.
Now you”re just lying. Many reconstructions falsify the hokey schtick which falsely claims that temperatures were constant through the MWP and LIA, and only began to rise after CO2 significantly increased – falsified on ALL claims and you indeed do show your complete ignorance by claiming that the hokey schtick is “established science”. Your claim that no reconstruction falls outside the error bounds of Mann 1999 is also flat false. Mann’s error bounds are exceeded by several reconstructions, especially during the LIA. So, I take it that no one told you to claim all the falsehoods and you made them up yourself?
“ Mann’s error bounds are exceeded by several reconstructions, especially during the LIA”
Name one.
Esper (updated by Cook). Smart people know that bluffing and blustering in a (feckless) attempt to cover up for a lie doesn’t work. Of course, a smart person wouldn’t have flung the manure that you’ve flung as you’ve only managed to cover yourself in it. Oh, and trying to get out of the hole you’ve dug for yourself by filling the hole with your own crap is just going to end up with you drowning in it.
Quite frankly John any rational person only has to read MM2005 with its Monte Carlo demonstration of the complete fail of the uncentred variance method used by Mann et al to know the hockey stick was dead and buried years ago.
Mcintyre has tirelessly and meticulously dissected the subsequent repetitive nonsense from The Team. Defending the indefensible has become a way of life for climate scientists.
If Mark Steyn ever gets Mann on a witness stand I think he will eviscerate him. People with huge egos take big falls and I still think its coming to Mann one day. In my opinion the history of science is not going to be kind to Dr. Michael Mann.
Single Study Syndrome. Any rational person should expand their reading. MM2005 was examined (eg by David Ritson and also in Von Storch 2005) and their critique found to have a negligible effect.
“Our results, derived in the artificial world of an extended historical climate simulation, indicate therefore that the AHS (Artificial Hockey Stick) does not have a significant impact but leads only to very minor deviations. We suggest, however, that this biased centering should be in future avoided as it may unnecessarily compromise the final result.”
https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2005GL022753
Exactly as I stated above. Any rational person could also read the Deep Climate blog for details on how M&M used inappropriate inputs (persistent, not detrended red noise) and data mined the results.
https://deepclimate.org/2010/11/16/replication-and-due-diligence-wegman-style/
https://deepclimate.org/2010/10/25/the-wegman-report-sees-red-noise/
As for ‘dead and buried’, it has recently been re-re-re-confirmed by the PAGES2k project. Here’s the result (Original hockey stick in blue, PAGES2K in green).
Funny, why don’t you show paleo-recons on the graph from outside The Team?
As for the snipes about 1 paper and the form of red noise, these pathetic arguments are so old you must be a well travelled warrior cut/pasting the same old arguments. You are clueless about my expertise or experience so have to resort to veiled ad homs to try and make a point.
As for the “wrong sort of red noise argument” this excuse from hockey stick defenders is an instant fail. They say “it’s the wrong sort” but fail to say which is the “right sort”. Why? Because its irrelevant, any sort will result in the same problem. It’s classic deflection. If there really was “the right sort of red noise” The Team would have been rushing to show how Manns uncentred method was valid. The fact they haven’t proves my point.
And before you say they don’t have to prove it, that’s on critics, I say when so much depends on it they jolly well ought to prove it – and smile and be cooperative whilst doing it. For Dr Mann purporting to be undertaking work so important as to save the world and humankind from disaster he should be secure enough to be helpful responding to ANY criticism.
Mann’s fragile ego and thin skin will be his downfall. Hubris and self delusion on a grand scale
“As for the “wrong sort of red noise argument” this excuse from hockey stick defenders is an instant fail. They say “it’s the wrong sort” but fail to say which is the “right sort”. Why? Because its irrelevant, any sort will result in the same problem.”
Nope. That exact question has been examined and you are dead wrong. The ‘right sort’ is noise with the same degree of persistence and autocorrelation as the tree ring data you are simulating. McIntyre used noise with an unrealistic degree of autocorrelation which would tend to exaggerate long term trends.
Remember, also, that the claim is that applying the MBH algorithm to random noise ‘nearly always’ produces a hockey stick. Again, nope, and the ones that do occur are tiny compared to the real thing and half trend down at the end and not up. So for display purposes M&M devised a ‘hockey stick index’, sorted on that and retained only the top 1% most upwardly pointing ‘sticks’. It is right there in their code.
https://deepclimate.org/2010/11/16/replication-and-due-diligence-wegman-style/
Here is Dr Mann’s response, from 2005:
https://www.realclimate.org/index.php/archives/2005/01/on-yet-another-false-claim-by-mcintyre-and-mckitrick/
Some myths never die, huh?
And some people will peddle crock and defend the indefensible for ever. And reference RealClimate as though its reliable. Home of Steig, Schmidt and all the rest. That’s Steig who thinks its ethical to anonymously peer review a paper that was showing the error in his own paper (Steig et al 2009)?
Talking of RealClimate, is it closed for the day and you and Dikran thought you would have a day out at a blog which actually gets some traffic for a change? Just like old times eh?
Now you are resorting to dragging in utter irrelevancies.
Do we take it you have no response to the substance of what was said? In short, the claim that the McIntyre and McKitrick MBH algorithm ‘nearly always’ produces a hockey stick shape from trendless red noise is shown to be an artifact of the type of the inappropriate noise model they employed (as far as I know they never specified the autocorrelation parameter, but it cannot have been realistic).
That the code they used to produce display figures mined for the top 1% most ‘sticklike’ results?
“If there really was “the right sort of red noise” The Team would have been rushing to show how Manns uncentred method was valid. The fact they haven’t proves my point.”
The 2005 Realclimate article I quoted cited the comment to Nature which demonstrated that the Hockey Stick is highly significant compared to the expectations from random red noise. You have apparently been lied to. There’s no reason to disbelieve the Realclimate article, but here are two others demonstrating the same thing
https://moyhu.blogspot.com/2011/06/effect-of-selection-in-wegman-report.html
https://deepclimate.org/2010/11/16/replication-and-due-diligence-wegman-style/
Referencing RealClimate in a discussion about Mann vs MM is hardly going to give you credibility.
I disagree that you actually provided any substance. But then opinons vary.
With regards to Steig and it’s relevance on this thread, perhaps I should gently remind you the thread is about peer review.
“Referencing RealClimate in a discussion about Mann vs MM is hardly going to give you credibility.”
“If you actually believe what you have written you need to spend more time at Climate Audit reading Mcintyre’s back catalogue.”
LOL. Sauce for the Goose.
If you actually believe McIntyre’s claims about MBH have not been found to be either insignificant or just wrong, you need to spend more time reading something other than his back catalogue.
Extraordinary linking to Wahl and Amman to justify MBH. You seem to have forgotten that W&A included the r^2 statistics that Mann said he didn’t calculate (but were on the ftp) that vindicated McIntyre. Mann’s work failed his own r^2 and he didn’t report it. W&A confirmed, inadvertently, that it was true it failed r^2.
Some of us have long memories.
You need to read the NAS report again. R2 is a lousy metric to use in the context of climate reconstructions, practically meaningless. Mann was right not to quote it.
“.…However, r2 measures how well some linear function of the predictions matches the data, not how well the predictions themselves perform. The coefficients in that linear function cannot be calculated without knowing the values being predicted, so it is not in itself a useful indication of merit”
(p92 in the pdf).
Classic McIntyre fanbase. Ignore pages of valid criticism and instead focus on, inflate and claim ‘victory’ over a meaningless stat.
“That’s Steig who thinks its ethical to anonymously peer review a paper that was showing the error in his own paper”
Who better? After all McIntyre was happy to be appointed to review Mann’s 2004 submission to Climatic Change. By the way were you aware that before he was unmasked the saintly Auditor was in the habit of logging into online discussions of the Hockey Stick under the fake name ‘Nigel Persaud’ to boost his own work and denigrate Dr Mann?
“I can’t believe that it took you so long to figure this out. My writing style tends to be pretty technical and I made no effort to modify my writing style and made very technical points. If you go back and re-read all of Nigel’s posts, you’ll see some pretty detailed observations.
By Steve McIntyre “
https://scienceblogs.com/deltoid/2005/08/21/climate-audiot2
Presumably these dodgy ‘ethics’ mean we should ignore everything written since then? LOLZ
Hi John,
Well its been lovely talking to you, bit like a trip down memory lane. I’m going to move on now, but before I do a couple of final points:
Firstly you say “Mann was right not to quote it.” referring to r2. Well we can agree to differ on the relevance of r2, but that is not the point about this sorry saga. The problem is not what you imply, it is that Mann claimed he didn’t calculate r2 when his own ftp directory showed he did – but then didn’t report it. Why didn’t he report it? Because it failed that pretty standard test. Now if Mann had reported it but explained why he thought it was not relevant, that’s different. But that’s not what happened.
On Steig being an anonymous reviewer (and after all this is a thread about peer review) about a paper which was demonstrating his own work wrong, your dissembling is quite extraordinary. That was a saga I followed very closely (at a time when I was probably still commenting at RC too). Your presumption that Steig and The Team are “honest actors” in such a role is utterly misplaced.
The one part I specifically remember is where Steig, as anonymous reviewer of the O’Donnell 2011 paper, made a suggestion to change part of the methodology in the paper. This criticism was taken on board. Then when the paper was published Steig publicly criticised the very change that he had pushed for as an anonymous reviewer. If you cannot see what is ethically wrong with that then…well you are past salvation.
Good luck with being an RC and Hockey Team Fanboy. Personally I am still hoping Mark Steyn will get Dr Mann on the witness stand and under oath. As I have said elsewhere in this thread, I don’t think history will be kind to the body of work left behind by Dr Mann (except, ironically, on the AMO now that Mann has disavowed that!).
I will move on to commenting elsewhere, so feel free to get the last word in. I know you RC fan boys like to dissemble forever so fill your boots.
Steig has a different recollection and demonstrates that your version of event is false. To be fair Ryan O’Donnell offered to withdraw the allegations, but I don’t suppose that will prevent you from parrotting them again – all the while accusing others of dissembling.
https://www.realclimate.org/index.php/archives/2011/02/odonnellgate/
On R2, you are relying on hearsay – McIntyre’s reporting on a panel discussion for which there is no transcript. Very unwise given his ethics. It is not me saying R2 is without merit as a measure of reconstruction skill, it is the NRC. One of the characteristics of Mcintyre and his coterie is misdirection, along with lying by omission and elevating irrelevant nitpicks to the status of major flaws. Never mind the salient point, reconstruction skill, here’s my hearsay interpretation of what happened…..
We will indeed have to wait for history’s verdict on Dr Mann’s contribution. However the scientific community have already made their appreciation for 200+ publications, articles and books known. He is a Member of the US National Academy, a Fellow of both the American Meteorological Society and Geophysical Union, the latter awarded him both their Hans Oeschger Medal for ‘outstanding achievement’ and their Climate Communication Prize. Other awards include the Stephen H. Schneider Award for Outstanding Climate Science Communication, the Award for Public Engagement with Science from the American Association for the Advancement of Science and the Tyler Prize for Environmental Achievement. I could go on.
This is not to appeal to authority, just pointing out that a fall from grace would require large numbers of scientists and professional bodies representing scientists at the top of their profession asking for their awards, commendations and medals back. To believe this will happen based on the false claims of a sockpuppeting blogger requires a stretch of the imagination that defies credulity.
I leave you to your delusions.
Hockey Stick graph, MBH1998, launched many investigations that have bolstered the evidence for a MWP.
Because it was an early work on that particular topic and the problems were not well understood *at the time*. As I said it is a sanity check, it might pick up more subtle issues like centering of PCs, but it is more than it is reasonable to expect. That is the way science works, be start of with promising ideas, through criticism they are refined and improved.
It does do a good job of avoiding publication of obviously incorrect ideas, such as *most* of the papers asserting that the rise in CO2 is natural, which are ruled out by the most basic observations. Papers do still get published on that topic though, partly because peer review does fail sometimes, or because the authors use pal-review (c.f. Harde example I have earlier) or because they submit it to a vanity press publication with very lax reviewing standards.
I do find the fixation with a 20 year old paper that has since been replaced by improved methods my multiple independent groups rather odd.
“I do find the fixation with a 20 year old paper that has since been shown to be a fraud, and has never been replicated by any methods, by any of potentially multiple independent groups rather odd.”
There – fixed that for you.
I find it rather odd that it hasn’t been retracted.
Anyhow, Dikran, keep up the good work.
https://en.wikipedia.org/wiki/File:T_comp_61-90.pdf
Lol …Wikipedia 😉
Was Mann dishonest?
“A disgrace to the profession…”
Dikran says:
“I do find the fixation with a 20 year old paper that has since been replaced by improved methods my multiple independent groups rather odd”.
Of course you would. Spoken exactly like a politician. Nothing to see here folks. Move on. Are you Tony Blair writing under a pseudonym?
Gavin, “I do find the fixation with a 20 year old paper that has since been replaced by improved methods my multiple independent groups rather odd.”
There’s no statute of limitation on scientific fraud.
And the rest of the proxy paleo temperature reconstructions are no more than pseudoscience.
Also, here.
Did you know that principle components have no distinct physical meaning, Gavin? Do you think Michael Mann knew that in 1998?
”It is a fairly reliable sanity check”
What does ”fairly reliable” mean? Does it mean usually right? Sometimes right? 50/50?
I’ve never seen any retractions and corrections in the media from the sane scientists you are talking about. Climate science is a mess, very few facts, a lot of conjecture.
Good comments about peer review, by Mark, especially as regards the rapid advance, in some sectors, of information about covid-19. There is another aspect of peer review, and that is the pronounced political weaponization of reports, peer reviewed or not. A good example is the ongoing medias disclaimer that they didn’t follow Trump’s stating that Covid-19 may be associated with the Wuhan Lab, which not following is being justified by saying they ignored everything else Trump said, so this included. In my business, mining exploration, peer review is constant and even intense, because when you drill an expensive test hole it’s either there or it isn’t. In mining exploration, and other forms of geology, the best peer review is to say, let’s go to the field and look directly at the rocks, because they don’t lie.
The peer review process in mining takes on special significance because that world is so filled with promoters of prospects.
Kevin, are you thinking of the Mark Twain comment: A gold mine is a hole in the ground with a fool at the bottom and a liar at the top?
I had to laugh out load here. Lately I have had people dispute the actual source of any Mark Twain quotation I make, so I have begun to avoid these tangential disputes by never mentioning Twain. I hope Twain actually did say this because it is a beauty!
Also, in his book “Roughing It”, Mark Twain describes his own personal misadventures with silver prospecting and mining.
His “peer review” on this matter was provided by his sidekick, who was just as misinformed about this activity as Mark was.
Mr., that’s the book, Roughing It, which clearly established Mark Twain as a Snarky Troll way before his time. However, this book is not for children or sensitive persons.
Peer review has its place; I would like to think that most competent scientists would want to have their opinions checked by reasonable, capable colleagues. But this essay is spot on. Peer review has morphed into pal review and gate-keeping by scientific “elites”.
Perhaps one answer is to publish the names of peer reviewers who signed off on studies that are shown to be wrong or retracted for scientific error. It might make people think twice before approving studies, and might make them choosier about what they decide to review. And maybe, as the essayist suggests, it would help us move away from the peer review system, and the false sense of security it offers, altogether.
Einstein had no peers.
Good point Joseph.
Peer review tends to discourage new discoveries and innovation. It entrenches the status quo.
Galileo and Copernicus also would not have done well in the modern peer review process.
Not only that,
Einstein caused many people, who thought themselves superior to Einstein, desperate to prove Einstein wrong, flailing for decades and often proving their own theories wrong for decades.
Allowing reviewers to hide behind anonymity is a key part of the problem. I never review anonymously. And reviewers comments (with attribution) should be publicly available with the paper.
Check out the Ryan O refereed rebuttal to Steig etc al 2009 to see how bad anonymous review can get. You know, the climate science one where Steig is the anonymous reviewer of the paper that’s going to show his is wrong. How could that possibly be a reasonable basis for review?
I also have first hand knowledge of an example of the corruption of peer review as subsequently revealed by the climategate emails.
As for putting in reviewers names, recalled this one, hadn’t checked on it lately, but they used to put in reviewers for each paper, including reprints. Some others would at least put the total list in the issue.
“The publication of this issue of Tulane Studies in Zoology and Botany (TSZB) marks the end of an era and the passing of a legendary figure in the biology of the southeast region of North America. This is final print issue of TSZB. [Vol. 32 No. 1 (2012)] The print version has finally succumbed to the challenges we have faced sustaining funding for printing and mailing this predominantly exchange-based periodical. With this issue, we are changing the format of TSZB to ‘Open Access’. The contents will be available for download free-of-charge via the Internet….”
Even before the internet there was a movement toward a more centralized system which discriminated against more “locals” such as the many state academies of science. Some not so local went under, Gulf of Mexico Science and Contributions in Marine Science in my realm. Exchanges changed to a sort of necessary bigotry in order to survive in today’s academia. It took the color away and publishing bureaucracies flourished.
Forgot about this result, poorer libraries are stuck. https://wattsupwiththat.com/2019/02/28/huge-pushback-on-science-paywalls-university-of-california-cancels-elsevier-subscription/ Not sure how this has played out.
Peer review is useless unless the review is unbiased and the authors of the reviewed work answer to any flaws that are found.
With regard to climate science, when flaws are identified, and I’ve identified many and made the authors aware, rather than address the errors, they’re either ignored with prejudice or summary dismissed, especially when an agenda is at stake.
I must admit that a peer reviewer did point out a large error in paper I sent for publication long ago. I eventually published that paper, and thanked my reviewer profusely. People have found fraud in scientific papers simply by looking for claims that are not credible (i.e. noise in data that looked suspiciously like noise seen elsewhere)– in other words peer review can actually work.
On the other hand, I have gotten many reviews where the reviewer pointed out all of his important papers I did not cite; my wife gets a lot of reviews of this sort. There are reviewers determined to keep out of print a paper that disputes, and has evidence against, long established doctrine. I have faced some of those. One involving climate science and one involving another topic entirely — the guardians of dogma are everywhere (one of my co-authors referred to them as “Agents of Rome” because the behavior is so ancient).
Lately, I have become a reviewer for a journal of the ASCE (I was only an adjunct in the civil engineering department) because, I think they have so much trouble finding reviewers. The papers are coming in overwhelming numbers, and mostly from overseas. Despite being worthwhile contributions, the English used is very poor making these papers almost unreadable. I spend huge amounts of time making editorial suggestions, checking all statistical claims, proofreading tables and figures, making sure that citations are correct. This is a minimally thorough job.
Peer review is in trouble partially because it suffers the corruption that always accompanies self-interest, but mainly, I think, because so few are willing to take the time to do a thorough job.
Kevin – thanks for “taking the time”!!
Good points. Use of English can ruin an otherwise good paper so I usually try and make some suggestions, but often the journal editor is faced with either largely rewriting whole sections or rejection with a caveat they can resubmit if they tidy up the writing.
The other bugbear is checking the references. I confess I abrogate my responsibilities here and tell the editor I haven’t had time to check them but confirm the ones I expect to see are there. You always check to see if they have referenced any of your papers, of course!
Kevin, thank you for your insight.
Space in a publication is obviously a scarce resource. Outside of the academic literature, this process seems to self-regulate fairly well. A work will be published if a publisher deems the work worth publishing, and it not, the author always has the option to self publish. In either case, there will be non-anonymous public reviews of the work, which will assist the public’s decision in either purchasing the work. I think the issue in the academic literature is the anonymity of the review process, which makes it a convenient and certain means to maintain academic hierarchies and paradigms, and to quash “politically incorrect” research. This is particularly true and important with respect to “big science”, which is typically government funded. Per Eisenhower, government funded science (GFS) should be heavily scrutinized. My opinion is that any time GFS identifies a “problem” and that problem is only “solvable” through the implementation of government coercion, there needs to be open debate on the science. This means that results published in journals that allow anonymous review would not be considered meaningful.
I have had a similar experience trying to review some Chinese papers on remote sensing. They needed to be cleaned up by an editor before asking someone to comment on content or conclusions. I finally refused any more papers.
It becomes a huge waste of reviewers’ time.
When someone peer reviews an article, spending a considerable amount of time with no remuneration, and often receives no public recognition by having one’s name and brief comments attached, is it any wonder scientific publications conceal the actual process?
Actually a lot of academic reviewers probably like anonymity as they can hide behind it whilst sniping and carping about stuff that challenges their pet beliefs
If the research was well funded- why isn’t some of the funding going to the reviewers?
I believe publishers of the leading scientific journals make quite a bit of money having a captive market – university libraries – and being paid by the authors.
I would like each paper to provide a section that includes a criticism of the paper. Many times the researchers themselves are well aware of potential flaws but don’t mention them.
It is called “lying by omission.”
Peer review is a quality assurance (QA) mechanism for scientific publications.
In politicized sciences, it became a tool of censorship. Starting with the climate, more and more sciences have been politicized. The outcome is what we see.
Peer review is broken because it is an unpaid slice of the establishment, with establishment views. With the peer review in my climate paper:
One reviewer said he-she could not pass the paper because it went against the climate consensus. (It was supposed to be ‘open review’, but none of the reviewers would give their names.)
Another asked why the main thesis had no references. I said this was because this section was novel original research, but he-she said that was not good enough. Perhaps I should have invented references.
Another said that plants at high altitude could not be starved of CO2, because the percentage of CO2 in the atmosphere is the same at high altitude as at sea level. I did ask why airlines carry emergency oxygen, but he-she failed the paper anyway.
Another misread insolation for insulation, throughout the paper. And therefore said the paper made absolutely no sense, and failed it.
I would have done better by handing out copies of the paper in a kindergarten, and asking for comments.
Ralph.
I remember doing peer review. My colleagues, my peers, would look over my work and point out where an error might arise. I peer reviewed a colleague; he had gotten a sign wrong. “Gah!” he said “The other time I caught it myself.”
However, the idea of peer review being a guarantee of anything is laughable.
I’ve worked with epidemiologists. Their science deals with data. Data is their lifeblood. If it not repeatable and provable it is not epidemiology. Physicians’ reports of treatment effectiveness are discounted as anecdotal. Such data might show an effect on survivability, but no epidemiologist would say so without controlled tests.
I recall one paper I was joint author on, an alternative way to perform a seismic inversion. So working in a commercial (geophysical contractor) environment.
Reviewer A rejected it on the grounds that it was mathematically invalid to solve the problem the way we did.
Reviewer B rejected it on the grounds that many other papers had already shown the approach before and therefore it wasn’t original enough.
Go figure.
As my colleague drily commented – “you don’t think they work for our competitors do you?”
Fortunately the editor was rational and competent and simply told us to ignore those parts of the review!
Was Charles Darwin peer reviewed? Didn’t his work have many critics? If they had been peer reviewing him, would his work have been published?
Well the Darwin Awards constantly give us observations to support his theory of Natural Selection.
He had so many critics that were it not for his ‘Builldog,’ the book might have passed into obscurity, to the delight of the fundamentalists.
So he might not have passed peer review- at least not right away? Good evidence of the limitation of peer review. It can be tainted by the zeitgeist.
Imagine if this set of ideas (published in NATURE) ever made it into Climate Science?
Pandemic researchers — recruit your own best critics
As researchers rush to find the best ways to quell the COVID-19 crisis, they want to get results out ultra-fast. Preprints — public but unvetted studies — are getting lots of attention. But even their advocates are seeing a problem. To keep up the speed of research and reduce sloppiness, scientists must find ways to build criticism into the process.
Finding ways to prove ourselves wrong is a scientific ideal, but it is rarely scientific practice. Openness to critiques is nowhere near as widespread as researchers like to think. Scientists rarely implement procedures to receive and incorporate pushback. Most formal mechanisms are tied to the peer-review and publishing system. With preprints, the boldest peers will still criticize the work, but only after mistakes are made and, often, widely disseminated.
It is time to adopt a ‘red team’ approach in science that integrates criticism into each step of the research process. A red team is a designated ‘devil’s advocate’ charged to find holes and errors in ongoing work and to challenge dominant assumptions, with the goal of improving project quality.
The scientific process needs to incorporate methods to include ‘severe’ tests that will prove us wrong when we really are wrong.
Pushback on each step of a research project should be recognized as valuable quality control and adherence to scientific values.
With research moving faster than ever, scientists should invest in reducing their own bias and allowing others to transparently evaluate how much pushback their ideas have been subjected to. A scientific claim is as reliable as only the most severe criticism it has been able to withstand.
https://www.nature.com/articles/d41586-020-01392-8
And that would have meant that Nic Lewis would have rejected the Resplandy paper before it ever was published, also ironically, in NATURE:
The Oceans Are Heating Up Faster Than Expected
https://www.scientificamerican.com/article/the-oceans-are-heating-up-faster-than-expected/
Yet, there is the Scientific American article, with no mention of Nic Lewis, with no mention of the retraction… continuing to falsely inform the voting public. However, instead of trying to correct the record, Scientific American has moved on to attacking Steve Koonin (who also ironically is calling for a RED TEAM evaluation of Climate Science).
To assert that Climate Science is an actual science, as it is presently conducted and constituted is a wild stretch. And I believe its corruption has led to the corruption/erosion of other scientific disciplines as well, because if they can get away with it, why not virology also?
It used to be that a retraction in NATURE would end your scientific career… now the only thing that will do that is speaking against the politically determined scientific orthodoxy.
It’s a great example. I pointed this out to the BBC and they claimed it showed that peer review works! Total bozos missed the point.
The Steig et al 2009/Ryan O et al 2011 saga is another good demonstration of the bankruptcy of peer review in climate science
Or using a pronoun that isn’t acceptable to the person being referred to. The inverse of Alex de Tocqueville’s concern about the “Tyranny of the Majority.”
Peer review is a publishing technique. To the extent that all parties employing it have a minimum amount of personal integrity, it can be useful. But Leftists destroy everything they touch. They debase and abuse everything into the dust.
For instance, a handful of Eurotrash Leftists tossed a Nobel Peace Prize to Obama for literally no reason whatsoever. And that was after giving Algore one for a fact-free B movie. They have debased the prize into a trinket Leftists give to Leftists for being Leftist.
Way too much of peer review has been morphed into “Leftist Approved” and “GroupThink Aligned”.
The fundamental issue here is conflation in the public mind that the Nobel Peace Prize somehow carries the same credibility and standing as the Nobel Prize for Chemistry or Physics.
Oh boy, where do we start? First, how many people in the USA have had Covid? Random anti-body testing was squelched as soon as it began, and showed much larger numbers than tested “Cases.” Why?
secondly, how many people died From it instead of With it? Tremendous amount of disinformation has been in the media.
Third, outdoor transmission. Has there ever been any at all? No evidence of same.
Fourth, masks. Did they prevent any transmission whatsoever? Evidence of this? There is none.
Assuming worst-case by the Public Health Wallahs has cost tens of
millions their jobs, hundreds of thousands their businesses, untold suicides/drug overdoses/child and domestic abuse.
History Will regard this as almost entirely self-inflicted.
Wow
“Third, outdoor transmission. Has there ever been any at all? No evidence of same.
Fourth, masks. Did they prevent any transmission whatsoever? Evidence of this? There is none.” Our great (/sarc/) leaders know this, as they demonstrated on Sunday enjoying their BBQ after the G7 pantomime. No masks, no distancing, arms round shoulders, the whole works. and no quarantine for them after flying in or out . The next day, BoJo tells the rest of us that we have to wait yet another month for these restrictions to be lifted.
So-called peer review is less about science than it is about protecting the reputation of the journal that publishes the paper. The real peer review takes place after the article gets published and those in the discipline get a chance to analyze the claims and sometimes even try to replicate the results. If the journal refuses to allow a rebuttal, then the process of the Scientific Method is corrupted.
So we ain’t doing THAT experiment, or we’ll adjust the data or suppress the results…
Re: Covid, it was how everything about it was ‘new’ – as if an infectious disease had never in human history been encountered before.
OK it had a new name and was from an exotic place but, but so what?
Humankind has endured plagues before now.
Harking back to my experience as a livestock farmer, the animals in my charge would be afflicted by ‘pneumonia’ on occasion = an umbrella term for respiratory disease that might the activity of any number of pathogens.
It didn’t matter what the actual bug was called or where it originated (why does such trivia fascinate folks so much these days?) – the remedy was always the same.
For the cows, calves sheep, lambs, hens, chicks or whatever, to get you all out of the jam you gave them:
And you did not think twice about implementing those actions. You did it by instinct.
In fact, it was breakdown of, or lack of attention to, the delivery of those things that more oft than not precipitated the disease outbreak..
Now – lets take that to the Human Population and Covid.
In the fixation on trivia and blind panic, all those things went almost the exact opposite way to what they should…. Panic and indecisiveness ruled the day, and in the UK still does.
What say you ‘Peers‘? Are you to be trusted with the keeping of a pet bunny rabbit or hamster?
(I really didn’t ask that question, did I?)
Peer review is analogous to “expert opinion” in Climate Science and we see where that’s gone today with COVID.
I’d like to see journal articles with an appendix detailing what issues were raised during the peer review and how they were addressed. I used to do nuclear safety calculations for commercial nuclear power plants and our calculations were reviewed by an independent reviewer and the results of the review and response was included in the calculation.The process was a pain in the neck but was extremely valuable.
I have found that “It’s not peer-reviewed” is typically employed as yet another logical fallacy (along with ad-hominem and appeal to authority – guess it’s a subset of that one) use to simply dismiss anything that doesn’t fit the desired narrative without having to engage with what is actually being said.
My brother will only read material written by “climate scientists” who contribute to the UNIPCC report. Said climate scientists must have a degree in the subject. Which almost none of them do.
According to Steve McIntyre, at least in climate science publications, reviewers hardly ever even ask for the data.
Financial reviews might not catch fraud in all cases, but whenever I submit an expense report they always demand the receipts.
The problem with “Peer Review” in fields like Climatology is that they have somehow become “People who agree with me reviews” for one side of the debate, and censorship for views they do not like.
If a paper contains statistics, it needs at least one expert on statistics in the review. If there are physics, then at least one person who is a physicist, If it’s a paper on climatology, then at most a few climatologists in the review and preferably ones that will provide real critiquing. This is how one catches mistakes and improves a paper – by feedback from non-friends who may disagree with the findings.
When you think about the term “peer review” it kind of smacks of the term “Old Boys’ Club,” doesn’t it? I heard the late Dr. Kary Mullis, the Nobel Prize-winning inventor of the PCS testing system, lamenting the fact in an interview that are no “wise old scientists” in academia and research anymore. Our scientific community is broken and MONEY rules the field.
The consensus is made up of “scientists” that when given the opportunity to prove their theory created models that all fail on an epic scale. Literally, the consensus consisted of “experts” that can’t model the observations they claim to be experts on. You can’t get more Orwellian than that.
https://imgur.com/1BNMuTq
https://imgur.com/q4RTtDw
https://imgur.com/iWasmOI
Peer review is valuable and necesary in highly regarded journals. There is hardly a paper that is not improved by it, and many dodgy papers are rejected by it.
But it’s not a universal cure-all. It’s not a guarantee that the paper is perfect. No one claimed it was.
It’s true that peer review is poor at detecting fraud if it’s cleverly done. It is pretty good at detecting fraud which is poorly done. I once submitted a review where I essentially said “Sorry, don’t believe it. The authors are going to have to present far better evidence or explanations to convince me.” The authors were unable to do that, and the paper was rejected. Score one for the good guys.
I’ve called it “the panicdemic” since Day #0.
Peer review protects journals, not science.
Science is doubt and replication, peer review is consensus and acceptance. We need recognition and funding for replication efforts – preferrably by junior researchers.
Peer reviews are group think of morons
Boston Consulting Group has a lot to answer for
The pin the tale on the donkey yellow postit note way of getting consensus has taken over the world
Thanks Charles – good find.
Presumably they do this to warn the reader that the research lacks the safeguards that peer review brings. The problem with that warning is peer review guards against nothing.
waaaaaaaaa.
every paper I’m an author on benefited from peer review.
The only real peer review is replication. Without that, peer review is only a superficial check. If papers are truly presenting unique information that is new knowledge, how do reviewers automatically know enough to judge what they never knew before. They are students, not professors when it comes to reviewing new knowledge.
The best that can be hoped for is that reviewers catch obvious logical and math errors. How often do reviewers question the all too commonly used phrase, “due to climate change”, with no information on direct cause and effect? How does this pass through review? How many studies show the process in sufficient detail to allow a reviewer to judge if p-hacking has occured?
As I said, replication is the only true validation of a study. Tax funded research should always be published with sufficient information (or be obtainable) to perform replication. You only have to look at the replication crisis to know there is a problem.
Good comment.
And of course one of the primary and valid criticisms McIntyre has made over and over again is that paleo and other climate papers deliberately obfuscate or invent methodologies and fail to fully disclose and publish all the data.
The claim is that the fate of the world is at stake. So maybe its a good idea to have someone check your work, Dr Mann, before we spend $trillions? What do you mean “how dare they!”?
Read the Peer-Reviewed Sea Level work and then look at these images. It is all a fraud.
https://imgur.com/a/siHPINY
The evidence that relative sea level is falling in the scandinavian countries is clearly explained as land rebound following ice un-loading after the last glacial.
It cannot be used as evidence that global sea level is falling – it isn’t. And glaciers melting since the mid-19th century are a significant cause of global sea level rise. There is no doubt that is true either.