From the Ed Begley Jr. department:
“If these scientists have done something wrong, it will be found out and their peers will determine it,” insisted Ed. “Don’t get your information from me, folks, or any newscaster. Get it from people with Ph.D. after their names. ‘Peer-reviewed studies is the key words. And if it comes out in peer-reviewed studies . . . “
Professor of Economics George J. Borjas writes on his blog:
I have a few pet peeves. One of them is how “peer review” is perceived by far too many people as the gold standard certification of scientific authority. Any academic who’s been through the peer review process many times (as I have) knows that the process is full of potholes and is sometimes subverted by unethical behavior on the part of editors and reviewers.
The reason I bring this up is because of a brewing scandal in my own discipline, economics. There has been online discussion about this for over a month…
…
The facts seem easy to summarize.
- Two young economists, Petra Persson and Maya Rossin-Slater, wrote a paper entitled “Family Ruptures, Stress, and the Mental Health of the Next Generation.” They submitted the paper for publication to the American Economic Review (AER), the premier publication of the American Economic Association.
- The paper was handled by AER co-editor Hilary Hoynes, an economist at UC Berkeley. All of the available information indicates that the paper went through normal reviewing procedures. Hoynes sent out the paper to four referees that she specifically selected to give her advice on whether the paper was sufficiently important and original to be published in the AER.
- After the referees wrote their reviews and the authors addressed the various issues raised by the reviewers, Hoynes accepted the paper for publication.
…
And here’s where things get really interesting. It turns out that Hilary Hoynes, the co-editor at the AER, happens to be a current coauthor of one of the junior economists who wrote the Family Ruptures paper. (Page 12 of her CV dated October 13, 2015, indicates Hoynes was working with one of the coauthors while the review process was ongoing). This is a big no-no. The editor, by selectively picking which referees will review the paper, has a lot of influence over how the “peer review process” turns out. A good editor has a feel for how particular economists will react to particular kinds of work, so that by choosing the right reviewers the editor can “nudge” the final assessment in a particular direction. The conflict of interest is so large and so obvious that the AER has written guidelines about this:
There are several rules that affect assignment of manuscripts. Coeditors are generally not assigned manuscripts authored by an individual at his or her institution, by an individual with whom the Coeditor has been a recent coauthor, by an individual who has a close professional or personal relationship with the Coeditor, or by an individual who has served as a graduate student advisor or advisee of the Coeditor.
To make matters worse, after the barrage of posts at EJMR pointed out that there existed at least one paper in the medical literature that resembled the now-forthcoming AER paper, Hoynes (and perhaps otherAER editors) attempted to resolve the problem by allowing the authors to add footnotes and a new section to the Family Ruptures paper. These post-acceptance revisions were apparently added sequentially in different rounds. Despite the additions and despite the new information, the paper was never again sent to the four referees to determine if the nature of the contribution had changed in light of the new information. Instead, the to-be-published version of Family Ruptures contains added-on passages with “Consumer Warnings”-like notes that stick out like a sore thumb.
(As an aside, EJMR has been referred to as a cesspool by some commentators. Retraction Watch published an article about the brouhaha last month, and quotes Hoynes dismissing EJMR because it is “unmoderated” and “not a legitimate source of information.” Unfortunately, she does not address how this unmoderated forum of illegitimate nonsense led to revisions in an already-accepted paper at theAER.
Read the entire post here: https://gborjas.org/2016/06/30/a-rant-on-peer-review/
Why is the blatant failure of peer review of an economics paper in that fields most prestigious journal relevant to us here where we discuss climate? Well for one, we saw failures of peer review such as gate-keeping and favoritism on display in the Climategate episode. Remember this one from Phil Jones at the Climate Research Unit of
East Anglia University?
“I can’t see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow — even if we have to redefine what the peer-review literature is!”
This failure in economic literature is relevant to climate science because not only are the dynamics we’ve seen in economic science demonstrated similar to climate science, but also both fields have large influences on public policy — in part due to politicians confidence in peer reviewed literature. (h/t to Larry Kummer)
Reblogged this on | truthaholics and commented:
“Why is the blatant failure of peer review of an economics paper in that fields most prestigious journal relevant to us here where we discuss climate? Well for one, we saw failures of peer review such as gate-keeping and favoritism on display in the Climategate episode. Remember this one from Phil Jones at the Climate Research Unit of
East Anglia University?
“I can’t see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow — even if we have to redefine what the peer-review literature is!””
Ed Begley Jr is an extreme Climate crusader except when he isn’t – Everybody has a price.
Hollywood celebrities caught on hidden camera accepting money from “Middle Eastern oil interests”
Clinton Foundation – “smoke & mirrors”
For the most part…science builds on previous peer reviewed papers
Let the first one slip through….and it snowballs from there
The ‘peer reviewed’ papers that broke climate science were those about climate system feedback. One by Schlesinger and another by Hansen were cited as the theoretical basis for climate system feedback and lent plausibility to the idea that positive feedback can turn 3.7 W/m^2 of forcing into the incremental emissions of 16.4 W/m^2 required in order to sustain an incremental surface temperature rise of 3C. This factor of 4 amplification ostensibly comes from net positive feedback, amplifying what they claim is the zero feedback gain of 1.6 W/m^2 of incremental surface emissions per incremental W/m^2 of solar forcing. The 1.6 is the result of the BB emissions of a body at the average temperature of the planet according to the Stefan-Boltzmann LAW being about 1.6 times more than the total amount of solar power not reflected by the planet which actually quantifies a closed loop gain of 1.6 after all feedback positive, negative, known and unknown have had their effect. Bringing this up often leads alarmists to absurdly deny the applicability of the SB LAW.
The error is a climate feedback model that assumes an active gain block, while the climate is more like a passive RLC circuit then an active operational amplifier. An OP amp measures its input and feedback to determine how much output to deliver from an infinite source, while the climate consumes its input and feedback to produce its output. This COE constraint is unrecognized by the IPCC’s self serving consensus because to do so undermines their reason to exist by eliminating the plausibility provided by claims of positive feedback. They have since canonized this error and obfuscated it by redefining the dimensionless ratio of gain as the non-linear relationship between temperature and power so that you wouldn’t notice that a 3C rise requires more than 4 times more surface emissions to sustain it than the amount of forcing claimed to cause it.
Even if.. if.. notice I said if, the method were true, the numbers arent. They used 1368 w/m^2 and not the correct number 1360. It lowers the feedback by a third. Wishful thinking to get a 3 C rise. They’d be lucky to get 2 C. Further, deep in their math is the bode formula in electronics. The problem with that is that to get the amplification they are looking for, there has to be an external source of power. ( the sun in this case is not an external source)
Rishrac,
They will be lucky to get 1.1C from 3.7 W/m^2 of forcing and that’s only if doubling CO2 is equivalent to 3.7 W/m^2 of solar forcing. There’s another inconsistency in the definition of forcing used by the IPCC, where on a joule by joule basis, an incremental increase in post albedo solar input is considered equivalent to an incremental decrease in power passing through the atmosphere from the surface.
All of each W/m^2 of incremental post albedo solar input can be considered to be forcing the surface (clouds are thermally connected to the surface through the water cycle) while the incremental decrease in emissions from the surface is an incremental increase in atmospheric absorption, where the atmosphere can emit the absorbed energy either up into space or back down to the surface in about equal parts, thus in LTE, 1 W/m^2 of incremental absorption by the atmosphere is equivalent to only 0.5 W/m^2 of incremental solar input.
Exactly my thoughts as well. I’m giving them the best case scenario based on their own math and methods which falls below any of the predictions.
AGW… ” flawed at best, fraud at worst “.
Very nicely summarized. I’ve never understood why even a cursory examination of 500,000 years of reconstructed history of global temperature did not put AGW theory out of its misery in its infancy. If the system response to CO2 temperature forcing were positive, we would not be here to argue about it.
Claude,
What makes this insidious is that Schlesinger has been aware of the errors in his analysis for over a decade. After much discussion with him, I brought up the errors I found in his feedback paper, at which point he got mad, signed me up to is propaganda news letter and no longer wanted to discuss this with me. He was introduced to me by Mike MacCracken as the expert on climate system feedback and he egotistically confirmed his ‘expertise’ to me. BTW, I’m an EE and understand control theory inside and out. I can only conclude that he either understood the errors I pointed out and their implications (specifically political implications) or he is absolutely clueless about control theory. Either way, this is where climate science broke. If this RICO stuff continues, Schlesinger has a target on his back.
Make that “positive feedback”.
These shennigans with peer review have been going on for a very long time, either publishing papers that shouldn’t be published or preventing publication because it’s ahead of your research or the research of a former student. I experienced the latter 40 years ago. I remember that every time I see “peer review” used as a gold standard.
I think the peer review process is an extremely useful tool and I know researchers who do conscientious jobs as reviewers. Like anything, the value depends on the integrity of the reviewers.
I agree, Bob – to a degree. But editors hold more power over this process than may be healthy. The integrity of editors (or lack of it) – as the example of this economics paper shows – can bring the whole thing to its knees.
Here’s an example (without naming names) that may lie somewhere in the middle – make up your own mind if the editor was justified in overreaching his power.
Years ago, a paper appeared in a journal that made some rather extreme claims about the age and origin of domestic dogs (which has been a very contentious topic for years). A colleague and I decided it needed to be countered. We submitted a scathing critique of the paper.
The editor sent it out to EIGHT reviewers. We were stunned. None of my colleagues had ever heard of an editor asking for (and receiving back) eight reviews on a paper.
I can tell you that satisfying all of the issues raised by so many reviewers was a logistical nightmare. It seemed an outrageous roadblock to publication but we did our best and sent it back, I admit a stronger paper than it had been.
Our critique was published and got a lot of support from colleagues. At least two wrote to me and said thank you for writing it because it saved them having to do it. I’m convinced that critique swung research back on track, where it continues today (still not settled but good research being done).
In this case, the editor was undoubtedly covering his ass over a hotly disputed issue – and perhaps feeling guilty for having let the first paper through without stronger review. Maybe he wanted our critique to be as strong as it could be. I don’t know.
But I do know that to this day, I’ve never heard of an editor using eight reviewers for a single paper. In this case, the outcome was in our favour – but it could have gone the other way.
If so, to whom would we have complained?
Excellent point, Dr. Crockford. In my anecdote, we had great problems publishing something from my dissertation. I happened to go to a seminar given by the reviewer’s former student who was using similar techniques. I don’t believe had published in that specific area. He knew who I was and refused to even speak with me.
I believe the peer review system in general makes for stronger publications, but the be-all, end-all, gold standard and you can’t question the science?
The issue in science is that is a human endeavor practiced by humans. Much like other cultural institutions, it can very useful, but it is not a magic wand. Take democracy / free and fair elections, for example. It may be the best way to select our leaders, but it is not perfect. If one party wins the election, do all of the other parties disband and pledge themselves to the Choice of the People? Are ethical and moral standards up for vote?
Similarly, science and peer review are a process that works reasonably well. However, if a peer-reviewed paper says that the moon is made of green cheese, it does not change the moon. Science is an argument about how the world works, and just because one side is getting more papers published doesn’t mean that they are inherently right. The response should be to do more studies, marshal more evidence, and make a solid case. There is a bit of scientific textbook consensus – for example, that gravity and electromagnetism follow an inverse square law, that CO2 absorbs infrared light, that glaciers formerly covered much of North America. However, sometimes parts of that get overturned – Helicobacter pylori and ulcer formation, for one.
It may work well in most fields most of the time but the issues in climate science are so egregious and so pervasive that they indicate a systemic problem. In corrupting itself, climate science has also lit the path for the political class to influence other fields. This is a battle between those who seek the truth and those who believe it is only subjective.
Still is the gold standard Bob?
The scientific method is the gold standard. Since a Peer reviewed publication shows that most peer reviewed papers are wrong, the gold is tainted, and in some fields merely paint on the surface.
What continues to amaze me, is the number of papers that correlate some small, presumably negative aspect of a biological system, recently noticed during the recent warming, that then get coupled with a General Circulation Model (GCM) that predicts (and let’s be honest, these are predictions, not projections) lots of future warming, leading, a priori, to biological catastrophe. When did “science” journals give their approval for this process?
I’m not saying that newly found correlations in biological systems should never be published. They can be, IF the assumption is built in that these correlations are nothing more than starting points for further research. But when we pair such correlations with a GCM, we suddenly and quite subtly, change the presumption of the correlation to a causal relationship, a fallacy of the first order. And that’s true, regardless of whether or not the GCM outputs are objectively correct. And we know that they are NOT. Even Gavin Schmidt admits that they are not. So now we are piling one horrible logical fallacy on top of known wrong computer models, giving a doubly awful result. When I read of a study here that does this, I already know that they will end by warning of future doom. It is a trivial tautology, because we know that the big GCMs generate results that not only are wrong, but consistently run too hot. These kinds of papers should be flatly and automatically rejected.
The GCMs themselves, internally, their algorithms, their calculus, their presumptions, should all be subject to peer review, along with classical scientific replication and falsification before anyone publishes one more paper about them or based on them.
I’ve said this many times before, but given the way the GCMs have evolved, the way science journals corruptly accept these horrible practices, the way activism has corrupted climate science in general, in the future the evolution in this field will make an excellent research case study for anthropologists studying our culture. They will learn how humans fooled themselves on a vast scale, with the blessings of, if not the insistence of governments, academe, and publicly funded, grant-based research.
“When did “science” journals give their approval for this process?”
This is standard practice in politics when the facts don’t fit your position. You take a kernel of truth and spin it out of control in order to discredit the other side or support yours. The rhetoric surrounding the current election, especially that coming from Hillary, is following this script exactly.
While climate science has been political since the inception of the IPCC, I blame Gore for replacing the scientific method with conformance to a narrative.
Economics? Who cares about economics, really? Does it in make a difference on people’s lives?
The real drama is in biomed.
I care about economics. As for biomed, meh, if they get it wrong it’ll just kill you. Economics has the ability to make life miserable for a long period of… for a lot of people.
Real people have interactions sometimes for money, not “economics”.
Economics is a field, not a reality. It only exists in reality if people take the field seriously. Few people do, except politicians (and then, only when it says what they want to hear).
OTOH, biomed hasn’t been taken seriously by realists for some time now, but it’s all we have: few people can intuitively see if some drug is probably effective. Also, drugs can be forced on people, and other drugs can be made illegal. This is why medicine is unlike most other commercial practices, and why many people want to regulate it strongly (which exacerbates the corruption of the field and the Deep State – you can’t even test the flu vaccine in the US, cause ethics!!!!).
Now I am wondering if there is an economic field of the biomed field: the economy of the medical publishing industry and the psychology of MD.
And now I am wondering if a field of study can be an hallucinating drug…
simple-touriste,
i didn’t know drugs could hallucinate. 🙂
medicine has the same problem with false positives as does economics.
In large part the problem stems from the pressure on academics to publish, along with a bias in publications towards positive findings.
for example, you conduct a study, you find that drug X has no effect on cancer. someone else does a study and finds that drug X cures cancer. which study will gain the headlines, which study is more likely to be read, which study is more likely to get published?
Of course the “X cures cancer” will get published, but the “X has no effect” study is by far the more important study scientifically, because it establishes that “X cures cancer” is a false positive. But since “X has no effect” is unlikely to get published, we end up with an epidemic of false positives.
Ferdberple is exactly right. In addition, editors increasingly do not like to publish rebuttals, even when the errors are egregious and catastrophic in a paper. If the sediment proxy is used upside down, the journal doesn’t want to hear about it. They are running as fast as they can to get papers out and don’t want to hear about any mess they left behind.
It’s worse than that: the ethics and scientific rigor defended by the editors effectively prevents correct information of the user! The editors won’t published a result that isn’t “significant” at the religious level of p<.05 (whatever it means in each case).
How many people have been taught that hypothesis testing is the essence of the scientific process?
Because we don't know anything about the trial and error of the (alleged) "scientists" to arrive at a given statistics, we effectively can't compute any "initial" p-value (the researcher could have thrown away many negative results). How many "significant" statistical results may actually be Texan sharpshooting?
We need to know the thought process of the researchers and not just the (hypothesis, experimental device, raw data, cooked data, statistics, yeah I achieved significance!) n-uplet which is just one part of the scientific process.
Hypothesis testing may be essential, but it isn’t the quintessence of the scientific inquiry. People may believe that hypotheses appear from the vacuum like virtual particules, but they do not. They come from a scientific culture, a sum of knowledge or believes which may be incorrect.
The Bayesian approach isn’t a magical tool, because no such thing exist; it isn’t even a decision method, as the scientific approach cannot be reduced to a simple algorithm that works in every case. The benefit of Bayesian statistics is that they force people to make the implicits explicit: don’t pretend you come from nowhere and have no intuition or pre-existing opinions, we know you have some. You aren’t a blank page and don’t need to be one.
People really need to make some effort to investigate themselves and their pre-established opinions, and how advertising influences us.
Once I had to review an article in which the authors reported three (5 percent) significant results. These were treated as pieces of gold since they were ‘significant’ for once and for all. The problem was that they employed sixty measurement moments and did sixty tests. I tried to explain that (assuming independence) the expected number is 60/20 = 3, if there is no effect at all. I could not convince them. Something similar happens on a large scale. See the publications by John Ioannidis.
“The problem was that they employed sixty measurement moments and did sixty tests”
At least they were honest about the number of tests… how many researchers just bury the failed tests?
Simple-Touriste, I’m having a hard time understanding your post without your use of the sarcasm button.
Fact is, individually, we deal in economic situations dozens of times per day. You may call it just money matters, but the reality is that they are economic transactions (micro-economics) that when applied to everyone’s transactions, lead to inflation and unemployment rates, interest rates and the like (macro-economics). Economics is a foundational scientific discipline.
It is also infected with politics to an even greater extent than climate science. Thankfully, the divisions are much more binary and straightforward, whereas the climate science infection is a strange double barrelled attack of ignorant greens and socialists, aided in the political arena by self interested slime balls like Gore.
Economics may use math and statistics, but it is not “scientific,” or at best it is a soft science.
true Phil, because a large portion of economics is based on human nature, and that is somewhat inconsistent, and in areas it is inconsistent such as the “hidden hand” often ignored for political expediency.
at one time the cost of distributing information was high and it was necessary to determine which information was worth distributing.
that age is gone.
the cost of distributing information is now almost zero.
yet, like luddites, we cling to the old information distribution model.
in our new information age the readers are the only “reviewers” that matter.
but we are glued to a model that was designed under entirely different circumstances.
weird, but true.
“the cost of distributing information is now almost zero.”
But the cost of parsing all this material is high, so we want “gatekeepers” of our minds.
Right. The cost of publishing a book is at a historic low. The result is that most books fail in the marketplace, lost in the noise of self-published drivel. The problem is in lack of disinterested gatekeepers.
I call it editorial censorship. The editor sends the paper to the ‘high priests’ of the prevailing wisdom. They accept the paper and the prevailing wisdom is preserved and science dosn’t advance – it is effectively settled.
Then others come along and count the number of papers published that support the prevailing wisdom and use the total as proof of the prevailing wisdom.
There was a mythical bird, a Wogga Wogga I believe, that flew round in ever decreasing circles and vanished up its own fundament.
Maybe we should be calling it “Crony Review!”
Always glad to hear your thoughts Dr. Ball. Any ideas for improving the process?
There must be some mechanism to decide if a paper is fit for publication in a particular journal. If not editors on the advice of peer reviewers, then who and how?
This Wogga Wogga bird sounds like a close relative to the Whirla Whirla bird that members of my rugby club used to sing about over a few post-game pints of beer.
Wikipedia has an entry on the topic: https://en.wikipedia.org/wiki/Oozlum_bird
Bismarck once observed that people should not investigate too closely the making of sausage or peer-review…or was it something else?
Peer review has helped me avoid a serious error on one occasion. On the other hand I have had peer reviewers pass information to third parties, suggest that i cite their very important work, demand that I correct my paper in a way that would have violated the second law of thermodynamics, and so on. There are also journals that sometimes ask the author for the names of referees. Why should anyone place much confidence in peer-review?
Bismark was talking about diplomacy in the Balkans.
Seems a fitting metaphor for peer review.
Then look at what’s being published in the “New Green Economics” research field ! Research?
In addition, the U.S.SEC financial reporting requirements are being lowered to conform to International financial reporting requirements which means that less financial information on publicly traded companies will not be available to investors or potential investors.
Going right on under peoples’ noses and gets no general public attention in the U.S. Companies can get away with not reporting financial information that at present is required.
Congressional Research Service, June 25, 2015
‘U.S. Capital Markets and International Accounting Standards: GAPP Versus IFRS’
For more information on changing from U.S. GAPP to IFRS a new global accounting standard.
https://www.fas.org/sgp/crs/misc/R44089.pdf
In a truly free society, companies wouldn’t be required to report anything and investors would be left alone to decide whether or not they trust a company that refuses to disclose their financials. Market forces would inevitably result in full disclosure by companies hoping to entice the investment they require to expand.
Capitalism works at all levels, including for publishing houses that practice poor peer review. Eventually, the latter are deemed unreliable and subscriptions decline. It is not a lack of laws that create the problems you bemoan, it is theit existence in the first place.
IFRS Report
P.8:
“Perhaps the greatest difference between IFRS and U.S.GAPP is that IFRS provides less overall detail and industry specific guidance.”
http://www.ifrs.com/pdf/ifrsupdate_v8.pdf
IFRS adopters:
Canada 2011
Mexico 2012
You mean in a truly honest society. But that’s Pollyanna Land. (Ask any casino owner.)
That was part of Alan Greenspan’s fanciful air-headed, and out of touch, logic while Fed Chair. He scoffed at fraud, and as a result, refused to do the proper regulation of mortgage banks–the Federal Reserve is the designated legal regulator of mortgage banks because they aren’t regulated under the federal banking law–even though–even though!–the FBI said in open testimony in September 2004 that there was an “epidemic” of mortgage fraud, 90% of US mortgages were fraudulent, and warned of a meltdown in the US economy.
Had a more complete story (specifics) about the hearing, but this will do.
http://www.cnn.com/2004/LAW/09/17/mortgage.fraud/
That’s why you have, and need, regulators. (Just talk to a pit boss.) But the Clinton and Bush admins allowed the regulators to not regulate, and convinced unthinking somnambulant Americans that it was somehow ‘federal government interference’. (Ask Steve Wynn if he would operate his casinos without eyes and cameras on his tables.)
Barbara: “… less financial information on publicly traded companies will not be available …”
Please ‘splain.
The problem with peer review is that science is complicated and hard. There is always some degree of interpretation of results, and many choices of methods and analysis. Very few reviewers actually understand a paper fully because no two scientists have the exact same skill-set and because a good paper will have novel elements.
The other problem of course is the herd tendency which causes fads and gate-keeping. People want to be liked and they want their work to be recognized. Both desires lead to conformism and me-tooism. Scientists understand that if a topic is “hot” they are more likely to get funding and to get published. In the softer sciences this can lead the lemmings off a cliff when a popular topic is basicly BS. Sad but true.
In the softer sciences this can lead the lemmings off a cliff when a popular topic is basicly BS.
==============
psychoanalysis comes to mind. competing theories over which brand of BS is correct.
More than agreed, most of the problems in the soft sciences come in the form of measurement error, even the complete lack of validity in the measures. It’s very easy to get a paper published that has a “significant” finding if it fits a pattern, or theme, that’s popular in the field. It’s also very easy to get the result you want through construction of measures, without regard for the proper psychometric tests to insure that the measures are meaningful, and don’t lead to biased self-selection. Subsequently, the “cooking of the numbers” doesn’t happen after the fact, it’s really happening before the fact.
In addition to distinguishing between the social sciences and the physical ones, you need to “divide the natural sciences, separating (A1) those concerned with homogeneous entities and deterministic (at least in the aggregate) relationships, from (A2) the ones that deal with chaotic processes (like climatology).
Most of the progress in knowledge and technology comes from the (A1) category. Although researchers in the other categories would like you to think they are making comparable contributions to society, they are not.”
See more at http://www.evolutionnews.org/2016/06/theres_no_grand102935.html
Ralph,
The chaotic processes involved with the climate are primarily in the paths taken from one equilibrium state to another which we also call the weather. The next state itself, or more precisely, the average of all possible next states (for example, the average of El Nino and La Nina), is deterministic and limited by the available solar input. Otherwise, the expected seasonal variability could not be removed from the data in order to arrive at anomaly plots, which per hemisphere are nearly perfect sine waves 180 degrees out of phase. BTW, removing the expected seasonal variability is another trick used to make the temperature appear to change more slowly than it actually does. Monthly temperatures vary by 8-12 C per hemisphere in response to seasonal change and owing to their asymmetry, the hemispheres cancel in phase, but not magnitude. Globally, the average temperature follows the sinusoidal seasonal variability of the N hemisphere at the reduced magnitude of about 3C peak to peak between the global average temperatures in January and June. They don’t want you to notice the rapid response to seasonal changes in solar forcing as that contradicts the requirement for the long time constant they need to explain the imagined effects they claim are imminent and unavoidable, but haven’t yet manifested a temperature effect.
Check and see which brand of analysis was successfully used by the OSS in WWII to predict Hitler’s actions. Hint: “Nobody will probably ever know how much Professor Jung contributed to the Allied cause during the war . . . His work [must stay] highly classified for the indefinite future.” –Allen Dulles [from Jung: A Biography: Deirdre Bair]
As the “science” of CAGW proves, the cliffs-and-lemmings phenomenon is not limited to the soft sciences.
Given the reliance of statistics over observation that is prevalent in climate science, it is hard to consider it a hard science, IMO.
Michael,
I interpret your comment as suggesting that you consider “Climate Science” – “the “science” of CAGW” so capturing your quotes – to be a hard science.
Or, of course, a steely [cf. Stalin] attempt to subvert democracy by redistributing wealth without informed consent.
Shall we agree on the latter?
Auto
I would say that climate is a soft science attempting to justify itself with the tools of hard science. To the extent that it does not ask questions but seeks to prove an assumption, it is not science at all.
Well, whether climate change science is a proper hard science or not, many of its practitioners have been trained in math, physics etc., and they have made some excellent lemmings.
Michael,
That so many of the lemmings have actual science training is why much of the public without it can’t accept the fact that people they think are very smart can be so wrong about something with such dire consequences. Besides, if the political party thinking for them agrees, why would they believe otherwise?
Climate science has far more in common with political science than with physics, math, chemistry or biology which tells me that adequate peer review alone will do nothing to fix the problems caused as the IPCC seems to have leveraged the ignorance inherent with partisan politics to push a demonstrably false narrative.
John Harmsworth,
What you describe is not science, but scientism.
co2isnotevil: well said.
It is interesting when one reads the history of physics, peer review is not the measure of a theory. experimental confirmation is the standard for acceptance.
There is good reason for this. Imagine I designed a computer model of the real world, of course it will confirm my theories, because I used my theories in the building of the model. If it didn’t I would either change the model or change my theories.
The real world however has no such restriction on its behavior. It may follow my theories for awhile, then shoot off at a tangent, proving I had it all wrong.
Therein lies the problem with peer review. Each of us has a mental model of the real world, similar to a computer model. In our mental model, the real world behaves exactly like we believe it will behave. This is true for the most brilliant scientists, as it is for the world’s biggest dunderhead.
An so, it is against this mental model that Peer Review is conducted. And as a result Peer Review cannot tell us anything about that which is unknown, it can only be a guide to what is known.
Exactly !
And the GHG theory for why the bottoms of atmospheres are hotter than their tops has none .
Nor does it have quantitative testable equations .
“And the GHG theory for why the bottoms of atmospheres are hotter than their tops has none .”
BzzzzT! Wrong!
I take it you don’t think mirrors work either, neither do Yagi-Uda directive arrays …
_Jim , You could answer simply by pointing us to the equations and/or experimental demonstrations .
re: Bob Armstrong July 5, 2016 at 7:57 am
Bob, I have, before. I’ve got BETTER things to do with my time at the present, unless, you can meet my ‘contract’ rate at 80 bucks an hour …
_Jim ,
So , off the top of your head , you know no links to the enabling equation .
I take it by “mirrors” you mean concave mirrors . Those and antennas concentrate energy by displacing it leaving a deficit elsewhere . They are irrelevant .
If you have presented the relevant equation elsewhere , I’m sorry I missed it . The only possible contender I’ve seen is the Schwarzschild radiative transfer equation . But playing with it to find an asymmetric domain is low priority in the development of my 4th.CoSy for implementing such things .
In the opening pages of IPCC AR4, report of Working Group 1, the authors announce the demise of falsifiability of claims. The replacement: peer review!
Also applies at historians. They often have to explain or interpret certain events, like the fall of the Berlin wall. Suppose, as thought experiment, that all written sources of preceding events were lost. A historian should nevertheless propose a reasonable interpretation in order to pass peer review. There may be several, but if he or she would, by a flash of inspiration, write down the real story, this would not be accepted as being obviously a delusion (as there are too many).
My feelings about peer review match those of Churchill about democracy.
We have an infinite (practically) number of scholars who are desperate to publish. We have the problem of deciding which to spend/waste our valuable time reading. Peer review promises (but does not necessarily deliver) some kind of curation.
We have limited time. We can’t read everything. What system promises to deliver the papers we need by winnowing out the junk, while at the same time not filtering out the brilliant but heterodox?
Commiebob, there is just such a system that I found very useful in the more traditional science disciplines . It is the review article in dedicated review journals such as Chemical Reviews , Reviews of Modern Physics , Annual review of Materials Science , etc . Good examples often start by taking the reader from the early publications concerning the relevant subject , and advancing to the latest articles on discoveries, materials , theories or models. They are sometimes effectively textbooks , but with the added advantage of the latest articles , sifted by the reviewer.
Whether the same can be said of the often contentious subject of Climate Science I do not know , but if there is something like , say , Annual Review of Geophysics , that might be a place to start – if you have a large private income or access to a University Library.
Another point that most people miss about peer review: the reviewer’s read the paper and comment on it. They do not rerun the experiment, or our reanalyize the data. Their function is editorial.
Peer review is not a guarantee of the quality of the substance of the paper.
Until a few years ago, I had the vague idea that peer review involved actually checking the data by redoing the experiment. Even science courses do not always make it clear how peer review really works. If science majors are unaware of how the review process works until midway through their degree, why would the average layman know? Explain the process to a non-academic and see if they understood it correctly before. Anecdotally, the people I have talked to about it are surprised at how unrigorous peer review is compared to their expectations.
Peer certainly isn’t replication. My own discipline is Computer Science, and I have been peer reviewed, done peer reviewing, and even been on an editorial board. I think it’s fair to say that peer review as practised in CS is pretty good at filtering out unreadable rubbish (you wouldn’t believe how much of that gets submitted), filtering out unsupported claims, checking the odd bit of mathematics to make sure the graphs aren’t upside down, catching plagiarism (including self-plagiarism where one result is reported in several papers), and noting where people aren’t up with recent literature. I think it’s also fair to say that some referees do a great job of helping people improve their papers; I’ve had just such help this year. Any time-consuming checks would be impossible. There are also in some fields deliberate barriers to replication: there was an education paper I was very interested in some years ago where the raw data were embargoed for a couple of years so the experimenters could milk it first BUT that meant reviewers couldn’t even replicate the *analysis*, let alone the experiment. (And if you think a journal can fund a replication of a multi-continent multi-year student, you must come from the Land of Free Money.) What peer review is very poor at is detecting deliberate dishonesty (other than plagiarism, thank you Google).
In short, peer review is at best an idiot filter, not a guarantee of Truth.
But there are idiots out there, so an idiot filter is a useful thing to have.
In case of Moon landing ideation (the word doesn’t spell check but I LOVE it), the idiot filter failed in a spectacular way, with editors resigning when they learnt that the 2nd smear paper was retracted…
More like idiots active sonar in this case.
Blind trust in “peer review” as a proof for allegedly “settled science” demonstrates a poor understanding of the scientific method.
http://notrickszone.com/2016/05/11/peer-review-veteran-peer-review-has-lots-of-problems-tends-to-stifle-non-dogmatic-thinking/#sthash.n1jajZC3.dpbs
Getting people you know to review your paper?
I always liked the term “pal review”.
Yep .
In my decade in grad school which ended when I actually understood something which required a solid understanding of undergraduate math , I witnessed the pal review process supporting the publication requirement for the utterly mediocre and banal . And I was at at “good” school .
I see this battle as largely between the natively capable and the established mediocrity .
The plus 20 years off pal review in the climate scam industri, have shown us that this is not the way to review scientific papers
ole jensen –
Ole, Ole, Ole!
Plus rather a lot!!
Auto, with apologies for using the appositeness of Mr. Jensen’s given name.
But – he is exactly correct.
The traditional standard of truth was constructed on reconciliation of multiple, independent sources.
The best antidote to sloppy peer review is to get Mark Steyn to write a book on your topic 😉
Ah, economics, the ‘Dismal Science’!!!
I receive papers for review from time to time. Some require detailed suggestions for technical corrections. Others, reviewing the same paper, will write a single dismissive line saying the work doesn’t conform to their own methods.
This means there is a second level of control. If the work is valid, or fixable, it may still be promoting another world view and the review process can be manipulated to serve primary (what gets published) and secondary (what world view gets bolstered) goals.
Obviously in ‘climate’ papers there are teams of colloborants who hate each other as well as macro groups whose world views are mutual anathema. This can’t be fixed with the present system.
As a reader (the reader/consumer matters a lot) I would be most impressed by a paper that has been reviewed by named persons I respect. When a thesis is published we know the names of the reviewers and their reputations are attached to the works they review. Good!
Let’s cut off the access to the smoke-filled back rooms where the academic scum like the UEA, Berkeley and Penn State’s usual suspects presently package their crooked deals.
There have been journals who published the names of reviewers with the paper. Tulane Studies did this at one time, at least. I have probably examined 10K papers in my interest in the last 15 years. Repetition and replication is rampant and old important questions are often either unanswered, ignored, or buried in the morass. Impact Factors should be killed.
The pressure to make a splash while not getting anyone wet has caused academics to publish lots and lots of stuff no one intends to read or cite.
We are living in the post-scientific era.
It started when scientists stopped being gentlemen – i.e., when they were no longer independently-wealthy enthusiasts who didn’t have to make a living.
I’m aware that this puts the beginning of the rot somewhere in the 19th century with Humphrey Davy and Michael Faraday, but at that time the spirit of the thing was unchanged, even if middle and working class interlopers were making inroads. Arguably, it was only the gentlemanly rules of the game that allowed Faraday to make his mark by the very unfair expedient of simply being right!
Nowadays it’s unlikely that an outsider could do this. Science now is not tainted by politics. Science is politics. We won’t have long to wait before the new science simply stops working, and we have to go back to magic…
I thought we already back to magic. I’m sorry, magic actually gets it right sometimes, and sometimes a lot. Unlike the hocus pocus of the predictions Climate Science has made. Hypocrisy freely runs rampant in the minds of CAGW if the ice field expands, that’s nature, if it shrinks well, now that AGW.
This is probably the most important, giant-killing article about science and peer-review in general I have read in a very, very, very long time. I can’t thank everyone who has commented, as well as Anthony, for having written their musings about it.
If peer-review was held to the same integrity it was years ago, there wouldn’t be the explosion in journals and “science” articles written today. If laypeople cared more about how the world around them works, instead of what new drama is coming out of the Kardashians’ hen-house, perhaps the love-fest they have with science “experts” would change.
Human nature is easily corruptible, for many reasons, and even for well-intentioned people. I can’t count the number of tools I’ve dealt with online who know nothing about climate or science in general but argue vociferously about how I’m a nobody and should publish my own peer-reviewed papers to have any credibility. It’s comical to a point and yet, sadly, shows blind allegiance to a process where they themselves don’t have to do any thinking at all.
+1000
I call it editorial censorship. The editor sends the paper to the ‘high priests’ of the prevailing wisdom. They accept the paper and the prevailing wisdom is preserved and science dosn’t advance – it is effectively settled.
U
Then others come along and count the number of papers published that support the prevailing wisdom and use the total as proof of the prevailing wisdom.
There was a mythical bird, a Wogga Wogga I believe, that flew round in ever decreasing circles and vanished up its own fundament.
Tom, I am with you on this one. It is priestcraft – an insider club that only admits to input from fellow priests of the same stripe. If a fellow priests disagrees with them, they cast him out as a heretic and denounce him in public for his failings.
A reminder: priestcraft is when the priests (or you) tells someone what something means – interprets the signs, as it were. When you look at the same signs and make your own (‘wrong’) interpretation your very soul is on the line and you have to be saved from ignorant yourself, they aver.
Keeping inner knowledge secret is part of the game, hence the manipulating or not sharing of data. That is a sign that priestcraft is being employed. This is not to say anything in general against priests. Some priests are great people and lead their chosen fields. But telling people to stop thinking and cease their independent investigation of truth is unacceptable and against the spirit of the age. A great deal of the ‘peer review’ circle-jerk is intended to stifle critical analysis and shut the traps of those who see through the facade.
The CBC had some guy from Guelph University on two morning ago telling us that “the signs of climate change are all around us. There is no use denying it.” Having successfully changed the meaning of ‘climate change’ to ‘global warming caused by humanity’, he was trying to lampoon those who bother to think and investigate what a crock he is selling. It is part of a multi-media campaign partly financed by the Provincial government which is running ads on TV with children sneering at people who don’t swallow the CAGW arguments. They even have child-priests telling the thinking public they are lost souls in need of salvation by the truest of believers, “It’s not like it isn’t happening…” Brainwashed children, the product of BS-academics who have forgotten what it means to be an educator.
Children priests? Like these ones?
Peer-review has always been a dubious process. Some thirty years ago the editor of the leading professional journal in the field in which I worked received a ground breaking paper. She sent it off for peer-review to two university academics. One liked the paper and wrote that, subject to a few minor corrections, it must be published. The other hated it and wrote that the entire paper was misconceived and incapable of redemption.
Under the journal’s protocol the editor sent the paper, together with the reviews, to a third referee for final review. Believing that the paper merited publication and knowing that I had worked closely with the author on other papers, I was chosen as that referee. The paper was published to acclaim.
So many bloggers who contribute to this and similar – if less frequented – sites have expert (even, if in many cases amateur) knowledge of different aspects of the various disciplines that make up climate science that I believe WUWT and several other sites provide better peer-review than any single journal can. Unfortunately such reviews can only occur after publication, by which time the authors are too far committed to amend and will, usually, not even bother to defend.
Solomon
What do you think we can provide here on WUWT to provide peer review before publication? Howe about a sub-site where papers can be flighted and worked on? I know that some things I write about take ages to assemble and would not happen were I employed ‘in the field’ directly. It just takes too long and in the field the pressure to publish is too great. I have been working on one paper for 6 years and the theoretical breakthrough has happened, so I can complete the work this year.
Many of the article contributors could easily have them published in a formal journal. Why not a WUWT journal? There is a lot of room for imagination on this subject because at the moment we are just trading ideas and setting them against the common understanding of a large crowd. This is crowd-reviewed science. After completion, the articles can be brought to ‘the front page’.
The value of a paper is the journal in which it was published, to a great extent, and the extent to which it is cited. There are whole communities in my field which will never cite a paper written by me because they don’t want to draw attention to alternative points of view which debunk their junk. That is normal. It means that one needs alternative journals in case a faction takes over the journal itself, as evidenced by the Climategate emails.
That’s what Anthony tried with http://theOAS.org/ .
I find the basic vetting of articles here and the instantaneous , sometimes brutal , informed and uniformed but honest , peer review here operating at Independence Day 2016 speed .
Who has time for paper ? My peers are here .
“…There are several rules that affect assignment of manuscripts. Coeditors are generally not assigned manuscripts authored by an individual at his or her institution, by an individual with whom the Coeditor has been a recent coauthor, by an individual who has a close professional or personal relationship with the Coeditor, or by an individual who has served as a graduate student advisor or advisee of the Coeditor…”
The Hockey Team and fans don’t see these rules as anything more than punchlines.
I’ve met Ed Begley Jr. He prays to the living deity of Climate Alarmism in a need to feel good about himself as self appointed savior of planet Earth…..
He’s scientifically illiterate and only listens to opinions that re-enforce his own.
You can quote Phil Jones about his desire to keep papers out of the IPCC and change the definition of peer review, but the proof is in the outcome.
What were those two papers? Were they included in the IPCC? Does Phil Jones have standing to change peer review and did he successfully change the definition of “peer review”?
We all express frustrations in e-mail — but was there any practical effect of these e-mails?
http://www.cato.org/publications/commentary/climate-scientists-subverted-peer-review
https://wattsupwiththat.com/2011/11/27/the-tribalistic-corruption-of-peer-review-the-chris-de-freitas-incident/
Can you delete any emails you may have had with Keith regarding the latest Intergovernmental Panel on Climate Change report? Keith will do likewise.
Phil Jones email to Michael Mann May 29 2008.
I’ve just completed Mike’s Nature trick of adding in the real temperatures to each series for the last 20 years (i.e. from 1981 onwards) and from 1961 for Keoth’s to hide the decline.
Phil Jones email November 16, 1999
Just expressing more frustration I reckon
Of course peer review is not perfect, as no system is. I think one change that would improve quality is for the authors to be anonymous to the reviewers. That would make it hard to nod through papers based on reputation or nepotism.
Gold standard? Well, perhaps a tarting point for deciding whether to take something seriously.
One thing that peer review generally does is ensure that the paper is put into proper context. The reviewer is able to check that the introduction provides a reasonable summary of the state of knowledge. Many non-peer reviewed articles are simply far too selective in the literature they cite, selecting only those that back up their conclusions. Obvious gaping holes will usually be picked up by the reviewers.
“I think one change that would improve quality is for the authors to be anonymous to the reviewers.”
—
That is difficult to ensure in practice. Editors select reviewers who are familiar with the topic at hand, and they would very often be able to figure out who wrote the paper from the specific angle, methodology, writing style etc. Papers are often incremental, and where that is the case, a new paper will refer to the previous one (which is published and no longer anonymous) all over the place. Even without such a specific reason, authors tend to cite their own works more heavily than those of the competition, which will give the game away.
It also works the other way around. I can guess who my reviewers were in at least, say, 3 out of 4 cases.
Ultimately, the peer review system is built on honor and integrity. Where these are lacking, the process becomes dysfunctional, and no formal rules and stipulations can save it.
Rather than the author being anonymous, would it be better [*] for the reviewers to be known and for their reviews to be published with the paper? The argument that it would take up too much space in the journal would no longer apply in our on-line world.
[*] NB. “better”, not “perfect”. No matter what method is used, there would be scope for a determined group to corrupt it.
A better solution is to have a team of engineers, statisticians, etc., preferably who actively hate scientists, review all government science studies. By law give them access to everything, data, code, etc. A line by line hostile review of any publicly funded science study.
Estimates of reproducible studies are as low as 11% and certainly less than half. There are so many problems from sample size to misuse of statistics that only a minority of studies have any value. There is no reason the let all those bad studies to escape, and be published, to be used as the basis for more bad studies. Drive the stake in early.
We are paying for good science. We should only get good science.
Peer review is ridiculous, not only does it allow nonsense to get through, it also blocks science which proves it wrong. Take Clovis first, peer reviewed and wrong. 20 years pass with peer review debunking all evidence which showed Clovis first was wrong. Ill also point out peer review,and consensus was ready to vote Einstein’s theory as wrong, only irrefutable proof stopped them, which brings me to point two. “Studies” are not science. The vast majority of science which is peer reviewed and published as fact are nothing more than studies.clinical studies are at best indicators of where to direct real scientific activities. If science was doing science, peer review would be simply recreating and confirming results of real experiments. Studies are nothing factual, which is why it is so easy to do them and pretend,they are,fact
Publish or perish, or so I’ve heard.
Take that for what it’s worth, coming from someone that struggles to complete a sentence.
EJMR? An explanation or link would be useful.
Earth Journal for Making Renimbi.
Each Journey Minimises Rebuttal.
English Jock-footballers Massively Redundant?
I have no idea, and wondered, likewise.
Auto
Google is your friend !! LOL
Economics Job Market Rumors
http://www.econjobrumors.com/topic/best-of-ejmr-the-archive-thread
TB,
Fundament. I like that. Much more acceptable in polite society than my usual terminology.
Hmmm. A term of endearment, in the genre of cryptic namecalling.
Peer Review??? The most useless crap going around. A bunch of fellow idiots thinking they know something.. A peer is nothing more than an idiot,following another idiot.
Peer review is a “publication” issue.
All it is doing is getting someone else, who may or may not actually check the work thoroughly, to say if they think your work should be entered into the scientific literature.
Peer review DOES NOT in any way imply that your paper is actually correct.
Yes, the purpose of publication is put the material out there for a wider range of experts to review. Peer review should be an ongoing process that continues long after publication since most pre publication review is incestuously performed by like minded people whose group think can get in the way of objectivity. IPCC driven climate science only tolerates post publication review when it doesn’t conflict with the narrative which seems to amplify this effect. Can you see the positive feedback acting on the science itself? This is also called positive reinforcement and unfortunately, some very bad behavior is being positively reinforced.
I’m guessing that if you look at past history of peer reviewed “climate science” papers…
…. then at least 97% would be incorrect in some way. 🙂
The phrase “appearance of impropriety” is unfamiliar to way too many in the peer review process.
There is nothing wrong with peer review per se. I do peer review at the rate of four or five submissions per year. Peer review is, on the whole, an extremely valuable service that greatly improves the quality of journal papers,
But let’s be clear what peer review is. It is an experienced set of eyes giving opinions and recommendations on submitted papers. Reviewers do not normally have the opportunity or resources to inspect raw data or repeat experiments. As such, reviewers have to trust that the authors are behaving in good faith.
Someone has to decide whether a paper meets the proper standards to be published in a given journal. The peer review system is as good as any. If sometimes things go wrong, well that is because humans are involved.
As I said above.
It is a review to see if the paper should be entered into the official journal/scientific literature.
Nothing more
The people often doing climate science review are called “gatekeepers”
Well someone has to decide whether a paper is suitable for publication. If not editors and peer reviewers then who?
And the term “gatekeeper” is obnoxious, an insult to the hundreds of thousands of peer reviewers who sacrifice their time to keep modern science moving forward, and who do their job with integrity. Unless you make a similar sacrifice of time, I recommend you back off from making flippant insults.
Re: AndyG55
THEN we have those who rush to another’s defense, short circuiting ANY POSSIBLE OCCURRENCE of critique.
Ring any bells?
@Killer Marmot et al,
Chill dudes. AndyG55 was specifically referring to climate science reviewers as ‘gatekeepers’, not to reviewers in general.
Those who hold up “peer-review” as being the be all and end all of debate, especially in an area like “climate science™”, they are the un-scientific people, who understand very little about peer-review.
Much of what does get through is manifestly JUNK-science…
Much of the real science gets blocked.
Have you ever done peer review? Have you ever had a paper peer reviewed?
Reading your posts, I would guess no. I can tell you that it is not how you imagine it.
I second Killer Marmot’s statements. What you’re saying bears no relation to my experience over decades of reviewing and being reviewed – and I say that having had some fairly bad reviews from time to time.
And I should add that I’ve had some great (and severe!) reviews that made a hell of a difference in the quality of the final work. So there are people who really apply themselves to the task of reviewing.
Perhaps if someone could link to the peer review comments of an actual climate paper it would be informative.
Linking to those comments is probably considered a micro aggression as it exposes faults.
Much of the junk that gets through is not new junk, but rehashed and recycled old junk which is a by-product of peer review insisting on the kind of references it does. It doesn’t help when so much of what you can reference is so obviously wrong, yet believed by many to be gospel.
Microaggressions are fine, only macroaggressions concern me.
There should be at least some peer review comments that are publicly accessible.
The published responses to papers and the comments of the current GISS director, to the extent they reflect actual peer review, are bullsh*t.
If that is the best science can do, science should be taken away from scientists, and the grants given to engineers who at least have a clue what they are doing (or at least fund them to do peer review)..
But I would be delighted to see an actual sample of peer review that looks intelligent and proves me wrong.
AndyG55 July 3, 2016 at 9:55 pm: “Much of the real science gets blocked.”
How ironic it would be you who pen this …
wogga wogga sounds like the sound that PacMan makes
After a paper is published, a paper *should* then be subjected to a massive peer review by everyone in the field. If you want to see the quintessential example of such a review failing, research Michael Bellesile’s work. It exposes the group-think of an entire discipline (history). Even those in the same field of research accepted his claims and wondered how THEY could have been so wrong. The parallels with that episode and climate change research is frightingly similar on multiple levels.
As far as getting published, I’m sure most of you are familiar with the DDT-eggshell-thinning research that helped launch the environmental movement. The original researcher (Bitman?) could not get his corrected research, showing NO such thinning, published in the same journal as the orignal paper because it was at odds with ‘accepted science’ that was primarily based on his flawed research!
I can see some problems with that.
First, it would be nice if obvious deficiencies were corrected before publication. People reading journal articles could then focus on more more important criticisms than, say, that the article was poorly organized.
Second, it costs money to publish. Articles have to be properly formatted, and this takes up the time of paid employees. And most journals still have paper versions. Indiscriminate publishing would add to these costs.
Third, when I subscribe to a journal I don’t want to have to read through a lot of dross to find the gems. I am paying the journal for its selectivity.
Fourth, it would mean researchers could fatten up their resumes by spraying poor articles all over the place without regard to quality.
No, I’m not arguing against anything you wrote. I’m saying that even after proper peer review, once the paper is published, all those in the appropriate field *should* look at the paper *skeptically*, and weigh it against their own expertise and research. The Bellesiles debacle showed that when the results confirmed personal biasis, experts ignored facts as they, themselves, had thought them to be, and were willing to believe that their facts were in error. Rather like discarding all known evidence of the MWP based on a single paper showing Man is creating climate change.
Jtom – this comment is a couple days late! But hopefully you can follow up.
No, most of us readers do not know the original DDT article or articles, and we are not familiar with the debunking or correcting article or articles.
You can spin any story you want this way. Could you simply add a bit more info, for anyone who wants to investigate further?
Since you did have a name in your first anecdote, I was able to google. Bellesile is a historian. He takes some aspect of our culture, considers one or more explanations to account for why things are the way they are, then examines a set of existing pieces of evidence to see how well the idea is supported, or which idea is more supported.
That is not science. And, as far as I can tell, his controversial book did not go through peer review – he found a publisher of a history book and it was published; for political reasons, it was later discredited in various ways, and a panel was composed to address the quality of his work, long after publication.
And, this was all quite unusual. Not the regular peer-reviewed-journal article reviewing I get asked to do regularly, and do 5-6 times per year.
Please add just a bit more info on anecdotes or cautionary tales such as these. thanks!
who Ed Begley? The name rings no bells on this side of the Atlantic.
He is an actor:
But here’s a sample of his thinking”
I would love to see journals insist on at least one reviewer from outside their community; preferentially a professional in the basic fields of math and statistics, a professional engineer with experience reducing claims like the paper to practice.
“……….This failure in economic literature is relevant to climate science because…………;” because in neither economics nor climate science is it possible to perform controlled experiments to test the validity of any hypothesis.
Thus the most influential members of each field, by throwing their support to any particular thesis, to a great extent determine and define what is considered the “correct” thinking.
First, you have to find honest peers who refuse to play the you-scratch-my-back-and-I’ll-scratch-yours game. Then you have to find peers who are not fearful of being taken to court, either by Mr. Mann for implying his research might be flawed, or by liberal Attorneys General who could demand that you turn over 40 years of correspondence to prove you’re not in league with ExxonMobil or any such ilk. Those who claim there are no consequences for honest scientists who rock the boat haven’t been paying attention lately.
First and foremost, peer review is political. Not conspiratorial political, just personality political. Also, who you are matters the most, not what you have to say. Third, this article and some scanned over comments here appear to give the impression that people are waiting around dying to give reviews. The typical case is that getting a paper reviewed is tedious and time-consuming, there is almost never 4 reviewers, and a review I had for a paper appeared to consist of reading the first sentence of each paragraph then asking questions answered in the following paragraph. Peer review is just to try to stop blatant quackery, and obvious goofs. It does nothing else on a consistent basis. The issues are so complex and the data so convoluted and noisy that interpretations are really just opinions of what may be going on.
A Toronto based CBC science reporter decided to take his 5 year old son on a Western Canadian train trip. They shared a seat with an Albertan Civil Engineer. As the train entered Alberta, the boy saw a black Angus bull in a pasture. He was so excited that he exclaimed to his Dad “look, Dad the cows in Alberta are black! The Dad got all excited too and fired up his lap top and wrote an article for the CBC based on the questions “Why are the Alberta cows turning black? Is it climate change? What has Stephen Harper been doing to hide this from working families? What kind of taxes do we need to implement to make this stop?”
As he finished his article he asked the Engineer to peer review it for him. The Engineer took the lap top and tapped away for a couple of minutes and handed it back to the CBC Science guy. The revised article read “What we know so far is that there is at least one bovine in Alberta, it is a bull, and it is black on at least one side”.
Gerry
A Toronto based CBC science reporter decided to take his 5 year old son on a Western Canadian train trip. They shared a seat with an Albertan Civil Engineer. As the train entered Alberta, the boy saw a black Angus bull in a pasture. He was so excited that he exclaimed to his Dad “Look, Dad the cows in Alberta are black!” The Dad got all excited too and fired up his lap top and wrote an article for the CBC based on the questions “Why are the Alberta cows turning black? Is it climate change? What has Stephen Harper been doing to hide this from working families? What kind of taxes do we need to implement to make this stop?”
As he finished his article he asked the Engineer to “peer review” it for him. The Engineer took the lap top and tapped away for a couple of minutes and handed it back to the CBC science guy. The revised article read “What we know so far is that there is at least one bovine in Alberta, it is a bull, and it is black on at least one side”.
Gerry
Gerry, Now that is hilarious and thank for the laugh, but your statement is so true.
Bravo! Jolly good speech. Now if we can just get people to take our stuff as seriously as the stuff that did actually get peer reviewed, we can end the scourge of vaccinations being forced on us by the NWO and UN and…. er….. sorry, wrong blog… my bad.
😛
And let us not forget Poor Phil Jones, who once declared (inter alia):
Full quote, source (and more!) at Phil Jones keeps peer-review process humming … by using “intuition”
Huh? No code, no data, no due diligence? What are they reviewing, writing style?
How does peer review work if the peer reviewers are receiving larger grants for the same field of work than the person being reviewed hopes to receive.
Economics contains elements of science and math but it is neither math or science. Like an oil painting may contain the mathematics of perspective and the theory of color and light, it is neither math or science. It is not a bad thing, just that to me it is closer to liberal arts, such as the study of sociology or religion.
The problem is as with certain fields of science, many fields in liberal arts study has replaced academic integrity and ethics with activism and political expediency. Look at democratic campaign tactics for the usual suspects that depend on weak, shoddy, unethical or plain stupid academics.
I refuse to bow at the feet of the priests of climate change. God gave me common sense – and the power of observation. I see Winter, Spring, Summer, and Fall. Nothing has changed – and if we can deal with those extremes, we will have no problem dealing with a couple of degrees difference in temps. We don’t control the climate; it controls us.
99% of ‘peer review’ is about the thorny issue of ‘originality’ and ‘importance’. Very, very few papers are rejected because methodologies were plain faulty. Far more are rejected because the data is not sufficiently novel.
One particularly difficult position is when several groups are researching the same thing and all submit work to different journals for review. At that point, shenanigans can occur as certain folks scheme to ensure that their paper is published and their rivals’ are not.
There is just as much dirty politics in the peer-review process as in the Republican Party, the Democratic Party and on Wall Street.
I wipe my ass with peer-review paper
http://www.adweek.com/files/imagecache/node-blog/blogs/mckinney-toilet-paper-hed-2016.jpg