Guest post by S. Stanley Young and Warren Kindzierski
Back in April WUWT reposted several articles from Dr. Judith Curry’s Climate Etc. website on How we fool ourselves. One of the articles – Part II: Scientific consensus building – pointed out that… ‘researcher degrees of freedom’… allows for researchers to extract statistical significance or other meaningful information out of almost any data set. This past week WUWT reposted one of our articles from Climate Etc. about how epidemiologists try to fool us with flawed statistical practices.
The elephant in the room that drives these types of discussions is the irreproducibility crisis that afflicts modern science. This is no trivial matter. We believe this problem has – over time – infected a broad range of government regulation and policy that has enormously harmful implications to the American public.
These are the questions people should be asking… How many federal regulations reflect irreproducible, flawed, and unsound research? How many grant dollars have funded irreproducible research? In short, how many government regulations based on irreproducible claims harm the common good?
Irreproducible science
Academic researchers, grantors, and journals (and peer reviewers) all play a part in contributing to the irreproducibility crisis. We previously revealed that academic researcher incentives reward exciting research with new positive (significant association) claims—but not reproducible research. This generally encourages scientists in academia to wittingly or negligently use a variety of questionable research practices to produce claims that cannot be replicated.
Well-published university researchers earn tenure, promotion, lateral moves to more prestigious universities, salary increases, grants, professional reputation, and public esteem—above all, from publishing exciting research. The National Academy of Sciences, Engineering and Medicine (NASEM) recently reported on a survey of academic researchers. NASEM indicated that researchers themselves believe they are responsible for addressing issues of reproducibility, but that a supportive institutional infrastructure (e.g., training, mentoring, funding and publishing) is needed.
That is all we need, more tax dollars used to fund academics so that they can learn how to do the job they are supposed to know how to do. What does that tell you about the current academic mentoring process? Do not look to university administrations to want to help solve the irreproducibility problem. American university administrations want to host grant-winning research, from which they profit by receiving overhead costs—frequently a majority of overall research grant costs.
The irreproducibility problem extends outside of academia. Our experience is that there will be little incentive to change current academic researcher practices unless there is also a commitment to change how grantors and journal editors (and reviewers) conduct their end of the research publication process.
Grantors (funding agencies and research sponsors) tend to emphasis novelty of research. These organizations want to fund the same sort of exciting research. Plus, government funders possess the added incentive that exciting research with positive results also supports the expansion of their organizational mission. The downside is that this encourages many researchers to further fiddle with their research until they come up with a publishable, novel claim (publish or perish) that still will not replicate.
Journals too seek novelty in research in part because of the competition for impact factors. An impact factor is a measure of the citation frequency, which offers a quantitative metric for evaluating journals and assessing their relative influence in the scientific community. The same incentives that drive academics affect journal editors, who receive acclaim for their journal, and personal reputational awards, by publishing exciting research—even if the research has not been vetted thoroughly. Editors are often rewarded for actions that increase the impact factor of their journal. Research which reports novel findings are more often highly cited and thus contribute to the stature of a journal and greater rewards for journal editors.
Academic researchers, university administrations, grantors, journal editors—each has an incentive to seek out exciting research that draws money, status, and prestige, but few or no incentives to double check their work. Above all, they have little incentive to reproduce the research, to check that an exciting claim holds up—because if it does not, they will lose money, status and prestige.
Compromised US government regulation
As we had previously discussed in our Climate Etc. post, how irreproducible science afflicts government regulation can be revealed by looking at how research is used by the Environmental Protection Agency to regulate outdoor air quality. Over the past 40 years the EPA has continuously considered and imposed increasingly restrictive regulation of air quality.
This is not necessarily bad. The EPA must develop air quality criteria as part of their mandate, informed by expert opinion, and describe their effects singly and in combination on the health and welfare of American citizens. Also, the EPA must set National Ambient Air Quality Standards (NAAQS), as a yardstick by which states and localities can measure their own air quality, and as a legal requirement for enforcement of NAAQS.
However, the debate about whether the EPA should make a particular regulatory decision based on scientific evidence raises questions central to the irreproducibility crisis—data accuracy, research protocols, statistical analyses, publication bias, sponsorship bias, etc. Complicating this are bureaucrats in government who fund research—these bureaucrats depend on regulation to support their existence. Why would they want to acknowledge an irreproducibility crisis if it were to affect their careers?
There is a burden to society that lurks behind the intentions of academic researchers and government bureaucrats who use their research to develop regulation. They collectively skew the process towards justifying increased regulation. The costs of insufficiently substantiated environmental regulation cannot be dismissed.
An example is estimated costs requiring ships to use “cleaner fuel” with less sulfur to reduce sulfur dioxide emissions. The EPA argues that the move to low-sulfur ship fuel could prevent 14,000 deaths each year by 2020 based on epidemiologic research that sulfur dioxide kills. Let’s ignore the large uncertainties around their estimate (which must be taken with a chunk of salt). Further, the EPA estimates a value of statistical life at $7.4 million (in 2006 dollars). Let’s also ignore large uncertainties around this estimate.
Using EPA’s numbers and roughly factoring in 2006−2020 inflation at 28%, health-related benefits are inferred to be more than US$130 billion per year in 2020 (14,000 x $7.4 million x 1.28) for this requirement. This may sound convincing to an environmentalist, but is it real? A growing body of research fails to support the EPA position. In particular, there is research providing evidence that sulfur dioxide in outdoor air is not associated with death, heart attacks, asthma or lung cancer.
Extensive regulatory schemes always come with excess costs. As to the true excess costs requiring ships to use fuel with less sulfur, who know? However, these are not zero. Fuel costs represent as much as 50-60% of total ship operating costs, depending on the type of ship and service. On the other hand, there is a distinct possibility that inferred health-related benefits (i.e., statistical lives saved) from this requirement are meaningless.
The EPA issues an extraordinary number of regulations, which affect every area of the economy and constrict everyday freedoms. Extensive regulatory schemes can amount to a competitive advantage for large companies over small ones. Large companies have greater capacity to comply with an extensive regulatory framework. Furthermore, regulatory costs are ultimately borne by American consumers. These costs can also have negative health implications, such as those that follow from increased unemployment—e.g., depression, substance abuse, domestic violence.
Development of air quality regulation by the EPA can be shown to reflect irreproducible, flawed, and unsound research. How many other federal regulations suffer this fate? That is the trillion-dollar question.
S. Stanley Young is the CEO of CGStat in Raleigh, North Carolina and is the Director of the National Association of Scholars’ Shifting Sands Project. Warren Kindzierski is an Adjunct Professor in the School of Public Health at the University of Alberta in Edmonton, Alberta.
Irreproducible science is actually rewarded at EPA and many government agencies in the modern agenda science era.
Is it really science?
Or scientism?
If it cannot be reproduced then it certainly is not science.
Most of the time replication is not attempted.
The one field where replication is routinely attempted is biomedical science. Drug companies scan the literature for interesting findings that might be used for new drugs. The first thing they do is try to replicate the experiment.
The book Rigor Mortis documents mainly the work of Amgen and Bayer to replicate published research findings. Amgen has found that as many as 90% of published research findings are wrong.
Let that sink in a bit. The overwhelming majority of published research findings are wrong.
Using unreplicated scientific research for ANYTHING at all is criminally negligent.
The replication crisis isn’t a secret.
The bureaucrats who base regulations on unreplicated research should be charged with criminal negligence. They know, or should know, about the replication crisis. They should be in jail.
I have a PhD son in virology research. He says research findings can be correct. The problem is that almost none of the research papers give enough specificity for all factors to allow experimental replication. In many cases this is done on purpose so that others can’t profit from the research. Think of an experiment using mice as the subject. Unless the dna of that mouse is specified perfectly and is from a strain that can be obtained for replication, later replication experimental results can vary wildly. It gets even more problematic when you have multiple process steps that must be duplicated perfectly. Leave out one step and the odds of replication goes down significantly.
I agree that regulations based on unreplicated research should have the regulators charged with criminal negligence.
I think you just raised the most salient point in all of this: How does one allow others to reproduce your results, without giving away your patentable innovation? I myself would only bother patenting a thing that cannot be replicated, what is the point in describing every detail to your competitors?
I do have one caveat: If your research is funded by taxes, even partially, or you use a public asset, like a building or state resources, or the ISS, or you are funded by public donations you publically solicited in a public campaign, then whatever you come up with, belongs to the public.
That way, we can then insist the inventor tours other publically funded institutions and individuals, where s/he will help said public servants to replicate the procedure, then all this knowledge the public helped pay for, can be tested and verified and published into the public domain.
As a side bonus, the threat of public humiliation will motivate the diligent scientists, and weed out the ones who get by now, by dressing up drivel in pretty language until his* (or her) peers cannot discern bad science from “exciting progress”.
*One problem with peer review is, my own personal “peers”, who would have, by definition, as much knowledge as me, cannot really crit my work, because, they don’t know any better than me, not so?
“Patent” means “open”. The whole point of patents is to publish details of your work so that one “learned in the art” can replicate your work.
“Think of an experiment using mice as the subject.”
I can think of one where the subjects were used lab mice – as in, had already been used in other lab experiments.
One area that needs much closer examination is the whole peer review process. If the name of those doing a peer review are not attached to the article they have reviewed, there is no way to discredit them because they have not done their homework properly.
I have seen open source journals that not only give the names of the reviewers but some brief comments – both positive and negative. This is to be commended and the scientific community can add their contribution.
However, for the top scientific journals making money is a priority. If a scientist submits an article and this is rejected without a single comment, the journal should be boycotted. If there is the briefest rejection comment – which indicates that neither the editor or reviewers have a clue about the content with no option for the researcher to question and get clarification – the journal should be boycotted.
It does appear that there is an old boys club when it comes to getting work published. If you are not part of the club or do not have someone in the club pushing the journal to publish your work, you are unlikely to get it published. This applies to the top journals. It is also offensive when a top journal rejects a study by a scientist but tells that person he or she can get it published in their second rate journal. The researcher has to pay the journal but it also has a much bigger captive market of university libraries.
Nothing changes.
In academia “wisdom” dictates.
In grade 1, in the morning line up outside the classroom before entering, I spilled the beans…
I told the class there was no Santa Claus.
Several classmates, bordering on tears, “ratted” me out.
Miss Allen, our teacher, told me not to say that anymore.
I agreed.
To this day, I remain.
Thank you for that beautiful little poem! My heart is ajoy.
Who told you there was no Santa Claus?
I do believe one of the very first articles I read on this site (or the site that sent me here) was about how some of these journals are happy to accept papers accompanied by a little ‘contribution’. The size of said contribution determines your column space, whether you can have coloured graphics etc. Apparently they employ people for their review process?
And, of course, you are perfectly right. One becomes known by becoming known to others who have already become known. Or, even faster, knowing those in the know, who made those others so well known…
Interesting turn of phrase. One of the side effects of an Old Boys Club is that people support people like themselves.
I wonder what the actual gender bias is for authors of published papers in various fields?
If such a bias could be statistically detected it would certainly be illegal. And the Journals would then need to address the peer review process.
Let’s not get silly here. Today the “Old Boys Club” is not about being male, or old. It is referring to a group of people at the top of their influence working together for their own benefit. In some academic areas, such as social welfare, the “Old Boys” are primarily women and people of color.
Precisely. Projections of future temperature increases cannot be validated EXCEPT by a) waiting for X number of years and comparing model predictions with actual temperatures as time marches by; or b) a reasonable test would be to compare recent history or even recent decadal history with what the best models today would predict for comparison with the actual temperature record over this time. There is a question on which historical data to use in the comparison – issue here is the official numbers have sometimes been changed to “hide the decline” talking about changes led by Thomas Karl e.g., in in 2015 (just before the Paris Climate meeting)… Science Express 4 June 2015, published online. Backcasting (option (a) is a very good approach since underlying conditions and drivers in recent history should be similar to those during years being projected. As a general comment, looking at the CMIP model projections in the past several decades vs. actual and future projections makes it difficult to not question the modeled future projections. This is also one of the key points that Steven Koonin is communicating right now.
“Projections of future temperature increases cannot be validated EXCEPT by a) waiting for X number of years and comparing model predictions with actual temperatures as time marches by; or b) a reasonable test would be to compare recent history or even recent decadal history with what the best models today would predict for comparison with the actual temperature record over this time.”
The problem for us today is those decadal temperature histories have been falsified by climate change data manipultors to present a false picture of the Earth’s temperatures. The people of the Earth are being lied to by the scientists the people are counting on to tell them the truth. Instead, the people are being lied to and scared to death by unscrupulous people with a climate change agenda.
True, accepted without reservation. But university department heads earn their place, and sometimes published lead researcher credit, by raising funds.
What we learn from history is that we do not learn from history. Nixon’s War on Cancer was based on a model that made eugenics look clear and rational.
A major mistake Nixon (among many) did was to make the EPA a separate agency, resulting in the “special prosecutor” syndrome of not knowing when to stop and a total lack of balance. The PM2.5 rules are an example of that sort of thinking, in they want to stop “pollution”, and will look very hard to find a rationale for a restriction.
A biologist I know has commented that historically the most glamorous research often proves to be the least beneficial. What appears to be mundane or even boring often gives far better value for money but these researchers have the most difficulties in finding financial support. The media does not help them either by backing the former.
Take the whole area of AIDS research. There have, over the past 20 years, been numerous exciting reports of breakthroughs that have disappeared as quickly as a morning mist. This also applies to research on “catastrophic” climate change. However, if you want to find funding for research showing that climate alarmism is a gross exaggeration, neither government nor private funding will be interested.
Now attach that thought to yesterday’s article about the “lawsuit” against Big Oil. Should Exxon and BP et al not be spending money on public education by now? Why are they being complicit? But this disregards the previous evangelic campaign, No Smoking, where people got real hurt for real crimes of conspiracy. So why is Monsanto (exact same crimes of omission) immune, but Big Oil not?
Is it because Monsanto is already under the One Account, and there are still too many operators in the energy business? Can we expect a slew of oil companies going bankrupt for various reasons soon? Ach, wot am I saying, it’s already happening for some times already yet.
That would be mighty powerful evidence for the theory that every single cent in the world is being collected into that One Account. I mean, it IS evident that climastrology is highly effective at empoverishing all social classes below Baal Gates & cie. yes? Gang Green is like a diode rectifier in the money flow…
And if science is reproducible, it isn’t accepted:
On 22 March, the European Medicines Agency (EMA) issued a statement that, after reviewing the evidence, they recommend against the use of ivermectin for the prevention and treatment of covid-19, outside of ‘well-designed’ clinical trials. The EMA claims evidence from laboratory studies, clinical trials, observational studies, and meta-analysis, but provides no sources, specifics or citations. We fill these omissions below.
British Ivermectin Recommendation Development Panel – Response to EMA Statement on Ivermectin for Covid-19
Until these agencies are challenged on their lack of compliance to congressionally mandated quality standards, this garbage will continue. OMB specifically points to problems in the improper utilization of “Peer Review” on any science produced by, or referenced by government agencies on issue that will have a significant economic impact.
Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency (epa.gov)
Opening Summary
OMB Guidelines In Section 515(a) of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554; H.R. 5658), Congress directed OMB to issue government-wide guidelines that “provide policy and procedural guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by Federal agencies….” The OMB guidelines direct agencies subject to the Paperwork Reduction Act (44 U.S.C. 3502(1)) to:
• Issue their own information quality guidelines to ensure and maximize the quality, objectivity, utility, and integrity of information, including statistical information, by no later than one year after the date of issuance of the OMB guidelines;
• Establish administrative mechanisms allowing affected persons to seek and obtain correction of information maintained and disseminated by the agency that does not comply with the OMB or agency guidelines; and
• Report to the Director of OMB the number and nature of complaints received by the agency regarding agency compliance with OMB guidelines concerning the quality, objectivity, utility, and integrity of information and how such complaints were resolved.
Anthony Watts also addressed the problem of poor quality in government research in 2008.
ISO-8000 Data Quality – something climate science could benefit from – Watts Up With That?
This post does a good job of laying irreproducible motivations. Journals want exciting results. Academics need to publish or perish, and publishing mainly reproducibility likely means perishing.
It focuses less on the means by which irreproducability results. That is the focus of this comment. I think there are four, the first three of which comprise at least borderline academic misconduct.
I think most irreproducibility arises from category 4 belief bias.
But for sure, in climate there is also a lot of provable academic misconduct, with many examples in Blowing Smoke. Not just Marcott’s deceptively redated coretops. Essay ‘By Land or By Sea’ exposed misconduct concerning WAIS instability during the Eemian. Essay Shell Games exposed misconduct concerning ocean acidification impacts on corals and oysters. Essay Burning Nonscience exposed misconduct concerning biome changes along Arizona’s famous Catalina Highway (attributed to global warming but actually caused by two massive forest fires in consecutive years, one on each side of the highway). Not good looks for climate ‘science’
EPA has just changed its “High and Low” U.S. temperature data chart going back to 1900 to hide the 1930 period and inflate more recent time periods. They have also invented a new “ Heat Wave” chart showing heat waves in the U.S. climbing continuously for decades.
looks like “Biden” climate science is now in full charge.
Someone should ask the EPA to explain why they chose to change all their charts.
It seems just about the sole focus of the Biden administration is to try to fool the American people into believing things that are not true.
The Biden administration lives in a False Reality and they want the rest of us to live in it, too.
No, thanks. I’ll take reality every time.
How do you replicate the Social Cost of Carbon and other GHGs that EPA uses to justify its climate regulations?
OMB taking comments on the social cost of GHGs
By David Wojick
The beginning:
“Biden’s Office of Information and Regulatory Affairs in OMB is taking public comments on the regulatory use of the world’s strangest numbers. These are the so-called Social Cost of Carbon and other Greenhouse Gases (GHG). The Social Cost is the supposed future climate change economic damage due to our near-term emissions. The numbers are huge because the Biden government projects endless damages for the next 300 years, from every ton of carbon dioxide, methane, etc., emitted now. Yes they have to go out 300 years to get the damages.
This opportunity to comment is important because the issue is the use of these looney SC-GHG numbers to justify massive climate change regulations that will govern our lives. Preventing these huge SC damages is presented as the benefits that justify the regulations in the required cost-benefit analysis. That SC-GHG is junk needs to be made clear, and stopped. It may be up to the courts to do this. One of the rules of the regulatory game is that if you want to ask the court to stop a regulation, you must have first put your objection on the record, in the comments. When the Agency ignores your objection, then you can sue them. This is called exhausting your administrative remedies.
So here are some objections that people might use. The more comments the better.
To begin with, the SC-GHG computer models are obsolete. The Feds use the results from three different “integrated assessment models” or IAMs. These models are now about 20 years old. The US alone has done something like $30 billion in climate change research since they were developed. The major climate models have been steadily upgraded, with a new generation every five years or so, but not the IAMs. Thus one could ask the Court to require the Agencies that want to use SCs in their cost benefit analyses to build new IAMs.
But it gets deeper than that, because some of the research calls into question the very feasibility of calculating a likely SC.”
There is a lot more in the article. See https://www.cfact.org/author/dwojick/
You do not have to be a US citizen or organization to comment. This is about the science.
I watched this system develop, individual researchers lost control over their research. Can’t get funding, experienced it, heard it from others. Even good peer review may not solve that. How many researchers, who should know their research best, suffer from this?
Stanley and Warren,
Yours is a timely contribution, thank you. Please permit me to endorse & expand.
Good science is used extensively in one of my earlier occupations of mineral exploration.
In mineral exploration there is little to no concern about the reproducibility crisis because it has never existed to any harmful extent.
Compared to what you write about this crisis in EPA work, there is a dominant, large difference. It is named “accountability”. In mineral exploration, those who fail to find new ore deposits come to be paid less than those who succeed. Researchers are accountable for their work.
Where is there an equivalent in your EPA cases? Have you any knowledge of government or academic scientists being rewarded because their research was shown to have been of a clear, measured benefit to society? Conversely, is anyone punished for a poor or fraudulent effort?
In mineral exploration, there is a dominant theme of honesty that says “There is no value in covering up poor data, no value in fabricating or adjusting measurements, no value in even putting forwards a shonky case for funding to those who hand out the project funds. You will be caught out. Sure, there has been an occasional case of people salting gold mines and so on, generally detected very quickly because of this honesty approach and usually not involving scientists. Jail terms for fraud like this have happened.
Success is definable. Either you find a new mine or you fail to.
Where is the equivalent in the EPA case? There are strong suggestions that some researchers who have produced publicised, counter-intuitive, game changing work that has later been retracted have gained in personal eminence when they should have been jailed. That they have not been is but one measure of the lack of accountability that is a key to fixing the reproducibility crisis.
There are known ways to counter the crisis. One way is to make the science more stringent in terms of quality and inspection of that quality.
A simple example is the non-use or wrong use of boundaries around measurement errors and uncertainty. Topics like confidence intervals are seldom treated properly in environmental research, but there are a few papers where it is done properly and well. The environmental research community must know about this topic because some of their mates do, but many remain too lazy, too poorly educated or simply fraudulent. For measurement errors, there are standard publications like the French-based Bureau of Weights and Measures to name but one and there are some attempts at national regulatory requirements that seek to impose a minimum quality on uncertainty estimates. These do not seem to have worked. Proper quantification of real errors and uncertainties would have led to rejection of more than half of the environment research papers I have read personally. They should never have made it to the print stage.
Unless accountability of research and its consequences is strengthened, where people are rewarded for fine work (the Nobel prize concept, but wider) and punished for poor work by demotion, lack of pay rises, sacking or jailing for fraud, then little progress will be made.
My preference, FWIW, would be to start with a procedure like measurement errors, make it a poster child to show where improvements are required, institute reasonable and practical mandated reporting and quality control requirements and dismiss those who fail them. This is not cruel, this is the way that the research community should advance unless its wishes for the horrible consequences of disorganised rabble run by intellectually crooked profit takers. It is like natural selection at work.
A final note – the buck in the EPA case needs to stop with (a) those who authorise research grants and (b) those who pass bad science into regulation and law. Both of these groups need a thorough audit and culling to stop them generating more damage. Throw out the incompetents. It is not hard to find them.
Meanwhile, enjoy the fruits of our hard, quality science in finding new mines for materials that you in wider society ask us to provide. Geoff S
For any system to function,”For the good of the people” you need honest bureaucrats..
Good luck with that.
We have observed the reversal of our imposed standard,we had demanded our policy makers use evidence,to justify the policies they proposed..
Now that is just so much work.
Especially if the planned policy has nothing incommon with the stated intent.
So as our helpers bent so did their standards .
Policy based evidence manufacturing is so much easier,predictable and reliable..
Good enough for government even.
In fact desired by government.
As it frightens every natural bureaucrat to ask questions for which they have no answers,as we have all seen with the pathetic theatre of Government Enquiries,which amazingly seem unable to see any facts, faults, or malice from their own ranks..
Not to mention the absolutely absurd PM2.5 nonsense. . . . . .
The PM 2.5 which has a proven impact on health and mortality?
(I rely on detailed UK research about that…)
Pray your property is never identified as having been graced with a Vernal Pool.
https://www.epa.gov/wetlands/vernal-pools
Or as my New England friends call them, wicked big puddles.
As a consulting forester in MA, I have to deal with regs over VPs. Here, the state says a logging job can’t get close to a VP. Fair enough- but in the first decade of this century, the state did some terrible logging on state forests- massive, mostly unnecessary clearcuts. I went out with some critics to see just how bad that work was. On one state forest I saw where a log skidder drove right through a VP. If that had happened on one of my projects- not only would I have lost my forester license but probably gotten some jail time. Yet, none of the state people were punished- it was all hushed up.
Consider, say, a $100 000 fine for spilling some substance. The small business immediatley goes belly up, while the multinational will just plainly ignore the fine, then sic one of their retained lawyer apprentices onto the regulator, then have it discounted, and after seven years, either everyone forgets about it, or the fine does get paid. Enter the bevy of shysters chanting “GAAP-GAAP-GAAP” until the fine turns into a “cost of doing business” and the taxpayer ends up subsidising the fine, the lawyer (which “costed” ten times the original fine) plus we pay for the original spill…
Regulation is necessary, but regulators are a breed of parasite that kills off small businesses to protect the interests of The Holy Investor. Note that it is the multinationals who campaign hardest for regulation, screaming “unfair interference” all along the way. After all, they literally own the politicians who regulate the regulator…
This subject has been discussed many times. If public money is used to fund research then that research needs to be fully documented.
As a minimum, I would expect all data to be available. If not then funds withheld.
I would expect all data processing to be run by a commented, single script. If not then funds to be withheld.
These simple measures will clean up a lot of research results.
It becomes more difficult when, as suggested earlier, that some data gathering requires or uses something unique.
In this case all I could recommend would be to video the data capture/experiment.
I understand that most journals require data to be available prior to publication. But many fo not enforce this rule.
How can you do a peer review without access to the data and the script(s)?
Correct. More generally, given that the motives of government are not magically devoid of the same incentives that are attributed to the private sector, it is imperative that any government funded science that just happens to recommend an increase in the power or regulatory scope of government must be reviewed and verified by independent researchers in the private sector.
…and then we have an open referendum on it. Because those “independent researchers”, as we have here and elsewhere shown, do not exist!
One issue with the US government’s SO2 health calculation is that SO2 produced near people might affect them. However, most people do not live near ships, especially at sea. As SO2 is quickly absorbed by water, any SO2 produced while under way is never going to hurt anyone except perhaps the sailors onboard. As this is at least 90% of the fuel consumption (100% in some ports that require the ship to power down its engines and plug into electricity) the analysis is fundamentally flawed, even if the other values were accurate.
Another example is the PM10 data which were falsified for years.
My two cents. Government regulations based upon “studies” must have either:
1) Sunset rule of 3 years without further studies confirming the goal of the original regulation has been met. IOW, if air quality is to be better, then proof the regulation met that goal must be provided.
2) Studies used in regulations must have independent replication performed if they are to bypass the sunset rule. This includes public release of all data used and modified data must be documented in each and every instance of change.
Jim,
You stated this better than I did. There is a definitely a need to limit an administrative state that can promulgate laws outside of the legislative process. I think setting up a red team / blue team to verify the science would work. In retrospect, I think Trump tried to go this route to some extent with climate and the pandemic, but was foiled by the allied forces of the bureaucracy, dems and corporate media.
The left’s march through the institutions has been bearing fruit for quite some time now, most notably in the EPA. They pass “regulations” which are in fact laws and then use governmental force to compel citizens to follow these illegal laws. If it does not originate in Congress it is illegitimate.
If it is not reproducable, is it science?
Or is it merely someone’s story about what they think they see?
Where are the canals of Mars today?
Well, I might call that merely an observation not science. Is it science to observe the landscape around us?
If I am not mistaken, all those “colorful” “space” pictures are actually colorized grey scale images from digital transmissions.
OK, not “all”. I’m not interested enough to make the time to spend several hours researching the topic of “space photography”. I recall coming across the “colorized” claim while reading something. Put it on my “To Do” list, but haven’t followed up yet.
This situation of non-reproducibility sounds like an extended form of Iron Triangle (see below) in which parts of the academic scientific community have been drawn into the morass.
The Iron Triangle is a well-known phenomenon related to policy making. It is shown in diagram form at several places on the internet and is described in written form by Endress in [Ref.1]. It is worth quoting at length from the latter. After describing the first- and second-best levels of policy making, Endress continues:-
<<Third-best is the world of political economy, wherein costs and benefits directly influence the formation of coalitions that compete for political and economic advantage in society. The pursuit of such advantage is called “rent-seeking” in economics and typically involves activities such as lobbying, public relations campaigns, political contributions, and, sometimes, outright bribery. Unfortunately, the expansion of government that accompanies intervention on second-best grounds can facilitate rent-seeking at the third-best level … A particularly powerful type of rent-seeking coalition, long studied in political science, is termed “the iron triangle” because of the strength of the collaborative relationships among a triad of actors: politicians who seek campaign contributions, votes and reelection; government bureaucrats who aspire to expand fiefdoms and budgets; and private sector interest groups who seek special privileges in the form of political access, favourable legislation, subsidies, protection of monopoly positions, and lucrative government contracts. The iron triangle is durable and impenetrable because it functions as a highly efficient, three-cornered, rent-seeking machine.
Nowhere (except perhaps in healthcare) do third-best politics sink first-best and second-best economic considerations as deeply as in the realm of energy policy. In assessing energy policy in Europe and the United States, Helm (2012) is especially critical of policymakers’ obsession with current technology renewable energy, which is not yet commercially viable without government subsidies and mandates … Consequently, renewables have remained ineffective in lowering energy prices, creating green jobs, and reducing carbon emissions worldwide. The result is high costs for little gain. In a review of Helm’s book, “The Carbon Crunch,” The Economist … highlights Helm’s observation that the entire renewable sector has become an “orgy of rent-seeking.” This outcome is not compatible with the sustainability criterion.>>
Once there is broad political agreement as to what the problem is then solutions can be sought. The current problem, however, is that a large part of the political spectrum in the West is happy with the current settlement. For them there is no problem – the rest of us just have to foot the bill.
Reference
Regards,
John.
“studies show…”
Yes, they do.
I think there is a simple solution. All journals, universities, government agencies and corporations should be forced to keep track of and publish all attempts to reproduce the research they have paid for or published. Most importantly how many times it was successfully reproduced. This should be no more difficult than tracking citations.
One of my favorite professors at Purdue, back in the late 1970s and early 1980s, was John Robert (JR) Osborn. He was a real wiseacre, which many of us students found endearing (professors found it annoying, maintaining that he “never grew up” – yet another endearing characteristic!). During class discussions, students would sometimes give “examples” of phenomena that were clearly bogus, usually because the person had misunderstood something he had read. When other students called him on it, JR would always say, “I believe him. In fact, I read it in the Journal of Irreproducible Results.” It was quite some time before I learned that there actually was such a thing. The internet version is, I believe, defunct now. But it was there for a while. It was a parody scientific publication, in case you hadn’t caught on to that.