On Dec. 10, Randy Schekman, a UC Berkeley professor, was awarded the Nobel Prize in Physiology or Medicine. The day before, he published an op-ed in London’s Guardian, titled “How journals like Nature, Cell, and Science are damaging science,” in which he announced that he will henceforth refuse to send manuscripts for peer-reviewed consideration to these prestigious science journals.
Schekman’s accusation is that these journals are distorting science by being biased towards the “flashiest” research, i.e. papers that generate headlines such as “Global Warming Will Kill Billions, Scientist Finds,” rather than the best research.
This matters more than one might think, because governments and universities disproportionately make their award and funding decisions based on the research published in the prestige journals.
So, if Science and Nature differentially publish flashy research, and publishing there will deliver funding and tenure, scientists are naturally going to gravitate toward trendy topics and produce flashy research. It’s a cycle that perpetuates Armageddon-style headlines that compel politicians to disburse more money, for more research, ultimately buying a beach house for the doom-saying scientists.
This leads to the question: do the journals’ propensity for flashy research result in biased research?
Unfortunately, yes; especially when it comes to climate science.
Just take a look at Science’s “Perspectives” pieces, which are really opinion pieces posing as literature reviews. Despite the fact that global warming has been prominent for about 25 years now, Science has yet to publish one Perspectives piece summarizing the body of refereed science indicating that far too much warming may have been predicted.
That should not be the case, because every new forecast of climate change should have an equal probability of producing a more or less dire result. That’s what happens with weather forecast models as new information comes in. Once it has been established that increasing atmospheric carbon dioxide should raise surface temperature a certain amount, each new piece of information should either raise or lower the forecast.
But scientists aren’t incentivized to look under the less-flashy rock. In fact, they threaten their own well-being if they do.
If scientists aren’t doing their due diligence, is Schekman right that the journals aren’t doing theirs either? It’s easy to find out. I reviewed 13 months of both Science and Nature, and sorted every article or story about climate change or its impact into three piles: worse, better, or neutral compared with previous studies.
Of the 115 entries, 23 made the “neutral” pile, 83 were in the “worse” stack, and nine were in the “better.” The probability of the journals not having a bias is as likely as a coin being flipped 92 times and showing heads or tails fewer than nine times.
The number is: 100,000,000,000,000,000.
You can look this up in a binomial probability table, which shows the average number of times you have to flip a coin 92 times to get this result.
The obvious “publication bias” by these two journals is very troubling, because the resultant public funding and tenure could have some pretty nasty consequences.
This creates horrific effects, especially when the issues are policy-related. Summaries of the scientific literature are used to guide policymakers, but if the published research is biased, then so must be the summaries; leaving policymakers no option — not being scientists themselves — but to embrace what is inevitably touted as “the best science.”
Recently, the United Nations’ Intergovernmental Panel on Climate Change released its fifth “Scientific Assessment” of climate change, which is, in effect, a massive literature review. Since the most prestigious journals carry the most weight, the literature that is reviewed is itself biased. The result? Even the most accurate and comprehensive review must create a biased picture.
The result is very bad policy: cap-and-trade schemes, carbon taxes, and ugly windmill and solar arrays that produce little power but appeal to the politician’s need to “do something.” All ultimately driven by scientists behaving rationally, but badly.
###
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Very clearly put. I expect Michaels will now be subjected to another round of character assassination.
Hi from Oz. Worrying indeed, Patrick, but what the hell can we (sceptics) do about this?
“… because every new forecast of climate change should have an equal probability of producing a more or less dire result.”
This strikes me as not-necessarily-so:
1. IF the initial forecast is essentially correct, AND the new forecasts are essentially done the same way with similar data, then the “problem” left is precision or just the normal variability, the chaotic part. Then the new forecasts should bounce around the initial forecast, some higher, some lower. (Even this assumes that the errors are unimodal and symmetical around the centre.)
BUT
2. IF the original forecast is based on different data or assumptions/models, then the new forecasts can go either way without a trend.
3. IF the original forecasts are based on assumptions that are fundamentally incorrect in some key fashion, and new forecasts are more correct, then the new forecasts can go either way also.
4. IF the original forecasts are based on assumptions that are fundamentally incorrect in some key fashion, BUT new forecasts simply tweak those assumptions in a similar direction each time, then the new forecasts will be progressive, headed in a similar way but again, in either direction.
There are many reasons why additional studies can be more alarming as time goes on. The fact that they do does not mean anything necessarily about their correctness or bias in assumptions.
and this will change when the money dries up
Very good article Pat.
Best, Allan
@Doug Proctor
…“… because every new forecast of climate change should have an equal probability of producing a more or less dire result.”.. This strikes me as not-necessarily-so:
Indeed.
In fact, I think there’s a simpler reason for the ‘bias’ – it’s probably that the numbers of papers submitted claiming that things were getting worse were 90%, and those claiming things were not so bad were 10%.
Under those circumstances it would be quite reasonable for a magazine to publish reflecting this number.
of course, it may be that equal numbers of equal quality are being submitted, and the magazine is only choosing the ‘worse’ ones. That would be bias. But you can’t tell that from the data you have. What is quite probable is that some crowd following is going on, and people are realising that if you submit ‘worse’ ones you are more likely to get published – so that’s what they submit. It would be very hard to establish bias unless you had full unfettered access to the paper’s publishing policy.
“All ultimately driven by scientists behaving rationally, but badly.”
Well said, sad as it is.
@John Gardner says “Worrying indeed, Patrick, but what the hell can we (sceptics) do about this?”
Answer: Just keep chipping away. The cracks in the Great Scam structure are widening at speed.
As someone from Berkeley (1960s era) that is the last place I expected to see someone take an honorable position on climate matters. Well done, and highly appropriate. Fan boys don’t belong in science and that is the culture those publishers attract.
Doug proctor is correct.
In short you need more information to show bias.
And the money will dry up. They can’t keep this ponzi scheme going on for another two decades, voters are bored, surface mean temps have stalled, winters are back with a vengeance, Antarctica is hanging cool, the Arctic is unsure, polar bears are thriving, penguins are still around in their millions, the US is freezing right now (snowfalls thing of the past), there is no trend in extreme weather allegedly caused by man…………..
I’m growing increasingly bored too, but I cannot stop now until the end of the game.
We are currently here:
It is a No Brainer; these magazines are abusing the reputation of science.
That so many scientists sit on the sidelines, ignoring the bad apples in their ranks, shows a lack of commitment and passion to their careers and beliefs. Thank heavens for Randy Schenkman and other scientists who show they do care and do speak out.
One day, maybe a team of sceptical climate scientists will win an Ig-Nobel Award.
Hi Patrick, great article.
If both within academia (via grant awards / status) and without, climate narratives are rewarded more for their ability to prosper (‘flashiness’ as you describe) than for their verifiability (factual content), then due to differential selection over the long-term, sensational / alarmist narrative aliances will form the kind of well characterised cultural entity that we now actually see with CAGW. I.e. consensus culture, ‘believers’, altered perceptions of reality (in this case corrupting proper science), mobilisation of society and infra-structure to ‘the cause’, and so on. Darwinian views of cultural evolution lend insight into this process, especially memetics. See:
http://wearenarrative.wordpress.com/2013/10/27/the-cagw-memeplex-a-cultural-creature/
For the record, what was the date-span?
Just to play devil’s advocate here. 83 studies showed it was worse because that is what their data show? I think you need to look farther than just plain numbers and conclusions. Perhaps most were in error? Who knows.
PS. I think Dr Libby will have her work from the 1970s shown to be correct and lord help us all in that case.
Flash, meet pan.
Not a scientist but have some expeirence with politics. AGW fits nicely into the old miltary industrial complex model. So nicely that it sprouted from the same labs. It is not a coincidence that nuclear science funding was drying up just as climate science funding was gearing up. The funding is the thing as you write.
“If scientists aren’t doing their due diligence, is Schekman right that the journals aren’t doing theirs either? It’s easy to find out. I reviewed 13 months of both Science and Nature, and sorted every article or story about climate change or its impact into three piles: worse, better, or neutral compared with previous studies.”
Um, a few details here would go a long way in convincing me you have any idea what you talking about. If you are doing citation analysis, older papers always do better than recent papers. But then, I am ony guessing what you did.
Schekman has his point in his rarified world, but he got _his_ nobel on his Nature and Cell papers, so easy for him to say. Most granting agencies accept papers in the secondary “specialized” journals as equal scientifically, and that all Schekman is suggesting I guess, but most people are still tempted to submit to S&N to get the holy grail on their CV. From what I have seen, people do their science and then submit, not decide where they are going to submit and then do the work, so maybe this changes the way articles are written not how the science was done.
Nature and Science have always been a litle bit more than about the science, and I think we do need some journals like that. Otherwise scientists are just gifted technicians.
Pipped Kool. You mis-spelled your handle in your rush to be knowledgey. You are lazy in your haste.
“””””…..Of the 115 entries, 23 made the “neutral” pile, 83 were in the “worse” stack, and nine were in the “better.” The probability of the journals not having a bias is as likely as a coin being flipped 92 times and showing heads or tails fewer than nine times……””””””
So does that mean that about 83 times out of 92, it lands on edge ??
Mike Bromley the Kurd “Pipped Kool. You mis-spelled your handle in your rush to be knowledgey. You are lazy in your haste”
Cellphones and autocorrect…
Statistics is ALWAYS done on sets of numbers that are known. There are no x, y, z unknowns in such data sets,
The statistics describes properties of that completely known set of numbers. It gives ZERO information about what the NEXT number will be. Can’t even tell you whether the next number will be higher than the last number, or whether it will be lower; tells nada. No predictive properties whatsoever.
Pipped Kool says:
January 7, 2014 at 5:46 pm
“Nature and Science have always been a litle bit more than about the science, and I think we do need some journals like that. Otherwise scientists are just gifted technicians.”
Well, that set off some memories…….
I was in Geology 101 lab my freshman year ( I was initially a Chem major), working off an “other science” requirement of my undergrad degree, when the head of the Geo department sidled up to me. To this day I cannot conjure up what it was I asked him, but his answer is still crystal clear after 4 decades plus now:
“We basically have two kinds of people that go through here. Splitters and Lumpers. Splitters are those that aspire to seeing things right down to the gnat’s arse. Lumpers are Big Picture, rapid-arm movement types. But every once in a while someone comes through here that can see things right down to the gnat’s arse and immediately recognize how that just fundamentally changed the Big Picture. You know what we call them?” he asked.
I did not.
He said “We call them geologists………”
The problem here, of course, is following Obi-Wan on some damn fool mission………
Nice write up.
Unfortunately, too accurate.
The digital world nowadays provides a much smaller funnel for information for most folks. Hence the opportunity exists, in a much easier manor, for dis-information to become systemic.
Now why does my handle continually change capitalization priorities between devices, Dog Gone it!
Another example of the unexpected results from the digitization of a species 🙂