Below is an excerpt from an excellent article in The New Yorker which describes a recognition of curious phenomenon spanning many different fields of science:
Different scientists in different labs need to repeat the protocols and publish their results. The test of replicability, as it’s known, is the foundation of modern research. Replicability is how the community enforces itself. It’s a safeguard for the creep of subjectivity. Most of the time, scientists know what results they want, and that can influence the results they get. The premise of replicability is that the scientific community can correct for these flaws.
But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology. In the field of medicine, the phenomenon seems extremely widespread, affecting not only antipsychotics but also therapies ranging from cardiac stents to Vitamin E and antidepressants: Davis has a forthcoming analysis demonstrating that the efficacy of antidepressants has gone down as much as threefold in recent decades.
For many scientists, the effect is especially troubling because of what it exposes about the scientific process. If replication is what separates the rigor of science from the squishiness of pseudoscience, where do we put all these rigorously validated findings that can no longer be proved? Which results should we believe? Francis Bacon, the early-modern philosopher and pioneer of the scientific method, once declared that experiments were essential, because they allowed us to “put nature to the question.” But it appears that nature often gives us different answers.
Read more http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer#ixzz1BYjefYnF
h/t to WUWT reader Edward Lowe
=======================================================================
This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology.
If I may, I propose the name for this could be: confirmation entropy
This is the result of permitting corruption be rationalized as post-normal science.
It has been very very lucrative: Climate science is perhaps the most extreme example.
If we can work up the backbone to reject post-normal science and return to real science, I will be pleasantly surprised.
Schooler demonstrated that subjects shown a face and asked to describe it were much less likely to recognize the face when shown it later than those who had simply looked at it. Schooler called the phenomenon “verbal overshadowing.” … But while Schooler was publishing these results in highly reputable journals, a secret worry gnawed at him: it was proving difficult to replicate his earlier findings.
The problem is that he put his research into words!
“Below is an excerpt from an excellent artic in The New Yorker which describes a recognition of curious phenomenon spanning many different fields of science:”
My guess is that it should read
“….excellent article in The New Yorker which describes a recognition of a curious phenomenon spanning many different fields of science:”
(presumably not)
“….excellent article in The New Yorker which describes a recognition of curious phenomena spanning many different fields of science:”
Science and science fiction are interwoven. This blurrs the truthful respected science from coming to the surface.
The most relevant section in the article is later on:
Researchers may attempt to be objective, but editors and peer-reviewers certainly aren’t. This might be termed “Black is SO last year” or “New Shoes” science.
“….if your experiment needs statistics, you need a better experiment” I wish I could remember who said this. It’s a big problem. We need better experiments.
An excellent article, worth reading in full.
This statement: “… the decline effect is largely a product of publication bias, or the tendency of scientists and scientific journals to prefer positive data over null results, which is what happens when no effect is found.”, combined with the pal review prevalent in climate science, seems to me to describe why cAGW still isn’t dead yet in scientific journals.
Kevin Trenberth is inciting hate speech against those who question the science of global warming, by calling them “deniers”. Trenberth is promoting hate speech by attempting to label those skeptical of global warming as “holocaust deniers”.
Calling someone a “denier” just because he/she questions the science of global warming is deeply offensive, especially to Jews. Since when has hate speech been ok? Calling a skeptic a “denier” is just like calling a gay person a “faggot”, or calling a Black person a “nigger”. But this is exactly what Trenberth is doing and getting away with, because no one in the climate science community has the cojones to stand up to him.
And note the timing of the release of his speech–just a few days after the Jared Loughner shootings! I think Trenberth did this on purpose: to encourage unstable individuals to physically harm global warming skeptics–people like the eco-terrorist that took people hostage at the Discovery Channel office.
So here is my request: please shame KEVIN TRENBERTH for inciting hate speech against global warming skeptics (which is 2/3 of the US population, according to recent polls). In whatever way you can. Stop this guy before his hate speech causes physical violence against global warming skeptics.
Thank you,
Reply: Yo CJ, your email does not appear to be valid. Please fix or be deleted per blog policy. ~ ctm
Does an explanation lie, unexamined, in the phrase ” ..across a wide range of fields, from psychology to ecology”. That’s not a wide range. I expect that no experimental result of mine has diminished in accuracy, save for any where (dear God, I hope not) I simply made an unrecognised blunder. But then my field is a “hard” science not a “soft” one. Heavens, isn’t that a large part of the distinction between hard and soft?
Grab yourself a cuppa tea before reading the whole article, as it’s well worth it.
I’m not a stat’s guy but would the mean, mod, whatever of society now be corrupt, meaning there is no part of society now where corruption is not the mean, mod, whatever?
With these comments here “This phenomenon doesn’t yet have an official name, and If replication is what separates the rigor of science from the squishiness of pseudoscience, where do we put all these rigorously validated findings that can no longer be proved?”
The answer to both is what we used to call Magic Numbers, and both deserves to go under that name.
Look to the Cochrane Collaboration, and evidence based practice.
============
Anthony,
Do you ever sleep?
“But it appears that nature often gives us different answers.”
I disagree.
Nature has only one answer.
It is our interpretations that are different.
Green friends tell me the science can be trusted because science is self-correcting.
But how long does it take to self-correct? And how do you know when you’ve reached that point?
They boldly apply the “it is self correcting” mantra to stuff that’s only a couple of years old!!
The truth is in the system pipeline somewhere, deep down, and will take centuries for it to come to the surface.
“In the late nineteen-nineties, John Crabbe, a neuroscientist at the Oregon Health and Science University, conducted an experiment that showed how unknowable chance events can skew tests of replicability. He performed a series of experiments on mouse behavior in three different science labs: in Albany, New York; Edmonton, Alberta; and Portland, Oregon”
…
“The premise of this test of replicability, of course, is that each of the labs should have generated the same pattern of results.”
(the results didn’t follow any detectable pattern.)
…
“The disturbing implication of the Crabbe study is that a lot of extraordinary scientific data are nothing but noise. The hyperactivity of those coked-up Edmonton mice wasn’t an interesting new fact—it was a meaningless outlier, a by-product of invisible variables we don’t understand. The problem, of course, is that such dramatic findings are also the most likely to get published in prestigious journals, since the data are both statistically significant and entirely unexpected. Grants get written, follow-up studies are conducted. The end result is a scientific accident that can take years to unravel.”
This is very interesting and very disturbing. If I understand correctly, we have a big problem here. What we now consider “statistically significant data” may have to change.
In my experience, this happens when the empirical ‘facts’ are expressed in terms of a simplistic model, which is subsequently found to be naive. When the simplistic model finally collapses, the textbooks must be rewritten.
This is not to say that old ideas don’t go out without a fight.
There are typically some folks who are heavily invested in the old orthodoxy, and they tend to scoff mightily at the new framework.
RR
Hmmm… This passage reminds me of something…
“It feels good to validate a hypothesis. It feels even better when you’ve got a financial interest in the idea or your career depends upon it. And that’s why, even after a claim has been systematically disproven, you still see some stubborn researchers citing the first few studies that show a strong effect. They really want to believe that it’s true.”
– John Ioannidis, epidemiologist, Stanford University
http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer#ixzz1BZnK5QtS
So that is what they meant by hide the decline!
This was an extemely well written and interesting piece. I would have found it more disturbing if I had not suspected it already.
My guess is that the likelihood of false results increases proportionally to the amount of money at stake and the degree of polarisation of competing theories.
However I won’t try to prove it because the effect will surely wear off in time.
It is not surprising that research produces results that reflect the bias of the scientists doing the research. Especially in climate science.
Very thoughtful article. I can see only one thing he missed: Failure to replicate shouldn’t tell us to abandon the line of research. If scientists were operating out of curiosity instead of the need to justify grants, a failure to replicate would guide us toward a better theory. His example of the cocaine-fed mice is perfect. The Edmonton mice ran a lot farther despite every known factor being held identical. The difference was dramatically beyond expectable variance; it should lead to seeking the unknown factor that isn’t identical.
But nobody [or rather, nobody who gets published!] is operating out of curiosity. Nobody has the time or money. All science is based on the need to satisfy grantors within an annual budget goal, so all published science is stuck on bad theories.
The drink 8 cups of water meme and DDT spring to mind. — John M Reynolds
When politics trumps what the scientists find
You get what you pay for – more of the same kind.
The peer review process
Abounds with sheer excess.
Who paid for it anyhow? – Ah, never mind.
It’s called cargo cult science… the return of the witch doctors has begun.
“It’s as if our facts were losing their truth”. Well, there lies the problem. In what sense were they ever ‘facts’ and in what sense were they ever ‘truth’? Hasn’t this shift been going on for centuries? It’s hardly going to stop now. We can argue about whether it is gradual or paradigmatic (after Kuhn), but there has been no time in the last 500 years at the very least when this has not been happening. I was amazed to find that (apart from the Aristotle worship) science books that were written in the Middle Ages were still being used as textbooks, seemingly without need for revision, for hundreds of years. With mathematics it was millenia – Euclid was still on the curriculum in the twentieth century.
Just see how science changes. Very little of what was considered science orthodoxy, whether in physics, astronomy, medicine etc in 1900 is accepted as orthodoxy today. It has always been thus if you care to study history. I feel quite confident in asserting that little of what is considered orthodoxy today will be considered orthodox by the end of the century. As Richard Lindzen and others have pointed out, folk will look back at today’s climate science and hysteria the way we look back and sneer and guffaw at the cranky ideas that scientists held not that long ago.
And as for science itself, there is no consensus either currently or through history as to what the objects of scientific investigation are, nor a scientific method, nor what relationship empirical results have with reality. The whole question of what scientist can know, and how they can know it has never been settled. Many have given up on the philosophical, epistemological and metaphysical issues and just got on with science in an instrumental way. There is far too much scientific practice based on fallacious reasoning, induction and design of experiments to find the thing you are looking for. Popper’s falsification criteria, while logically true and powerful, is considered by many to be an impediment to scientific progress.
Anyone who says ‘the science is settled’ is either mad or moronic.