I was searching the early edition of PNAS for the abstract of yet another sloppy “science by press release” that didn’t bother to give the the title of the paper or the DOI, and came across this paper, so it wasn’t a wasted effort.
Steve McIntyre recently mentioned:
Mann rose to prominence by supposedly being able to detect “faint” signals using “advanced” statistical methods. Lewandowsky has taken this to a new level: using lew-statistics, lew-scientists can deduce properties of population with no members.
Josh (N=0) humor aside, this new paper makes me wonder how many climate science findings would fail evidence thresholds under this new proposed standard?
Revised standards for statistical evidence
Valen E. Johnson
Significance
The lack of reproducibility of scientific research undermines public confidence in science and leads to the misuse of resources when researchers attempt to replicate and extend fallacious research findings. Using recent developments in Bayesian hypothesis testing, a root cause of nonreproducibility is traced to the conduct of significance tests at inappropriately high levels of significance. Modifications of common standards of evidence are proposed to reduce the rate of nonreproducibility of scientific research by a factor of 5 or greater.
Abstract
Recent advances in Bayesian hypothesis testing have led to the development of uniformly most powerful Bayesian tests, which represent an objective, default class of Bayesian hypothesis tests that have the same rejection regions as classical significance tests. Based on the correspondence between these two classes of tests, it is possible to equate the size of classical hypothesis tests with evidence thresholds in Bayesian tests, and to equate P values with Bayes factors. An examination of these connections suggest that recent concerns over the lack of reproducibility of scientific studies can be attributed largely to the conduct of significance tests at unjustifiably high levels of significance. To correct this problem, evidence thresholds required for the declaration of a significant finding should be increased to 25–50:1, and to 100–200:1 for the declaration of a highly significant finding. In terms of classical hypothesis tests, these evidence standards mandate the conduct of tests at the 0.005 or 0.001 level of significance.
From the discussion:
The correspondence between P values and Bayes factors based on UMPBTs suggest that commonly used thresholds for statistical significance represent only moderate evidence against null hypotheses. Although it is difficult to assess the proportion of all tested null hypotheses that are actually true, if one assumes that this proportion is approximately one-half, then these results suggest that between 17% and 25% of marginally significant scientific findings are false. This range of false positives is consistent with nonreproducibility rates reported by others (e.g., ref.5). If the proportion of true null hypotheses is greater than one-half, then the proportion of false positives reported in the scientific literature, and thus the proportion of scientific studies that would fail to replicate, is even higher.
In addition, this estimate of the nonreproducibility rate of scientific findings is based on the use of UMPBTs to establish the rejection regions of Bayesian tests. In general, the use of other default Bayesian methods to model effect sizes results in even higher assignments of posterior probability to rejected null hypotheses, and thus to even higher estimates of false-positive rates.
This phenomenon is discussed further in SI Text, where Bayes factors obtained using several other default Bayesian procedures are compared with UMPBTs (seeFig. S1). These analyses suggest that the range 17–25% underestimates the actual proportion of marginally significant scientific findings that are false.
Finally, it is important to note that this high rate of nonreproducibility is not the result of scientific misconduct, publication bias, file drawer biases, or flawed statistical designs; it is simply the consequence of using evidence thresholds that do not represent sufficiently strong evidence in favor of hypothesized effects.
=================================================================
The full paper is here: http://www.pnas.org/content/early/2013/10/28/1313476110.full.pdf
The SI is here: Download Supporting Information (PDF)
For our layman readers who might be a bit behind on statistics, here is a primer on statistical significance and P-values as it relates to weight loss/nutrition, which is something that you can easily get your mind around.
Gross failure of scientifical nutritional studies is another topic McIntyre recently discussed: A Scathing Indictment of Federally-Funded Nutrition Research
So, while some dicey science findings might simply be low threshold problems, there are real human conduct problems in science too.
@ur momisugly Diego Diaz
You are (I imagine) referring to programs like Thomas S. Ray’s Tierra. While this emulates certain aspects of evolution, it does not generate novelty. This is the aspect of evolution that has me most fascinated and frankly, being so seems to get me into hot water every now and then. So it goes… I have always been keenly interested in the hard questions.
Good luck with your work and your rainforest. It sounds like a fascinating place to work. The Git lives adjacent to a rainforest in a much cooler part of the world: Southern Tasmania. Our indigines are supposedly extinct which makes them justifiably angry. However, it’s probably better to be angry than extinct 😉
Diego Diaz says:
November 15, 2013 at 12:48 pm
Dawkins does not say that about HGT. Search and find out what he really does say.
—————————————–
Dawkins talks with human genome entrepreneur Venter on HGT:
http://www.edge.org/documents/dawkins_venter_index.html
Diego Diaz says:
November 15, 2013 at 8:06 pm
Don’t know if you’re more interested in attempts to recreate evolution through computer simulations or in using nature to design programs, but this journal covers them both, more the latter:
http://www.mitpressjournals.org/loi/evco
I am back again. Tierra I have not used. It is old and lacks.
My work area is not rain forest. It is pampas, llanos and savannas of Beni, Bolivia from where come such crops as cassava, cotton, peanut, sweet potato, tobacco, vanilla, etc.
Thanks for wishes.
@ur momisugly Diego Diaz
I have long believed that the conundrum I face with the evolution of novelty in biology would eventually be solved computationally. Gregory Chaitin seems to believe the same, but estimates that it’s going to take a long time to achieve. IOW The Git will be dead when it happens. The problem with Tierra and its ilk is that information comes from outside. Dawkins’ example of evolution from a random string to “METHINKS IT IS LIKE A WEAZEL” presupposes the existence of the final string: “METHINKS IT IS LIKE A WEAZEL”. Also, IIRC, Dawkins acknowledged this and says something like “but evolution is very similar” or some such. This is arm waving to say the least.
One question I remember asking all these many years ago was “Why would ‘junk DNA’ persist under natural selection pressure?” A few years ago I heard a geneticist on the radio saying how remarkably well conserved it was (more so than expressed DNA) and that it had been renamed ‘silent’ rather than ‘junk’. It is my belief, and this is not original with me, that the silent DNA is akin to a computer operating system and that the various organisms inhabiting Earth are running different software on essentially the same operating system. However, it’s vastly different to anything like what we currently run on our computers in that it automagically generates new stuff within an apparently closed system. There is no way that I can conceive of MS Word evolving into MS Excel gradually while retaining usefulness.
Chaitin tips his hat to at least two panspermia (new information comes from outside) advocates: Fred Hoyle and Francis Crick. And that’s enough for him to be condemned by several biologists I have come across, but that’s life so to speak. He also notes how indebted he is to John von Neumann, Alan Turing and Kurt Gödel, three other thinkers who had a major impact on my thinking. And now I return to finishing his rather short book.
milodonharlani said @ur momisugly November 15, 2013 at 9:19 pm
As it happens, Doctor Who had promised to take me to that event in August 2008 and return me to October 2005 which was when I wrote my piece. However, he has so far failed to turn up and fulfill that promise. It seems likely that he has been delayed (if that is the correct word in this context) in another part of the multiverse and will eventually do so and history might very well become instantaneously rewritten.
rgbatduke said @ur momisugly November 15, 2013 at 2:20 pm
One of the problems I was having in the mid 2000s was being tasked for giving too much weight to current/recent work and that I should merely accept the older, simpler prescriptions of the modern synthesis (dare I say consensus). So it goes…
You also write:
The ID argument as exemplified by Paley et alia is most decidedly based upon empirical evidence; it’s certainly not a priori. We just do not find it compelling or sufficient within the context that it is presented. For example, the creationists present the evidence of cnidarians (jellyfish) with well developed lenses, but no brain or retina to process the information from the gathered light. The creationist says: “God done it; The Git says: “The genes for lens formation did not arise in the jellyfish, they appear to have been transferred horizontally from some other organism; Dawkins said: “Retinas and brains precede lenses” (paraphrase from Climbing Mount Improbable). This latter is about as far from empirical as you can get.
It is interesting though that these design arguments do not go away. “One possibility is that they really are better arguments than most philosophical critics (including The Git) concede. Another possibility is that design intuitions do not rest upon inferences at all.” But that then becomes another argument that I am loath to pursue.
Thanks for your supportive comments Robert.
rgbatduke says:
November 15, 2013 at 1:35 pm
A school of thought in canine evolution is that dogs originally domesticated themselves. It is hypothesized that outcast wolves scavenged the scraps & excrement left behind at human camps. These hang-around-the-camp wolves started following the people & became accepted by them not as threats but alarm systems against predators in the night. Later selective breeding occurred as people adopted the most tractable puppies. Later still, when the human diet came to include more grains, dogs evolved the ability to digest starches better than their wild ancestors, reliant primarily on meat.
In experiments with foxes, domestication also produces up-turned tails & big floppy ears. Dogs retain adolescent wolf traits into adulthood, like barking. They are in this sense retarded-development wolves. As you note, they’re still the same species, but a distinct subspecies. For that matter, dogs interbreed with coyotes & produce fertile offspring (however there are some behavioral barriers to frequent mating). Although wolves appear to have arisen in NE Asia rather than North America, one hypothesis of their origin is that highly social wolves evolved from largely solitary coyotes, in order better to hunt bigger game of the Pleistocene. Genetics tend to support the hypothesis, so either coyote ancestors once ranged into Asia or fossil or molecular (rocks & clocks) evidence of earlier wolves hasn’t been found.
The Pompous Git says:
November 16, 2013 at 2:27 am
That was just one of many hits I found in doing the suggested search. I realize that you wouldn’t necessarily keep up on developments in evolutionary biology after taking a course in the philosophy of biology, but as you’re interested enough in it to comment on it here, maybe so. However in a fast-moving discipline like evolutionary biology, 2005 was long, long ago, if not also far, far away. The Human Genome Project wasn’t even completed until 2003. And who knows how long your teacher had been giving the same philosophy lecture?
As I said, I’ve not read Dawkins, 1976, but Kurland thinks he addresses in that book the issue of gene transfer, if not in depth:
http://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=146787&fileOId=625105
http://www.embl.de/aboutus/science_society/conferences/conference_2000/participants/kurland/
Dawkins was certainly aware of the latest developments in HGT, since recombinant DNA was a hot topic in the ’70s, especially at my then-university Stanford, where the 1947 discoverer of HGT, Lederberg worked. Recombinant DNA was first proposed by Lobban, graduate student of Kaiser in the Biochemistry Dept. at Stanford Med School. The first publications describing successful production & intracellular replication of recombinant DNA appeared in ’72 & ’73. Stanford applied for a US patent on recombinant DNA in 1974, listing the inventors as Cohen & Boyer.
I have to agree that Dawkins should have discussed HGT in more detail than he apparently did. I never attended his lectures at Oxford while he was writing that book, but recall when his thesis adviser Tinbergen got the Nobel in 1973. Kurland above mentions Dawkin’s path-finder Hamilton, the adviser of Judson, whom you think has it all wrong on genes.
The generation of novelty isn’t an issue in biology. It’s observable in the lab as well as in the field. It arises either by adaptation of an existing structure or process or by the sudden or gradual development of new structures or functions. It’s a commonplace both in the fossil record & the genome. Examples fill books. No intelligent guidance required. In fact, as I noted, an intelligent designer would rarely do it the way that nature has been forced to do.
Pretty up to date discussion on cnidarian & earlier opsin evolution & innovation in photo reception:
http://genomewiki.ucsc.edu/index.php/Opsin_evolution:_key_critters_%28cnidaria%29
Note important paper from 2008 cited.
http://hwmaint.pnas.org/cgi/content/full/105/26/8989
milodonharlani said @ur momisugly November 16, 2013 at 10:25 am
[TIC]No, 650 Mya was long ago[/TIC] Our philosophy lecturer, as I have said before, knew nothing to speak of about biology prior to the commencement of the course. His first degree was in physics where he had worked on multi-value logics in quantum physics and his second degree was in philosophy. I cannot remember what his doctorate was in. What I do know is he was much younger than I and extraordinarily bright and an excellent pedagogue.
The biology grad students in the class provided a lot of input on biology minutiae as needed, but the philosophy of biology is about broader issues, such as “what do we mean by species?” and I note you did not fundamentally disagree with our text’s conclusion that there are a flock of them.
Another issue was the concept of memes, with Richard initially believing that they were a useful concept, though far less so when confronted by cogent argument from the class participants. This is in the nature of philosophy. Different positions on an issue are supported by argument (logic being prominent) and it’s the argument that is evaluated. That is, to take an extreme example, the proposition that God intervenes in his creation might be true, but nevertheless be deprecated on the grounds of lack of evidence (true propositions), or valid arguments (deduction or other inference). As I have said, interminably it seems to me, no-one has ever yet to my knowledge ever stated what might be considered grounds for belief in God. Certainly not the Discovery Institute, I have said that Richard Swinburne has demonstrated that God;s existence is more likely than not, but that is no more proof of God’s existence than the output of AOGCMs are proof of CAGW and is really only a coda to Aquinas’ Summa written over 700 ya.
As it happens, the issue of novelty arising in evolution was not a topic in that philosophy class, or at least not important enough to receive any attention that I recall. That was a separate research project of mine and very much predated the class. Much juicier for philosophy is the issue of denial of teleology in biology, despite abundant use of teleological language in the discipline and how might biologists rid themselves of its use. Up until 2005, or thereabouts, every instance I was given of observed novelty arising either was not what I meant (think Galapagos finches, dogs and herring gulls), or it appeared to be the result of a pre-existing gene sequence transferred into the organism. You may fault me for not finding what I sought, but I refuse to take the blame for the impoverished, unsupportive examples given me when I requested them from people at the coalface so-to-speak. If they could not point me in the right direction, who would you suggest?
Note that I did try very hard at the time and I am not at all accusing those I sought assistance from as acting in bad faith, or lacking sufficient erudition. It was as if they were blind to the issue of the difference between observation and inference. That in itself was an important learning in two regards. If someone is blind to something, no amount of request/badgering/pleading is ever going to result in generating a meaningful response. If the person you interrogate is blind to something, then what of oneself? Am I too blind to certain things? Perhaps something akin to this blindness prevented me from accepting the existence of God on faith and still does.
Very early on in this private research I came across Tierra (and some other I cannot recall) and thought that this looked like a promising direction for research into how novelty might arise. I had decided very early on during my foray into evolutionary biology that this was probably a more rational approach than trying to decipher actual genetic code. The concept was supported by Dawkins’ approach in <The Blind Watchmaker, but disappointingly so. Dawkins, an apparently intelligent agent, was doing the selection. It was no further an advance than Darwin’s initial argument from artificial selection to natural selection by inference to the best explanation. My investigation of Tierra also disappointed.
As a generalist/polymath/jack-of-all-trades [delete whichever is inapplicable] and having more avenues of learning to pursue before I die than I have time for, I decided to pursue some of those. Six years may be a long time at your age, but at mine it seems terribly short. I had also been offered part-time employment that would carry me through to retirement (I thought) and allow me to pursue my other interests. I make the observation that Chaitin’s proposed research project looks very interesting indeed, though I will (sadly), not be around to see it’s fruits.
I end this with an amusing (to Pompous Gits) observation. Einstein is widely believed to have believed in God, mainly because he said he did. Dawkins disagrees and in his book The God Delusion declares him to have been an atheist. Conversely, Sir Fred Hoyle is widely declared to have been a believer in God and I have been told by a number of correspondents that he was a creationist. I possess several of Fred’s books wherein he asserts that he is an atheist. From this I draw the conclusion that no matter what one says, there are always those who will interpret your words to mean what they demand you ought to have said or written. It’s an interesting world to live in…
Your Most Exalted Gitness:
I’ll be 63 next month, if that matters.
If your philosophy of biology course were taught by the physicist Dr. RGB of Duke, I’d say it was liable to be a highly worthwhile course. For a generic physicist, not so much.
Hoyle was an atheist, no doubt about it, & never wavered, despite the disturbing to him implications of his study of life outside his area of expertise. Like many mathematicians & physicists who slide into ID territory, however, he erred in treating the emergence & evolution of life as strictly mathematical, probabilistic phenomena. One of the foremost such questioners in the 1970s was honest enough publicly to change his mind when he learned more about how the genetic code really works. When you actually look at the biochemistry involved, especially in light of advances in molecular biology & physical & organic chemistry of recent decades, both abiogenesis & evolution appear less & less against the odds & more & more inevitable, given the rules of our universe. Hoyle has been somewhat discredited for his opposition to the Big Band Theory, but IMO is still a thinker worth reading. Panspermia, his solution for the largely non-existent problems he saw, remains a defensible hypothesis. The latest discoveries of organic chemistry in the cosmos don’t necessarily support it, but don’t render it less likely either.
Einstein was always coy about his theological beliefs. He may never have worked them out systematically to his own satisfaction. He definitely did not believe in a personal God guiding events on earth, let alone in individual human lives, counting the fall of sparrows & hairs on heads. Both he & Hoyle have been hijacked by creationists who twist their words, just as they lie about biology in general & Darwin in particular. They are shameless, paid liars.
The proper study of novelty in evolution is biology, not a prior philosophy. The history of life on earth is the story of naturally accumulating innovations. We have only just started to be able to read the history book, but everything more science learns only reinforces the basic lesson already apprehended, ie that new “genes”, structures & processes arise naturally from preexisting building blocks.
I know your time is limited, but here are some recent papers or summaries of present understanding dealing with novelty on both the genetic & organismal levels. Examples could be multiplied in both camps & all fields in between.
A good survey of current knowledge of how variation & novelties accrue at the genetic level. It’s very good, IMO:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2945180/
An important example of how innovation rapidly increased in unicellular life in response to the greatest catastrophe & opportunity for “advancement” in the history of life on earth, ie the monumental Precambrian oxygenation event:
http://www.ncbi.nlm.nih.gov/pubmed/21170026
Moving far forward in time, how evolution adapted a preexisting structure to a novel use, ie the gill arch into the gnathostome jaw:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1571356/
How, after some 300 million years of further evolution, the descendent of that first jaw evolved into a novel structure, the mammalian middle ear:
http://www.ncbi.nlm.nih.gov/pubmed/21170026
Which, interestingly, happened twice in mammalian evolution, both in monotremes & in the common ancestors of marsupials & placentals. That is, two different groups of proto-mammals who had both the “reptilian” jaw joint (which was increasingly used to augment hearing) & the mammalian joint at different times went the whole hog & moved the now little bones at the back of the jaw into the skull, mating them with the inner ear structure:
http://evolution.berkeley.edu/evolibrary/article/evograms_05
BTW, the otherwise nice graphic is marred by using the term “eutherian”, which included placentals but excludes marsupials. That’s a mistake. Our middle ear emerged in a common ancestor of both your continent’s wonderful pouched creatures & us placentals.
@ur momisugly milodonharlani
Your most esteemed milodonharlaniness doesn’t quite roll off the tongue, so I’ll likely not use it again 😉
You should be aware that Richard’s class was at a fairly high undergraduate level being both a second year and third year class and marked according to which year one was in. The biology students were graduate students. Presumably they were either required to undertake further undergrad classes, or were there because of special interest. Given their enthusiasm for the topics, I’d plump for the latter.
As for Richard, he may have been a little short of understanding key concepts in biology in the early stages of the course, but he seemed well up to speed by the end. As I wrote earlier, he is very bright. Note that we were discussing (and Richard’s lectures were closer to tutorials than classical lectures) keyconcepts, not minutiae. Philosophers of biology such as Sober, Ruse, Sterelny (who resembles me, not me him), Griffiths, et alia do not get bogged down in biological minutiae; they are usually far too busy getting bogged down in philosophical minutiae! But enough…
Waaaay back then and a correction, seven years, not six, I came across Richard Lenski’s long-term e. coli experiment. Now according to my nemesis (and Gould) I should have expected Lenski to find divergence between his bacterial lines. When I reported my finding to my nemesis he wrote that convergence was exactly what would be expected. Now how something can be simultaneously divergent and convergent is beyond all except… post-modernist philosophers such as Derida and Foucault. I don’t believe I’ve mentioned yet that I am as far from being an admirer of their ilk as it is possible to be. And unlike those philosophy courses that people take “because you can’t fail” well, those who thought that and took any of Richard’s had a most unpleasant surprise. As that physicist turned philosopher Phil Dowe told first year students: “If your brain isn’t hurting, you are not doing philosophy!”
Anyway, back to the farce. I revisited Lenski and his team’s work today and came across this juicy quote: “Natural selection is critical for the process of adaptation, yet its role in producing key innovations is less clear because, by fixing variants that improve existing functions, selection might strand populations on local adaptive peaks and thereby prevent them from discovering new functions.”
http://www.sciencemag.org/content/335/6067/428.abstract
I suspect we are at the stage of talking past each other. If I took you entirely at face value, then I would have to declare Chaitin’s proposal as p!ssing in the wind. I don’t think he is and I do believe that putting (naturalistic) evolutionary explanation on a genuinely secure footing is a very long-term project. I also note that if life is not irreducibly complex, then it will terminate in an ultimate organism/ecology and proceed no further. Sort of a biological version of Hawking’s belief in a “mind of God” equation. My intuition is that it is irreducibly complex and therefore must continue to evolve even unto the heat death of the universe. YMMV.
Really crazy thought #64,348
Suppose that this static organism that can no longer evolve (discover new things) at the end of a John Wheeler universe (serial big bangs and crunches) always says: we’ll design the next one to really be irreducibly complex! :-))))
@ur momisugly milodonharlani
I just visited the Amazon page to read reviewers’ comments of Sterelny & Griffiths’ Sex and Death (there weren’t too many (any?) the previous time I looked). They are interesting and deal with the problems the evolutionary biology tackled in that the class I took all those many long years ago.
http://www.amazon.com/Sex-Death-Introduction-Philosophy-Foundations/dp/0226773043
I also just remembered you calling attention to my misunderstanding randomness in biology (entirely possible). Nevertheless, extensive discussion of randomness occurred in the cosmology class (same teacher and slightly more than the two dozen in the phil. biol. class). Despite the conceptual difficulties encountered by those unfamiliar with, for example, quantum physics, The Git was assessed as having an excellent understanding of underlying concepts. Perhaps physicists do have an entirely separate understanding of randomness than biologists… So it goes…
@ur momisugly milodonharlani
I must admit to being utterly perplexed. You referred me to Henrik Kaessmann’s Origins, evolution, and phenotypic impact of new genes. On page two, I read
and
Yet you wrote: ““Ohno was wrong about that. He didn’t know then what we know now, & was already shown wrong even before we found out as much as we have about genomics.” The second quote above seems to echo what I have been saying; changes in the duplicated gene are not random. Indeed, something cannot be simultaneously predictable and random. What, precisely, makes Kaessmann correct, and The Git wrong when they are saying substantially the same thing?
The Pompous Git says:
November 16, 2013 at 11:20 pm
Natural selection doesn’t ordinarily create innovation at the genetic level. It acts upon it. It could be said to create novelty at the organismic level, as in changing a gill arch into a jaw bone or jaw bones into ear bones. I don’t know why this would even be considered an issue.
The processes that create novelty at the genetic level are those mentioned in my link on that, such as gene duplication (or multiplication), gene transfer & various kinds of mutation (which while more or less random result from natural processes operating all the time at more or less predictable rates in various environments or the whole planet).
The Pompous Git says:
November 17, 2013 at 9:00 am
I’ll try again on randomness.
That this or that base pair in a genome might be hit by a passing cosmic ray is more or less random. Yet the rate at which mutations by cosmic rays will occur is not random, but predictable based upon cosmic ray flux.
The same applies to many if not most other mutagenic agents.
The “accident rate” in transcription errors is also a statistical phenomenon. Its rate can vary based upon a number of parameters, but that such “random” mistakes will happen is predictable.
Random “accidents” in biology result in predictable mutation rates. That such accidents will happen is a certainty, although of course there are other sources of genetic variation besides mutations. Organisms also effectively have some degree of control over their mutation rates because environmental stresses or changes subjecting a population to selective pressure can & do increase the accident rate. Conversely, a steady selective pressure in an unchanging environment can lead to maintaining a population’s genome more stable from generation to generation. That’s how you get “living fossils”, like the one-lunged Queensland lungfish, with the largest known vertebrate genome.
I hope this helps.
@ur momisugly milodonharlani
Since according to you Ohno was wrong about neofunctionalization and the document you referred me to says the opposite I do not believe it’s worth continuing with the conversation your majesty. Philosophers are averse to contradiction.