In the wake of what Willis recently pointed out from Nassim Taleb, about how “In fact errors are so convex that the contribution of a single additional variable could increase the total error more than the previous one.”, I thought it relevant to share this evisceration of the over-reliance on statistical techniques in science, especially since our global surface temperature record is entirely a statistical construct.
Excerpts from the Science News article by Tom Siegfried:
Science is heroic. It fuels the economy, it feeds the world, it fights disease. Sure, it enables some unsavory stuff as well — knowledge confers power for bad as well as good — but on the whole, science deserves credit for providing the foundation underlying modern civilization’s comforts and conveniences.
But for all its heroic accomplishments, science has a tragic flaw: It does not always live up to the image it has created of itself. Science supposedly stands for allegiance to reason, logical rigor and the search for truth free from the dogmas of authority. Yet science in practice is largely subservient to journal-editor authority, riddled with dogma and oblivious to the logical lapses in its primary method of investigation: statistical analysis of experimental data for testing hypotheses. As a result, scientific studies are not as reliable as they pretend to be. Dogmatic devotion to traditional statistical methods is an Achilles heel that science resists acknowledging, thereby endangering its hero status in society.
…
More emphatically, an analysis of 100 results published in psychology journals shows that most of them evaporated when the same study was conducted again, as a news report in the journal Nature recently recounted. And then there’s the fiasco about changing attitudes toward gay marriage, reported in a (now retracted) paper apparently based on fabricated data.
But fraud is not the most prominent problem. More often, innocent factors can conspire to make a scientific finding difficult to reproduce, as my colleague Tina Hesman Saey recently documented in Science News. And even apart from those practical problems, statistical shortcomings guarantee that many findings will turn out to be bogus. As I’ve mentioned on many occasions, the standard statistical methods for evaluating evidence are usually misused, almost always misinterpreted and are not very informative even when they are used and interpreted correctly.
Nobody in the scientific world has articulated these issues more insightfully than psychologist Gerd Gigerenzer of the Max Planck Institute for Human Development in Berlin. In a recent paper written with Julian Marewski of the University of Lausanne, Gigerenzer delves into some of the reasons for this lamentable situation.
Above else, their analysis suggests, the problems persist because the quest for “statistical significance” is mindless. “Determining significance has become a surrogate for good research,” Gigerenzer and Marewski write in the February issue of Journal of Management. Among multiple scientific communities, “statistical significance” has become an idol, worshiped as the path to truth. “Advocated as the only game in town, it is practiced in a compulsive, mechanical way — without judging whether it makes sense or not.”
Commonly, statistical significance is judged by computing a P value, the probability that the observed results (or results more extreme) would be obtained if no difference truly existed between the factors tested (such as a drug versus a placebo for treating a disease). But there are other approaches. Often researchers will compute confidence intervals — ranges much like the margin of error in public opinion polls. In some cases more sophisticated statistical testing may be applied. One school of statistical thought prefers the Bayesian approach, the standard method’s longtime rival.
…
Why don’t scientists do something about these problems? Contrary motivations! In one of the few popular books that grasp these statistical issues insightfully, physicist-turned-statistician Alex Reinhart points out that there are few rewards for scientists who resist the current statistical system.
“Unfortunate incentive structures … pressure scientists to rapidly publish small studies with slapdash statistical methods,” Reinhart writes in Statistics Done Wrong. “Promotions, tenure, raises, and job offers are all dependent on having a long list of publications in prestigious journals, so there is a strong incentive to publish promising results as soon as possible.”
And publishing papers requires playing the games refereed by journal editors.
“Journal editors attempt to judge which papers will have the greatest impact and interest and consequently those with the most surprising, controversial, or novel results,” Reinhart points out. “This is a recipe for truth inflation.”
Scientific publishing is therefore riddled with wrongness.
Read all of part 1 here
Excerpts from Part2:
Statistics is to science as steroids are to baseball. Addictive poison. But at least baseball has attempted to remedy the problem. Science remains mostly in denial.
True, not all uses of statistics in science are evil, just as steroids are sometimes appropriate medicines. But one particular use of statistics — testing null hypotheses — deserves the same fate with science as Pete Rose got with baseball. Banishment.
Numerous experts have identified statistical testing of null hypotheses — the staple of scientific methodology — as a prime culprit in rendering many research findings irreproducible and, perhaps more often than not, erroneous. Many factors contribute to this abysmal situation. In the life sciences, for instance, problems with biological agents and reference materials are a major source of irreproducible results, a new report in PLOS Biology shows. But troubles with “data analysis and reporting” are also cited. As statistician Victoria Stodden recently documented, a variety of statistical issues lead to irreproducibility. And many of those issues center on null hypothesis testing. Rather than furthering scientific knowledge, null hypothesis testing virtually guarantees frequent faulty conclusions.
10. Ban P values
9. Emphasize estimation
8. Rethink confidence intervals
7. Improve meta-analyses
6. Create a Journal of Statistical Shame
5. Better guidelines for scientists and journal editors
4. Require preregistration of study designs
3. Promote better textbooks
2. Alter the incentive structure
1. Rethink media coverage of science
Read the reasoning behind the list in part 2 here
I would add one more to that top 10 list:
0. Ban the use of the word “robust” in science papers.
Given what we’ve just read here and from Nassim Taleb, and since climate science in particular seems to love that word in papers, I think it is nothing more than a projection of ego from the author(s) of many climate science papers, and not a supportable statement of statistical confidence.
One other point, one paragraph in part one from Tom Siegfried said this:
For science is still, in the long run, the superior strategy for establishing sound knowledge about nature. Over time, accumulating scientific evidence generally sorts out the sane from the inane. (In other words, climate science deniers and vaccine evaders aren’t justified by statistical snafus in individual studies.) Nevertheless, too many individual papers in peer-reviewed journals are no more reliable than public opinion polls before British elections.
That ugly label about climate skeptics mars an otherwise excellent article about science. It also suggests Mr. Siegfreid hasn’t really looked into the issue with the same questioning (i.e. skepicism) that he did for the abuse of statistics.
Should Mr. Siegfreid read this, I’ll point out that many climate skeptics became climate skeptics once we started examining some of the shoddy statistical methods that were used, or outright invented, in climate science papers. The questionable statistical work of Dr. Michael Mann alone (coupled with the unquestioning media hype) has created legions of climate skeptics. Perhaps Mr. Siegfeid should spend some time looking at the statistical critiques done by Stephen McIntyre, and tell us how things like a single tree sample or upside down data or pre-screening data begets “robust” climate science before he uses the label “climate deniers” again.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Vaca=cow. Vaccine=a generic word derived from the brilliant idea use cowpox to stimulate immunity to smallpox. Vaccine evaders=those who avoid doses of actual data to immunize against superstition.
Anti-vaxxers don’t know how short the half-life of methane is in the atmosphere.
=============
I suggest a read of “Statistics Done Wrong: A Woefully Incomplete Guide” http://smile.amazon.com/Statistics-Done-Wrong-Woefully-Complete/dp/1593276206/
It goes into the many ways that statistics are abused in scientific papers.
Maybe a little off topic, maybe not…
http://law.indiana.edu/instruction/profession/doc/16_1.pdf
“Why Most Published Research Findings Are False”
by John P. A. Ioannidis
Published: August 30, 2005
http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
Probably including his research findings on that.
The Lancet editorial has justifiably caused a lot of stir. The scientific method is a process. Scientific knowledge is what the process produces. If the product is bad, then the production process is not good. Many causes upthread.
One that has not yet been mentioned. It used to be (like when I was learning econometrics) that doing statistical analyses meant getting your hands dirty. Understand the math, write code to do the calculations. Plenty of things could wrong, but understanding what you were doing was not one of them. Understanding things like heteroskedasticity, kurtosis, autocorrelation…These days you just put data into a stats package and menu select the stats you want. And that ability to do without first understanding leads to a lot of really bad stuff. Eschenbach’s recent post on autocorrelation’s impact on effective N (and hence the significance of any statistical procedure) is one of many examples of what can go wrong.
My personal favorite in climate science is Dessler’s 2010 paper claiming via OLS a statistically significant positive cloud feedback (its touted on NASA’s website) in a near perfect data scatter (near perfect defined as r^2 of 0.02!). Dessler flunks stats 101. So do his peer reviewers. So does Science, which published this junk paper including the scatterplot. Essay Cloudy Clouds.
I am not a climate scientist, but reading that here and elsewhere, I think there are very few climate scientists for whom the words don’t constitute an oxymoron. More accurately they are economists, psychologists, social and political studies proponents … who wish to claim they are scientists for whatever public credibility and influence that may gain.
At my former university, some faculty have proposed a new masters degree program in social “sciences” that has not a single course in research designs, methods, or statistics. For them significance will be a belief system without any probability. Gradual students will be reading/viewing materials for which they have not the ability to judge, but will only acquire feelings. Several senior faculty have just taken early retirement, having issues with these ethics of the modern university. I left over a decade ago, having fought this trend for several decades and recognizing much more meaningful things to do.
Whenever I read “Scientists have found, believe, discover … “, that bilge only merits publication in National Enquirer or other grocery checkout rags. We actually purchase those for recreational reading while sitting in the outhouse at camp. (composting toilet, by the way)
On a more topical note, I see very little evidence that climate science has any clue about the nature of its various data. How is temperature data or other distributed? Is any sample independent? What is the extent of auto-correlation among the samples? Statistics requires knowledge of the underlying nature of the data before you go drilling.
As soon as one invents a mathematical formula through statistical correlation:
Global Ave T = 0.0005XCO2 + baseline, one has created a spurious mathematical correlation with no justification for its basis in nature. The end justifies the means. I see very little natural justification in this “field”.
There is a huge chasm between statistical significance and worldly importance. A 2 degree C temperature change isn’t even weak horseradish. OK /end rant
How interesting that Stephan Lewandowsky is an “award winning teacher of statistics”…
Nothing wrong with teaching statistics. It’s at least as interesting as teaching Origami.
g
“My personal favorite in climate science is Dessler’s 2010 paper claiming via OLS a statistically significant positive cloud feedback (its touted on NASA’s website) in a near perfect data scatter (near perfect defined as r^2 of 0.02!). Dessler flunks stats 101. So do his peer reviewers. So does Science, which published this junk paper including the scatterplot. Essay Cloudy Clouds.”
I’ve been promoting the idea of creating a Scientific Data and Conclusion Validation Agency to perform double blind tests on research funded by the government and used to form public policy. It would be much like the FDA is to drugs, and the EPA is to chemical approval. We simply need to apply the same rigorous analysis that the FDA does drugs and the SEC does to stock market firms. Global Warming, err, Climate Change has proven itself to be the most corrupting and fraudulent movement in my lifetime. The science, data and models simply don’t support the conclusions. They have published results that prove that. Simply applying double blind statistical analysis to climate change research, and prosecuting a few of the lead fraudsters will end this nonsense of CO2 driven climate change forever. Demand Congress look into the scientific practices being funded by our tax dollars.
Billions spent, and these nit wits can’t even create a computer model to demonstrate their fraud. Bernie Madoff could have done a better job with a lot less money.
http://www.cfact.org/wp-content/uploads/2013/07/spencer-models-epic-fail2-628×353.jpg
Actually, for many climate papers you don’t need to do any statistical replication. The conclusions are somehow falsified nonparametricly from first principles. McIntyre just provided an example of upside down varve use. Karls paper relied on Huang’s 2015 ocean buoy temperature adjustment of 0.12C. Huang did not give a confidence interval. Huang used the method of Kennedy 2011, who also computed 0.12C, BUT plus minus 1.7C! GIGO, another deliberate choice to fudge results. There are many other similar examples in various climate essays in my ebook. Marcott’s mess, Shakun’s mess, Cazenove’s ‘statistical’ explanation for a supposed SLR slowdown, Fabricius OA coral studies, Thomas extinction estimates that became the sole basis for the AR4 estimates, Bebber poleward spread of plant pests, and many more.
Reading the headline, I thought this might be about robust statistics, a favourite topic if mine. In short, robust statistics can not be thrown off by a moderate fraction of wildly corrupted values. But upon further reading, it doesn’t appear that’s really what is addressed here.
Historically statisticians have depended on least-squares estimation, which is notoriously sensitive to even one bad value. There were many reasons for using least squares (including tractability and the bloody central limit theorem), but with the introduction of modern robust techniques and a better understanding of the nature of real data, these has have fallen by the wayside.
I don’t know how often robust statistics are used in climatology. It certainly appear to be a good candidate for it.
Improve statistical tests 500% for significance
As rbgduke cites above “p happens”. Because of the frequent lack of reproducible significance in papers, mathematician Valen Johnson calls for 5 time more stringent statistics for results to be significant or highly significant in PNAS:
PNAS vol. 110 no. 48 Valen E. Johnson, 19313–19317, doi: 10.1073/pnas.1313476110
P happens in a Gaussian population sample. Not otherwise, under which circumstances it can be computed but is meaningless.
In light of recent Supreme Court responses, I have to comment on the reference in Tom’s writing to the poorly done gay opinion research. While I don’t argue with the need for surveys to assess public opinion and to do them with at least a modicum of excellence, the results of such a study, had it been done right, or the retraction of such a study, cannot be applied in a constitutional case. Such a decision hinges on the constitution, not solely on science, as the deciding factor.
It is clear in this modern age, that all “men” must be interpreted to be “all humanity” and as such, are equal and endowed with unalienable rights specified by our Declaration of Independence as well as confirmed in our Constitution. It does not matter the opinions of one person or billions of people regarding the rights of consenting adults to pursue happiness in equal measure just like anybody else. Yes, each one of us has the right to not like it. We each have the right to not care. We each even have the right to say we don’t like it, or don’t care. But we don’t have the right to deny such pursuit to one or more persons, even to billions of people. That includes the right to buy a fricken cake in a store that declares itself open to the public. Any other decision would harken back to the days when as a woman I could be denied the right to walk into a public clubhouse, tavern, or other such business, denied work, or denied the right to solely own property, the list goes on, simply because I was considered to be unequal and subservient to men. Yes, I mean subservient. In 1901 my greatgrandmother had to resign as principal of the Lostine High School. Why? She got married and her place was in the home serving her husband. Married women were banned from a teaching career the moment they said “I do”.
Not to put too fine a point on this, would I marry a woman? No. Not my cup of tea. But I would take up arms to protect my fellow gender’s individual freedoms specified in our founding documents.
Ah, well, the ‘pursuit of happiness’ is about as poetic as Kennedy’s fare when normally considered. Consider: Ted Bundy was free to pursue his means of getting his jollies off by murdering women. That is, the government did not ban his existence or incarcerate him in advance and on account of his chromosomal makeup. But being free to pursue his happiness — being free of prior restraint — doesn’t mean that we are required to let him go unpunished for what things he’s gotten himself up to.
But if you want to get into historical points: ‘all men’ always meant ‘all humanity,’ this has always been well understood. And while unfortunate, the 14th Amendment was never intended to apply to ‘real’ biological differences. When it was ratified it was explicitly and expressly not meant to equalize law between men and women. Only to prohibit differences in law based on politically constructed designators — such as ‘black,’ ‘white,’ or ‘asian.’
But with respect to homosexual marriage, the government does prevent you from pursuing your happiness by shacking up and getting horizontal with whoever you like. But it does install a difference in law on the basis of whether you’re married or not. Which, should be noted, was also expressly denoted as a violation of the 14th Amendment. The solution here isn’t for government to acknowledge more and different kinds of marriage so that more and different kinds of religiously or sensually motivated couplings can get preferential treatment under law — it is to ask the government to start obeying and enforcing the law as it has been written for around 150 years.
Don’t get wrapped up too deeply in the ever shifting political climate.
Do you know all 57 genders currently available for selection; probably will be used in the next census ??
I’m not sure if Hermaphrodite is even one of the 57.
g
In my experience, the problem has been that too many people underestimate the amount of random variability, or don’t believe in it at all. In large numbers, those people will have overconfidence in the power of the their experiments, and overconfidence in their results, no matter what disciplines of statistics they employ. The documented problems with inability to replicate research results will continue. All these issues have been publicly debated in statistics and in psychology for at least 50 years.
“Journal editors attempt to judge which papers will have the greatest impact and interest and consequently those with the most surprising, controversial, or novel results,” Reinhart points out. “This is a recipe for truth inflation”
For climate papers it must work opposite.You find hardly any controversial results, but instead a lot of papers confirming the same old story seen from different angles.
In engineering design we use tools to perform simulations to help us design things, and to determine if the design will do what is desired. Generally, these tools work pretty well, although it is common to end up “at the bleeding edge” where results are not so consistent.
Without regard to that, one way to help insure the outcome is to never use parts that have a tolerance larger than 10% of the component value. Designs with this approach tend to be well behaved over time/temp/build.
But as I’ve said before, the model output is no guarantee of the real world performance. A very good engineer will almost always require two “spins” of the Real World design to “get it right”… and then it will still require some tweaks in production.
When I see anything where the Tolerance (possible error) is more than 10% of the Value, then I Know that the output will be shaky or outright crap.
If we add the possibility of measurement error and noise in the measurement (noise is often not evenly distributed, therefore unpredictable) that is also on the same order as the desired measurement… then all is lost.
Anyone who has worked in design or a production environment can tell you these things. The only truth is what you can measure. And a good production engineer can shake you confidence in your data in ways you cannot imagine.
that is why we shoot the engineers and ship the damn product.
Way too much of that going on. Is that a personal philosophy or a critique?
If you design a zoom lens for your Canon or Nikon SLR camera, and you use 10% tolerances on variable values, you won’t even end up with a good Coke bottle.
But then again, I have designed amplifiers with 20% tolerance components or even a range of 10:1 on some parameters (forward gain) but then desired operation is restored with just two 1% or even 0.01% tolerance components.
As they say; it all depends.
g
“Should Mr. Siegfreid read this, I’ll point out that many climate skeptics became climate skeptics once we started examining some of the shoddy statistical methods that were used, or outright invented, in climate science papers.”
One of the many reasons I no longer subscribe to “Science News”. It’s not about science any more. It’s about editorials and “global warming” and sensationalism. And the writing has been dumbed down. It’s a real shame.
Given the birthday paradox, the probably of two independent variables both being outside the bog-standard 95% confidence is 1-e^(-n(n-1)/2*possibilities), where possibilities is 20 and number of variables n = 2, that’s 4.9%. With 5 variables that’s a 40% chance, and with 10 variables it’s a 90% chance that two variables are outside the confidence interval. I haven’t calculated it for “two or more” yet, but I should.
Too bad there are two threads on this. not sure where to post this.
If you reverse this and you want 95% confidence that no two independent variables are outside their respective 95% confidence interval, for n=5 variables you need p=0.008 for each variable and for n=10 you need p=0.001 for each variable. (found via goal-seek in Excel).
This analysis should be distribution independent but IANNT (I Am Not Nicholas Taleb)
Since most measurement error bars are posted at 95% confidence (2-sigma), then this applies to real world measurements. If I combine those measurements into a model I’ll get increasingly likelyhood (quickly!) of GIGO as I add measurements to the model. It should also apply to multiple ANOVA or any model that involves multiple variables that involve some sort of distribution of those variables.
Feel free to smash away at my bad assumptions and math. If you really need help programming the simple equation into Excel I’ll post it to dropbox on request…
Peter
references:
https://en.wikipedia.org/wiki/Birthday_problem#Approximations
“Rather than furthering scientific knowledge, null hypothesis testing virtually guarantees frequent faulty conclusions.”
Yup. so much for the natural variability null.
Science is more about understanding and less about null testing than people think.
especially in the observational sciences.
The other odd thing is that a while back folks clamored for more statisticians in climate science
Note that generalizing from papers about psychology to other fields is an unwarrented assumption
Except that it isn’t. Why? Because unknown variables are replete in both areas of research: psychology and climate. There is a mathematical construct you are missing Steven. A high number of variables will more likely create an actual false positive (rejects the null) than it will a false negative (erroneously accept the null), irregardless of statistical choices or machinations. Climate scientists fail to tread lightly through their fragile data and instead stomp all over it as if it is hardy and robust against error.
meant (erroneously rejects the null)
“Except that it isn’t. Why? Because unknown variables are replete in both areas of research: psychology and climate. ”
Assumes the same effect in both fields.
Pamela my dear, you are indeed an exotic creature. On the one hand, you consistently post highly intelligent comments, which I almost always enjoy, and on occasion — despite myself — actually learn something new. And then, as if to toy with us, you deploy a word like “irregardless.”
It’s the little things that bring color to an otherwise Gray world.
🙂
Lol! It’s the country bumpkin in me. The double negative.
I prefer science than has practical applications … electrical, mechanical and chemical engineers don’t need statistics … [they] don’t get paid to have something work 95% of the time …
never designed a chip.
Do U know why Intel first did speed binning? what was the design speed of the first processor to be speed binned?
or better what its the intial defect density of a chip design?
how does chip yeild ( the % that actually work) increase over the production life span
LOL! Even though we often differ in our opinions, your breadth of knowledge is impressive.
Well binning for any variable is a way to maximize the economic return.
If a memory chip on a wafer may have a 3:1 spread in speed over the wafer for various process variable reasons, then by binning them for speed, you can sell all the functioning units to somebody for some price.
You can get mucho bucks for the 5GHz ones, and the 1.5 GHz ones are fast enough for people to use to write Microsoft word documents.
I once asked a semiconductor salesman what was the cheapest semiconductor device he ever took an order for.
He said he sold a bunch of silicon diodes to a crystal set kit manufacturer for one cent each. Typical fast switching signal diodes were going for maybe $1.20 in 10,000 quantity.
So I asked him, what was the spec.
His reply: NO opens. NO shorts
He could have sold the guy AB resistors ( at a substantial loss) and they wouldn’t even work.
” or better what its the intial defect density of a chip design?
how does chip yeild ( the % that actually work) increase over the production life span”
Defects are proportional to the area of a chip and I suppose the % of that area that has active area, and if you fab is consistent the only increase you get is when you decrease the die area, and binning was just a way to sell more die, nothing was wrong with them but process tolerance was squishy and some (or a lot) were just slow, our tolerance was about 20% iirc, it was almost 35 years ago.
But all of the dead die were almost all photo resist flaws, rarely did I see a wafer that was missing a layer.
No reference to the failure of reproducibility in the bogus sciences is complete without reference to the sustained generation of fake results and papers by the con-man social psychologist Diederik Stapel.
Since his work was highly referenced and he found many imitators during his years of “success”, the discovery that his work was almost entirely faked must surely cast serious doubts upon the robustness of related work by other so-called “researchers”.
It is especially interesting to note that people were willing to accept and report his results because he was fishing for results that supported tragically simplistic progressive assumptions.
As with Lewandowsky, his secret was to tell the political left exactly what it wanted to hear.
It turns out that apparently skeptical people will cease to take interest in the trustworthiness of a “scientific result” if it fits into what they wanted to believe about the world.
http://www.nytimes.com/2013/04/28/magazine/diederik-stapels-audacious-academic-fraud.html?pagewanted=all&_r=0
“Robust statistics seeks to provide methods that emulate popular statistical methods, but which are not unduly affected by outliers or other small departures from model assumptions.” Oh dear, shurely some mishtake. Ignore those pesky outliers in favour of a pre-conceived closed-form presumption.
Multi-modal assessment of reality. Whether through the nature of things (aleatory) or our lack of knowledge of things (epistemic), the evidence is that we can most reasonably describe the world as multi-modal – many possible outcomes. Not at all the familiar view of uncertainty as a variability around a mean outcome.
Colin Powell said it:
“Tell me what you know. Tell me what you don’t know. Then tell me what you think. Always distinguish which is which.”
from “It Worked for Me” (http://www.amazon.co.uk/It-Worked-Me-Life-Leadership/dp/0062135139)
As Lubos Motl so nicely put it in http://motls.blogspot.com/2010/03/tamino-5-sigma-and-frame-dragging.html :
That’s why until I understand , ie : have the algorithms running , to calculate , ie : quantitatively explain , our 3% excess over the gray body temperature in our orbit , to the accuracy of our measures of the spectrum and distance of the sun and our spectral map as seen from outside , ie : ToA spectral map , I don’t see 5-sigma , or even the business world’s 6-sigma any constraint .
James Hansen’s quantitative howler that Venus is explained as a runaway greenhouse effect is way outside those bounds yet not universally repudiated . So , until the field ( not just the best here ) understands the non-optionality of those calculations of radiative balance , I’m far more interested in getting the stack frames vocabulary in my 4th.CoSy solid and defining the recursive operators required to express these computations succinctly .
My 2013 comment to WUWT is germane to this argument. I will add another quote from the article which should be mandatory reading for anyone delving into statistics:
“William Feller, Higgins professor of mathematics at Princeton, is in a fighting mood over the abuse of statistics in experimental work.”
http://wattsupwiththat.com/2013/05/14/the-beginning-of-the-end-warmists-in-retreat-on-sea-level-rise-climate-sensitivity/
Neil Jordan May 16, 2013 at 1:32 pm
Re rgbatduke says: May 14, 2013 at 10:20 pm
Abuse of statistics is also covered in this old article which is unfortunately not on line:
“A Matter of Opinion – Are life scientists overawed by statistics?”, William Feller, Scientific Research, February 3, 1969.
[Begin quote (upper case added for emphasis)]
To illustrate. A biologist friend of mine was planning a series of difficult and laborious observations which would extend over a long time and many generations of flies. He was advised, in order to get “significant” results, that he should not even look at the intervening generations. He was told to adopt a rigid scheme, fixed in advance, not to be altered under any circumstances.
This scheme would have discarded much relevant material that was likely to crop up in the course of the experiment, not to speak of possible unexpected side results or new developments. In other words, the scheme would have forced him to throw away valuable information – AN ENORMOUS PRICE TO PAY FOR THE FANCIED ADVANTAGE THAT HIS FINAL CONCLUSIONS MIGHT BE SUSTAINED BY SOME MYSTICAL STATISTICAL COURT OF APPEALS.
[End quote]
Correction: I was able to locate the article on line at:
http://www.croatianhistory.net/etf/feller.html
The PDF can be downloaded here:
http://www.croatianhistory.net/etf/feller_too_much_faith_in_statistics.pdf
Penultimate sentence of the post “pre-screeing data” should read “pre-screening data”.
Oops. Not penultimate. Final(ultimate) sentence.
Oops, not penultimate but final(ultimate) sentence.
Third attempt to correct my error.
Should be final sentence not penultimate.
[Regardless of the final sentence location, “screeing” is changed to “screening” 8<) .mod]