Here’s a quote related to the McShane and Wyner discussion brought to light thanks to Gavin Schmidt and Michael Mann at RealClimate that I happen to agree with. Yes I know, that’s a shock to some. This quote is from L. Mark Berliner in discussion here (PDF) and speaks powerfully to the whole of climate science:
The problem of anthropogenic climate change cannot be settled by a purely statistical argument. We can have no controlled experiment with a series of exchangeable Earths randomly assigned to various forcing levels to enable traditional statistical studies of causation. (The use of large-scale climate system models can be viewed as a surrogate, though we need to better assess this.) Rather, the issue involves the combination of statistical analyses and, rather than versus, climate science.
That’s a keeper.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Eric Worrall says:
December 14, 2010 at 5:12 am
In other words We know we are right, regardless of what the data says.
What DATA ? if it is HIDDEN!
See:
http://wattsupwiththat.com/2010/12/12/an-open-letter-to-dr-subra-suresh/#more-29321
This quote isn’t bad either (from the discussion paper by Nychka and Li 2nd para.)
“Although Section 2 of this paper is lively reading, we feel that the viewpoint is not balanced and emphasizes statistical correctness over the broader issues of scienti c understanding.”
Statistical correctness gone mad?
I think many of you are missing the point that Anthony is making.
Yes, much statistical analysis can be performed on the weather record. The problem is that we only have ONE weather record. In other words, we only have one data point.
This is the same problem quantitative modelers have with the stock markets. There is only one stock market history to use.
What would make statistical analysis better is if we had 20 to 100 weather records or 20 to 100 stock market histories.
RR Kampen says:
December 14, 2010 at 6:27 am
Nope. Continental drift has been measured. No need for anything “statistical” there.
I am wondering why you cannot run a controlled climate experiment to test the CO2 hypothesis?
Why couldn’t you fit a football stadium (I understand the Metrodome recently became available) with a infrared neutral roof, flood 3/4’s of the field with saltwater (just like Earth) then fill it with a couple of thousand ppm of CO2. You could control temperature, CO2 concentration, air and water circulation, and surely if it was as big as a stadium you should see measurable changes if the theory is correct.
As an added bonus, if AGW is right, you should go well over the tipping point and the temperature should rise to 50 degrees centigrade.
All you have to do then is rig up some heat pumps and Bingo! you can shut down every coal powered plant on Earth and replace them with free CO2 powered football stadium heat generators solving global warming!
Seriously though, does any one know if large scale climate (physical) models have ever been attempted?
Quote
The problem of anthropogenic climate change cannot be settled by a purely statistical argument… Rather, the issue involves the combination of statistical analyses and, rather than versus, climate science.
Unquote
I think that means that if the statistics don’t give you the answer you want, you have to fiddle with the results a bit.
RR Kampen says:
December 14, 2010 at 6:27 am
“This proves we can also do away with the theories of evolution and continental drifting.”
Not so. There is ample physical data that has been collected, and can be statistically analyzed, to support these to theories.
For continental drift, there is magnetics, plate motions, paleo, and radiometric dating to support the theory.
For evolution, the entire field of cladistics centers around statistical methods for testing osteological characteristics to identify primitive and derived forms in similar organisms. Don’t forget DNA analysis.
Climate science is quite different, and as the quote points out we are yet to sort out cause and effect making statistical analysis difficult. That’s a biggy!
During my first seven years after graduating (physics) I took part in the statistical analysis of a very complex industrial problem. The science was based on some Nobel prize winning work and was thought to be well understood – but the results did not always correlate with theory. The project involved about 50 engineers and scientists over this period so we were not short of resources. Moreover we were able to measure thousands of variables related to the process and the product with great accuracy.
I won’t go into the detail (although I think it is fascinating) but the fact is that the thing that stopped us discovering the truth was our belief in where the truth lay. We were measuring things we thought should be important and not measuring things that happened to correlate with what was actually happening. In other words the science got in the way of discovery.
Half way through the project we adopted a technique called variation analysis developed by Shanin. His approach was to ignore preconceptions and just look at the data and learn to read the story it told. In discussions no one was allowed to make any statement without having the data at hand to prove the conjecture. It was very frustrating but, at the same time, stimulating and instructive.
For this to work the data had to be relatively niaive.
For example in our case we had many batches of products made in lines built of many machines run continuously. His first question would be “where do we find the biggest variation?” Is it :-
Within a piece
Piece to piece
Batch to batch
Time to time
Machine to machine
Line to line etc.
Even this simple approach was not without its traps. It was two years before we realised that one of the dependent variables we were trying to optimise was actually varying within the product by more than the amount it varied between products. This stupid oversight was caused by the fact that when it varied within the product we called it a blemish!
There are two reasons for recounting this experience.
The first is that all the breakthroughs came from the statistics alone. Only when we had good data showing clear correlations with things we had not expected could we turn on the search light of experimental science to determine the cause. Only after that could we use our theoretical knowledge to explain the causes. Climate scientists try to do this in reverse and it is impossible.
The second point is one I have made before. If I were doing research in climate I would use variation analysis to try and generate more data to inform the debate about mechanisms.
We already know that the variation is not the same in every location but by taking an average we throw away the information that comes with that distribution. We then throw away more information by taking the average temperature during the day or month or year.
I feel all this smoothing and averaging is using statistics to produce pretty pictures and not to generate insight.
The problem is that we already know the main variations.
We know that the biggest variation are: within the world, within the year, within the day. We also know of other variations like the Milankovitch and ocean cycles. Trying to determine even the reality of a long term drift in the face of these huge effects is daunting. I do not think it has been done yet. If we do find a trend I believe we will find it strongest over land or over sea, during winter or summer, during the night or the day. These facts will inform the cause and not the other way round.
That’s an OK quote and if interpreted literally and carefully, I subscribe to it, too.
However, in reality, this kind of quotes is used for something totally different than what it actually does. The quote is used as a synonym of the following argument:
“If you find errors in our statistics, it doesn’t matter because we’re also climate scientists. If you find errors in our climate science, it doesn’t matter because we also have some statistics.”
In other words, this kind of an argument is used to deflect any criticism and incoherently change the focus of any discussion whenever it becomes inconvenient. Needless to say, people like Gavin Schmidt and Michael Mann are as lousy climate scientists as they’re lousy statisticians so the equal treatment of the two approaches is legitimate, indeed. 😉
I agree with the statement and I suspect that nearly all climate scientists would as well. Everyone knows that experiments provide much better certainty of interpretation than observations. In medicine, ecology and other fields, experimental studies have brought the most rapid progress. However, since we have only one earth and no controls or replicates, we need to due the best that we can with observational data. The situation is comparable to understanding the role of smoking in cancer. We can’t do a randomized, controlled experiment, but at some point the observational/correlation data become convincing.
The whole McShane-Wyner “Rejoinder” is worth reading. Rather like a firm swat-down of an unruly student by a tolerance-limited teacher.
Fundamentally, they show temperature proxy data possess little signal and much noise, especially tree rings (duh). When Schmidt, Mann, and Rutherford (SMR) select 59 proxies from 95, McS-W “can only be skeptical of such improvisation” because “the application of ad hoc methods to screen and exclude data increases model uncertainty in ways that are unmeasurable and uncorrectable”.
McS-W also note SMR’s use of RegEM code (rather than simpler, more tested stats code) is questionable. “RegEM appears to be a classic, improvised methodology with no known statistical properties” and “we cannot rule out the possibility that RegEM was tailor-made to the specific features of this simulation”.
Wow. Cherry-picked code used on cherry-picked data. That must be what Rosanne D’Arrigo meant about making cherry pie.
In the end, McS-W conclude “climate scientists have greatly underestimated the uncertainty of proxy-based reconstructions and hence have been overconfident in their models”.
Class dismissed.
vukcevic says:
December 14, 2010 at 4:55 am
Irrelevance of the CO2 global warming hypothesis
. . . . . One thing is clear that on any relevant time scale the CO2’s heat retention (storage capacity) is insignificant.
I believe this is the first time I have seen this “heat retention” of carbon dioxide characteristic (or lack thereof) expressed. I guess I’ve missed important contributions to the research literature.
That sounds awfully similar to the 2nd paragraph of my “An Atheist’s View of Global Warming” where I state:
“The most basic tenet of AGW is that correlation must equal causation – because carbon dioxide has increased at the same time temperature has increased then one must be causing the other. Unfortunately, correlation has never and can never prove causation – the only way to establish causation with regards to global warming would be to take a few identical planets, alter their CO² levels and then measure the effects. This is clearly beyond our current means and since such a scientific experiment is not possible, the AGW hypothesis cannot be disproved which makes it completely at odds with the scientific method.”
I’m happy to see a high-level statistician agree with me since, while my statement was based on statistics, I’d only taken a few courses in it.
If anyone is curious then here is the rest of my article:
http://pediawatch.wordpress.com/2010/02/10/atheist_global_warming/
Really?
I wasn’t aware that there was a resonable way to divorce climate science from the statistical sciences. How would one do that? How can you have a versus when you are dependent, one upon the other. Large data-sets and dBases, smoothed station data, etc.
The “bone” has been that those people who make a livelyhood (spent their professional lifetime) in the statistics field have gone over the methodologies, and data, of the climate scientists: they found their conclusions inappropriate and their methods wanting, according to accepted, proven, verified methods of statistical data analysis.
The “bone” was picked by climate scientists when they diputed the statisticians results were “flawed” because they couldn’t appreciate the “nuances” of climate science. Numbers are numbers, period. The same numerical/mathematical laws apply to climate science data analysis as they do to particle physics or any other scientific discipline that deals with large data-sets.
When I read comments like the original quoted material, I wonder if these people are even scientists at all. There are two ways to determine the validity of an hypothesis: perform experiments or make predictions and verify their accuracy.
Obviously while verifying Wegener’s and duToit’s hypothesis of “continental drift,” there was no way to perform experiments on the Earth’s crust. But predictions were made of what to look for in crustal sediments, and the discovery by the Glomar Challenger of the geomagnetic reversal striping at the Mid-Atlantic Ridge are considered the definitive evidence for plate tectonics.
Verifying the theory of evolution proceeded in much the same way, by predicting structures and fossils that should exist if that mechanism was at work. I see no reason why climate science can not function in the same mode. The problem I do see is that when predictions are made, and then fail to materialize, the lack of evidence is handwaved away and more tweaks and exceptions are made to the “theory.” Epicycles come to mind when I see this taking place.
Until climate science becomes dominated by people who are seeking the truth, and not looking to validate their personal agendas, it will remain the Lysenkoism of earth science.
“Rather, the issue involves the combination of statistical analyses and, rather than versus, climate science.”
What about a statistical analysis of some climate science such as
CO2 going up steadily http://www.esrl.noaa.gov/gmd/ccgg/trends/ and now at 390ppm. No let-up or pause.
Ocean heat content remains stable since 2003 http://www.climate4you.com/
Global sea ice remaining steady within natural variation (N/S combined)
Falling sea levels now http://sealevel.colorado.edu/
No statistically significant atmospheric warming since 1995. Dr Jones and Dr Spencer
http://www.drroyspencer.com/latest-global-temperatures/
How long before the penny drops amongst the AGW supporters that perhaps CO2 has less of an effect than the “alarmists” will have us believe. To my simple mind these indicators, ocean heat, sea ice, sea level and atsmospheric temperature are the real indicators. All steady now or heading down. Getting ready to showel snow again Thursday here in Scotland.
Steve McIntyre has an excellent discussion going on at Climate Audit!
http://climateaudit.org/2010/12/14/mcshane-and-wyner-discussion-2/
I don’t think anyone is claiming otherwise. The claim is that it’s difficult to use statistics to prove/disprove global warming since we only have ONE set of historical temperature data (for the one planet on which it was recorded). Ideally, statisticians prefer to have large number of data sets in order to compare.
Obviously, this will never be possible since there is only one earth and earth only has one history.
I’d like to add something to my above earlier comment. Stock market modelers are still capable of being successful in predicting market movements by “creating” multiple stock market histories by comparing multiple segments of history to other similar segments of history. This is possible because of the huge amounts of accurate data (which doesn’t need to be “adjusted”) related to the stock market. But good luck using the pathetic excuse of a weather record that exists in the world to make this work for climate predicitions.
In other words, AGW is a “wicked problem” as stated by Judith Curry. If you’re not familiar with the definition, take a look:
http://en.wikipedia.org/wiki/Wicked_problem
There’s also some interesting information on the page about “social messes” and “super wicked problems”.
Patrick Hadley says:
December 14, 2010 at 6:36 am
Hansen had claimed that closing the power station would be justified because their actions were designed to prevent immediate harm to human life and property from climate change.
His sworn statement of evidence contains a lot of rather questionable opinion presented as fact. http://www.guardian.co.uk/environment/interactive/2010/dec/14/james-hansen-evidence-ratcliffe
I wonder if an action for perjury would be possible? A legal opinion would be “interesting”!
L. Mark Berliner is absolutely correct. I am a statistician working with a group of materials scientists. Statistics cannot tell you to violate a physical law. If Galileo had used regression analysis on his data, he would not have come up with the law of the pendulum. Statistics alone can lead you into confusion unless you have an understanding of the process.
I had hoped this was an opportunity for committed researcher to work through a difficult physical / statistical problem and arrive at at least a solution. I think the discussion on RealClimate has been much more reasonable. Finding nothing here but a strange triumphalism (over what I cannot divine?) and a willingness to pursue trench warfare, I will take my leave.
thegoodlocust says:
December 14, 2010 at 9:37 am
That sounds awfully similar to the 2nd paragraph of my “An Atheist’s View of Global Warming” where I state:
“… the only way to establish causation with regards to global warming would be to take a few identical planets, alter their CO² levels and then measure the effects. This is clearly beyond our current means and since such a scientific experiment is not possible, the AGW hypothesis cannot be disproved which makes it completely at odds with the scientific method.”
As has been pointed out in some of the other replies, this view is a misunderstanding. Of course you can establish causation based only on observation without having to do an experiment. Many branches of science (geology and biology has been mentioned) is based on this as well as much of the legal system and indeed basic common sense. You *can* prove causation beyond anything but the most absurd counter arguments without having to do an experiment, but it requires meticulous collection and analysis of data.
To me, it seems statistical analysis of physical observations/measurements rule the day. I agree with Cal at 9:05, having helped implement Statistical Quality Control in IC manufacturing beginning in 1980 at a large US firm. Measuring and minimizing the correct variable(s) was paramount. The most counter-intuitive things can be found out through rigorous data collection and analysis. The bulk of what I’ve seen supporting a dramatic affect from CO2 doesn’t come close to what is the daily routine of scientific diligence in IC manufacturing.
dixonstalbert says: December 14, 2010 at 8:58 am
I am wondering why you cannot run a controlled climate experiment to test the CO2 hypothesis?
[…]dixonstalbert says:
December 14, 2010 at 8:58 am
I am wondering why you cannot run a controlled climate experiment to test the CO2 hypothesis?
Why couldn’t you fit a football stadium […]
Seriously though, does any one know if large scale climate (physical) models have ever been attempted?
I think that is a great idea. There are some big dirigible hangers too, one along 101 in CA, south of San Francisco, IIRC. I heard they are big enough to have weather inside, at least to the extent of cloud formation.
Jim Cole, 12-14 9:24 am;
“In the end, McS-W conclude “climate scientists have greatly underestimated the uncertainty of proxy-based reconstructions and hence have been overconfident in their models”.”
Translation:
Climate scientists express totally unwarranted confidence in nearly meaningless data, and are talking through their simulated hats.
Some great comments here. I’ll mention cal and also JamesS, but there are a lot of others that hit the nail on the head, like GregR. JamesS said:
“Until climate science becomes dominated by people who are seeking the truth, and not looking to validate their personal agendas, it will remain the Lysenkoism of earth science.”
Given the way the field has developed, the corruption of major science and MSM publications, and the amount of tax money still being poured into AGW propaganda, I don’t look for a lot of truth from climate science for another ten years, minimum. This is going to be a long haul.