Guest Post by Willis Eschenbach
On an average day you’ll find lots of people, including NASA folks like Gavin Schmidt and James Hansen, evaluating how well the climate models compare to reality. As I showed here, models often don’t do well when matched up with real-world observations. However, they are still held up as being accurate by the IPCC, which uses climate models throughout their report despite their lack of rigorous testing.
XKCD, of course.
But if you ask me, that evaluation of the models by comparing them with reality is not possible. I think that the current uncertainties in the total solar irradiation (TSI) and aerosol forcings are so large that it is useless to compare climate model results with observed global temperature changes.
Why do I make the unsubstantiated claim that the current uncertainties in TSI and aerosols are that large? And even if they are that large, why do I make the even more outlandish claim, that the size of the uncertainties precludes model testing by comparison with global temperature observations?
Well … actually, I’m not the one who made that claim. It was the boffins at NASA, in particular the good folks at GISS, including James Hansen et al., who said so (emphasis mine) …
Total solar irradiance (TSl) is the dominant driver of global climate, whereas both natural and anthropogenic aerosols are climatically important constituents of the atmosphere also affecting global temperature. Although the climate effects of solar variability and aerosols are believed to be nearly comparable to those of the greenhouse gases (GHGs; such as carbon dioxide and methane), they remain poorly quantified and may represent the largest uncertainty regarding climate change. …
The analysis by Hansen et al. (2005), as well as other recent studies (see, e.g., the reviews by Ramaswamy et al. 2001; Kopp et al. 2()05b; Lean et al. 2005; Loeb and Manalo-Smith 2005; Lohmann and Feichter 2005; Pilewskie et al. 2005; Bates et al. 2006; Penner et al. 2006), indicates that the current uncertainties in the TSI and aerosol forcings are so large that they preclude meaningful climate model evaluation by comparison with observed global temperature change. These uncertainties must be reduced significantly for uncertainty in climate sensitivity to be adequately constrained (Schwartz 2004).
“Preclude meaningful climate model evaluation” … hmmm. Of course, they don’t make that admission all the time. They only say things like that when they want to get money for a new satellite. The rest of the time, they claim that their models are accurate to the nearest 0.15°C …
Now, the satellite that the NASA GISS folks (very reasonably) wanted to get money for, the very satellite that the aforementioned study was written to promote, was the Glory Mission … which was one of NASA’s more unfortunate failures.
NASA’s Glory Satellite Fails To Reach Orbit
WASHINGTON — NASA’s Glory mission launched from Vandenberg Air Force Base in California Friday at 5:09:45 a.m. EST failed to reach orbit.
Telemetry indicated the fairing, the protective shell atop the Taurus XL rocket, did not separate as expected about three minutes after launch.
So … does this mean that the evaluation of models by comparison with observed global temperature change is precluded until we get another Glory satellite?
Just askin’ … but it does make it clear that at this point the models are not suitable for use as the basis for billion dollar decisions.
w.
Bishop Hill / Josh has something almost designed for Willis’ thread:-
http://bishophill.squarespace.com/blog/2011/5/1/climate-no-science-josh-96.html#comments
Climate ‘science’ seeks a new low here. As the facts are becoming increasingly inconvenient for the Team and the AGW cult, they have now found something much more reliable than facts and data, namely people’s long ago memories!
Well, I suppose just about anything is more accurate than an IPCC climate model, but this may be pushing the envelope out just a little too far.
http://www.bbc.co.uk/blogs/thereporters/richardblack/
Ferd Berble’s comments here about climate models are some of the best I have seen.
Start with pre-conceived ideas and then make a computer model to fit those pre-conceived ideas by careful weighting of certain, hard to detect, input parameters to match the past and your expectations of the future – and voila, it becomes reliable fact accepted by the Team and the IPCC.
No wonder climate ‘scientists’ never want to release their code and interpretation methodology to the outside world.
Re Peter Miller says:
May 1, 2011 at 2:09 am
Peter, now this is interesting. I have to deal with a branch of post normal science in which anything like this is dismissed as “anecdotal”!
They will eventually find that good anecdotal beats junk science hands down
Thanks Willis. Right on the button, as usual, but never boring!
‘Dagfinn says:
April 30, 2011 at 11:12 pm
Hansen’s “99 per cent certain”…“If we also burn the tar sands and tar shale, I believe the Venus syndrome is a dead certainty.” ‘
As far as I know Hansen’s “Venus Syndrome” is still just a hypothesis with little facts to support it.
A company that I worked for has been making the same product since the 1940’s, so they know every detail of the product intimately. Computer modeling was used from the earliest main frame computers to super computers. Development testing of the product was very expensive, costing millions of dollars. To bring and new product online cost in the neighborhood of a billion dollars. To develop a product faster and at a lower cost, management pushed for more computer modeling and less product testing. This replaced the old-timers philosophy of “build them and bust them.” Also, “You don’t advance the technology unless you have a few failures,” and ” If you don’t have a failure your design isn’t agressive enough.” Computer models were indeed a benefit, but Mother Nature can indeed be cruel. Models did not eliminate test failures, which led to costly delays and expensive redesigns. In this business, unlike climate science, it was not possible to tweak the experimental data, especially when a poduct explodes, to agree with the model. Models can be valuable “tools” to understand the theory, but need to be constantly matched against empirical data. Events such as the Pinatubo eruption were used to advance understanding. Conversely, there seems to be a lack of effort to model the cooling since 1998 to understand what is going on. There are, however, efforts to match the data to the models.
@kwik
“On Michael Chrichtons’s (rest in peace) official homepage we could read this a couple of years ago;
http://www.crichton-official.com/speechourenvironmentalfuture.html
From IPCCS ”Third Assessment Report” ;
“In climate research and modeling, we should recognize that we are dealing with a coupled non-linear chaotic system. and therefore that the long term prediction of future climate states is not possible.”
Unfortunately all his climate stuff is now removed from his homepage.
I think most scientists realize that this is a basic fact.”
I believe you might find all his writings in the archives:
http://web.archive.org/20050101000000*/http://www.crichton-official.com
When the models don’t agree with reality how do they know if the problem is in the physics model itself or whether they have a bug in their programming?
Dave W
The errors in computer modeling – and the struggle by warmists to hide those errors – are reflected in the Orwellian evolution of their terminology.
We’ve gone from “global warming” to “climate change” to “climate disruption” to “extreme weather” to the more recent “reframing” (inspired by George Lakoff) of “climate pollution” and currently the more nebulous “greenhouse gas pollution.”
All to describe the same and exact speculative hypothesis (CO2 forcing) that I’ve found in New York Times archives dating back to the mid-1970’s. The only thing that’s advanced in climate “science” is its evasiveness in both terminology and practice.
Was doing some other background research on another subject today and had seen these data/phenomena papers and downloaded to read later. So here they are, if of any use to your work Willis. I have not read them as yet. Theo G spoke of Kuhn some time ago which to my mind was interesting.
1. Harris T (2002) Data Models and the Acquisition and Manipulation of Data Philosophy of Science 70 (5) Proceedings of the 2002 Biennial Meeting of The Philosophy of Science Association
This paper offers an account of data manipulation in scientific experiments. It will be shown that in many cases raw, unprocessed data is not produced, but rather a form of processed data that will be referred to as a data model. The language of data models will be used to provide a framework within which to understand a recent debate about the status of data and data manipulation. It will be seen that a description in terms of data models allows one to understand cases in which data acquisition and data manipulation cannot be separated into two independent activities.
References of this paper are below, as could be gleaned from the web.
2. Bogen J & Woodward J (1988) Saving the Phenomena The Philosophical Review 97 (3) July p303-52
Our general thesis, then, is that we need to distinguish what theories explain (phenomena or facts about phenomena) from what is uncontroversially observable (data). Traditional accounts of the role of observation in science blur this distinction and, because of this, neglect or misdescribe the details of the procedures
by which scientists move from claims about data to claims about phenomena. In doing so, such accounts also overlook a number of considerations which bear on the reliability of scientific knowledge.p314
3. Woodward J (1989) Data and Phenomena Synthese 79(3) June p393-472
4. Nagel E, Suppes P & Tarski A (1962) Data in Models Logic, Methodology and Philosophy of Science Stanford Uni, eds. Proceedings of 1960 International Conference p252-61
http://suppes-corpus.stanford.edu/articles/mpm/41.pdf
5. Hunting for another author and this popped up – Podnieks (2010) The Limits of Modeling University of Latvia
http://philsci-archive.pitt.edu/5475/1/Podnieks_Limits_of_Modeling.pdf
(2009) Is Scientific Modelling an Indirect Methodology
http://www.ltn.lv/~podnieks/
ferd berple says: April 30, 2011 at 10:15 pm
Illuminating.
I had understood they used various teams of researchers from a multitude of professions and thus [experts in their chosen] variables. Seemingly unconnected, until someone comes up with the grand narrative (usually when the science is questioned) to achieve this result.
Y’know. Putting proper error bars on projections is quite difficult. Nonetheless, I think that if “climate scientists” made an honest attempt to do so, we’d all learn a lot. Why don’t they? The obvious answers are a) it’s hard to do. b) It would probably reveal that some of their past claims may not have been very realistic. And c) that future funding has at least as high a priority in their minds as searching for truth.
I sort of think that models with realistic error estimates might actually be good for something. Particularly when/if climate science grows up and the error bands shrink.
“ferd berple says:
What you do is take historical temperatures for the past 150 years, then splice a synthetic (artificial) data set to this, for the next 150 years into the future, with temperature going up at 3C per doubling of CO2.
You then train the model for the entire 300 years; so that the weights give you are reasonable fit over the entire period. Then you remove the artificial (future) dataset. The resulting model will then recreate the past accurately, and continue to predict the future that you have trained it to predict. Giving the level of CO2 sensitivity you built into the future data.”
That is a fit, not a model. A model has to have some basis in reality and each of the constants used have to have a known elasticity.
To work out the elasticity of each input parameter you have to change it and keep all the other constants the same. If a constant is altered by, say 0.05%, causes a change in output by 50%, then you have an over reliance on a single input.
Each model should show the output, and the outputs for each input, when changed bu +/-5%.
When ever you give curve fitting programs to young graduates, they fall in love with polynomial fits, you can tell them that these fits are only good for performing calculus, but they love them anyway.
B. Kindseth at 4:14 am: This replaced the old-timers philosophy of “build them and bust them.” Also, “You don’t advance the technology unless you have a few failures,” and ” If you don’t have a failure your design isn’t agressive enough.”
I watched a fascinating documentary recently about the development and deployment of the $1+ billion Hughes Glomar Explorer and its successful “blacks ops” attempt to recover a sunken Russian sub in the 1970’s (it’s called “Azorian: The Raising of the K-129”).
It’s a fascinating engineering story, and several project and engineering managers are interviewed at length. Naturally, they make comments about the engineering tests, failures, and delays, which are all explained as necessary elements of the project.
My question is: What’s the difference between a model and a picture? It seems that if one “parameterizes” enough, what people call a model is actually just a picture. For example – if a modeler says to himself, “clouds are too difficult” and then just ascribes, based on observed probabilities, that a cloud appears in the cell of a general circulation model, is something being modeled or is it being pictured?
I continue to see the models missing a more simple reason why they say we should be lots hotter but we aren’t. The tropospheric hot spot is hard to find, and based on greenhouse theory, it is not there like it should be. In fact, it should be easy to find. And it gets the gang of four’s (or three Stooges’ to give one of them a pass) knickers in a twist because it is not behaving as it should. Instead, one day its there, the next day it has disappeared from that spot. It (they?) moves around like an army of ghosts fading in and out in random fashion.
I think the problems with the models lay at the elementary level and is why we get for example freeze warnings, instead of an early warm spring, while CO2 is still increasing. Radiational cooling is one of those simple processes that I think the models get wrong. Another would be pressure differential driven winds moving things up, down, and sideways, or stagnating things over land and water.
The Earth is not encased in a firmament. It is surrounded by layers of filmy, ethereal gases that expand and contract, and are filled with holes that open and close in random fashion, which is exactly what the upper troposphere is doing, and why it is not heating up at a steadily rising pace. The warmth is escaping in bits and pieces here and there.
You can add and subtract aerosols, clouds, and greenhouse gases, including water vapor, and all it does is prompt the wind and ghostly layers to adjust their movement, random holes behavior and translucency. Sure, major additions of these things and we would see real climate change. But this stuff we are getting our knickers in a twist over is easily handled by I think, the simple things that bring about short and long term weather pattern variations.
DocMartyn says:
May 1, 2011 at 6:05 am
That is a fit, not a model.
All machine learning programs – which includes climate models – involve curve/surface fitting of some form during the training process. It may not be called curve fitting. It might be called neural nets for example, but it is a form of curve fitting.
Here is a very simple example:
If I have temperature data for 1850, 1851, 1852 … 2011; then if I fit the correct function (curve) to this data such that:
Function(1850) = Temperature(1850)
Function(1851) = Temperature(1851)
…
Function(2011) = Temperature(2011)
Then, if you want to know the temperature for 2100 for example, then I just plug the year into my function, and it will calculate the temperature in 2100.
Temperature(1850) = ?
Temperature(1850) = Function(2100)
How is this done in climate science?
Say you have forcings such as solar, CO2, aerosols, albedo, land-use, etc. How much does each of these contribute to the global temperature? The answer is that no one knows. There are guestimates at best.
So, to build a model of climate for temperature, climate science assumes everything is linear and uses a formula like this:
Temperature = (X% solar) + (Y% CO2) + (Z% albedo) + (etc) + (etc)
Then by plugging in “actual” numbers by year to temperature, solar, albedo, etc you solve for X,Y,Z to give you the best fit with historical data.
This then gives you a formula, and by plugging in your estimates for future solar, CO2, albedo, etc., in say 2100, the formula will calculate future temperature in 2100.
So, the key then becomes the values you choose for X,Y,Z. Even a small change can have a large effect going forward, because temperature is cumulative. If the earth heats up in one year, that heat is still there at the start of the next year.
So, this allows you to make small adjustments to X,Y,Z today, which can greatly change the forecast of your model in 2100. This allows you to forecast just about anything you want by selecting the correct X,Y,Z. By having not just 3 weights, but 150 as found in climate models, this gives you a whole lot of wiggle room to make adjustments.
Which is what happens in climate science. Those models with values of X,Y,Z that don’t give the answer that climate scientists expect are discarded. Those with the values of X,Y,Z that deliver the answer they expect are retained. Thus, the models are not forecasting climate, they are forecasting the expectations of the climate modelers.
Had climate models been done correctly, using double blind techniques used in animal training studies, they would have predicted that temperatures would level off around 2000, because they had done this before, in a 60 year cycle. However, the climate scientists discarded those models, because they didn’t expect temperatures to level off. They expected them to continue rising unchanged, which is reflected in their models.
The fault in climate science models is that do not recognize that any training study, be it animal training or machine training is subject to contamination by the observer’s own unconscious actions. Thus you must isolate the model from the expectations of the scientists involved in creating the model, which is extremely hard to do.
To be accurate, the model builder cannot see the results of the model before the model is finalized. Otherwise the model builder can and will unconsciously use this information to modify the model to meet his/her expectations, which contaminates the result. Equally, the model builder cannot choose to select one model over another, based on the result. this is cherry picking, which also contaminates the result.
Climate models are being used like the reading of tea leaves and climate scientists are acting like the Oracle of Delphi, removing science from the modern world to antiquity. .
Willis. Your 1 May 3:50 am comment on the “A Prediction Market For Climate Outcomes” thread over at Climate Etc deserves wider coverage here. Perhaps a separate post. Succinct and penetrating, as usual.
One of the main problems with models is they contain an “interpretation” of the physics, not physics itself. As such they are prone to human bias.
If they tried to get at the physics, the models would run for decades just to model a few seconds. That is, it can’t be done. So, what we end up with is researcher bias and nothing even remotely resembling a real physical model.
Keep in mind that those models that do not meet the expectations of the model builder are rejected by the model builder in a process similar to “selection of the fittest”.
For example: A climate model that predicts a doubling of CO2 will result in no change in temperature will be rejected by the model builder and “fixed”. A model that predicts a coubling of CO2 will result in a 10C increase in temperature will be rejected by the model builder and “fixed”.
How is a model “fixed”? Not by changing the physical laws within the model, but rather changing the weightings.
For example, how important is water vapor, or aerosols, or evaporation rates, or convection or land use. No one knows the true answer to these questions, so the models assign weights to these, and the weights are adjusted so that the models will hindcast AND deliver a future projection that is within the moldel builders range of expectations.
This last point is the critical issue because it give rise to the “experimenter-expectancy effect” the is well recognized in animal studies. We know from animal studies that you need to isolate any organism capable of learning from the experimenter through double blind controls.
What climate science has missed is that climate models are machine learning programs. These programs are subject to the “experimenter-expectancy effect” if proper controls are not used during the model training process.
This means double blind controls. That the model builder cannot have access to the output of the model during the training process. Only after the model is fully trained and no more adjustments will be made can the model builder see the results of the model.
This isolation is not done in climate science, which results in models that are not predicting future climate. Rather they are predicting to match the model builders expectations of what will happen. This generates a feedback loop between the model and the model builder which reinforces the ego of the model builder, to the point of an addiction-dependency. The model becomes more real to the model builder than reality.
BTW, does anyone know if a model has been built to model one cubic centimeter of air? It seems like one might be able to model the physical activity and see what happens as the content is modified to add more CO2. If a cubic cm is too much make it even smaller. At some point one should be able to model every molecule and its interaction with others.
Seems like good PHD thesis topic.
The climate models become unstable as you increase the resolution. Unlike reality where the accuracy of the answer improves when you increase the resolution.
Ferd “To be accurate, the model builder cannot see the results of the model before the model is finalized.”
When one makes a model, as opposed to a fit, one uses data sets to independently arrive at the range for a constant. You examine data sets where only one variable is changed, or a close approximation to it.
The example you state:
Temperature = (X% solar) + (Y% CO2) + (Z% albedo) + (etc) + (etc)
is not a model, as it is not made up of components that are individually testable. It is just a mathematical fitting function where the inputs have been given more complex names than is normal.
in models, there are constraints. Want to measure the effects of areoslos? Go to the desert and at 11 O’Clock fly planes one of which is dumping SO2 and one one isn’t. Measure the difference in temp, spectrum and SO2 density with respect to time.
This is just science.
Dagfinn says: April 30, 2011 at 11:12 pm
and Olen says: May 1, 2011 at 7:54 am
I had been thinking Pygmalion (Rousseau’ Galatea) after reading on Hans the Horse.
The Earthly Paradise: Pygmalion and the image, William Morris (1868)
http://www.victorianweb.org/authors/morris/poems/pygmalion.html
Thank you Pamela and ferd (x2) for your informative postings.