It has been an interesting couple of days. Today yet another scientist has come forward with a press release saying that not only did their audit of IPCC forecasting procedures and found that they “violated 72 scientific principles of forecasting”, but that “The models were not intended as forecasting models and they have not been validated for that purpose.” This organization should know, they certify forecasters for many disciplines and in conjunction with John Hopkins University if Washington, DC, offer a Certificate of Forecasting Practice. The story below originally appeared in the blog of Australian Dr. Jennifer Marohasy. It is reprinted below, with with some pictures and links added for WUWT readers. – Anthony
J. Scott Armstrong, founder of the International Journal of Forecasting
Guest post by Jennifer Marohasy
YESTERDAY, a former chief at NASA, Dr John S. Theon, slammed the computer models used to determine future climate claiming they are not scientific in part because the modellers have “resisted making their work transparent so that it can be replicated independently by other scientists”. [1]
Today, a founder of the International Journal of Forecasting, Journal of Forecasting, International Institute of Forecasters, and International Symposium on Forecasting, and the author of Long-range Forecasting (1978, 1985), the Principles of Forecasting Handbook, and over 70 papers on forecasting, Dr J. Scott Armstrong, tabled a statement declaring that the forecasting process used by the Intergovernmental Panel on Climate Change (IPCC) lacks a scientific basis. [2]
What these two authorities, Drs Theon and Armstrong, are independently and explicitly stating is that the computer models underpinning the work of many scientific institutions concerned with global warming, including Australia’s CSIRO, are fundamentally flawed.
In today’s statement, made with economist Kesten Green, Dr Armstrong provides the following eight reasons as to why the current IPCC computer models lack a scientific basis:
1. No scientific forecasts of the changes in the Earth’s climate.
Currently, the only forecasts are those based on the opinions of some scientists. Computer modeling was used to create scenarios (i.e., stories) to represent the scientists’ opinions about what might happen. The models were not intended as forecasting models (Trenberth 2007) and they have not been validated for that purpose. Since the publication of our paper, no one has provided evidence to refute our claim that there are no scientific forecasts to support global warming.
We conducted an audit of the procedures described in the IPCC report and found that they clearly violated 72 scientific principles of forecasting (Green and Armstrong 2008). (No justification was provided for any of these violations.) For important forecasts, we can see no reason why any principle should be violated. We draw analogies to flying an aircraft or building a bridge or performing heart surgery—given the potential cost of errors, it is not permissible to violate principles.
2. Improper peer review process.
To our knowledge, papers claiming to forecast global warming have not been subject to peer review by experts in scientific forecasting.
3. Complexity and uncertainty of climate render expert opinions invalid for forecasting.
Expert opinions are an inappropriate forecasting method in situations that involve high complexity and high uncertainty. This conclusion is based on over eight decades of research. Armstrong (1978) provided a review of the evidence and this was supported by Tetlock’s (2005) study that involved 82,361 forecasts by 284 experts over two decades.
Long-term climate changes are highly complex due to the many factors that affect climate and to their interactions. Uncertainty about long-term climate changes is high due to a lack of good knowledge about such things as:
a) causes of climate change,
b) direction, lag time, and effect size of causal factors related to climate change,
c) effects of changing temperatures, and
d) costs and benefits of alternative actions to deal with climate changes (e.g., CO2 markets).
Given these conditions, expert opinions are not appropriate for long-term climate predictions.
4. Forecasts are needed for the effects of climate change.
Even if it were possible to forecast climate changes, it would still be necessary to forecast the effects of climate changes. In other words, in what ways might the effects be beneficial or harmful? Here again, we have been unable to find any scientific forecasts—as opposed to speculation—despite our appeals for such studies.
We addressed this issue with respect to studies involving the possible classification of polar bears as threatened or endangered (Armstrong, Green, and Soon 2008). In our audits of two key papers to support the polar bear listing, 41 principles were clearly violated by the authors of one paper and 61 by the authors of the other. It is not proper from a scientific or from a practical viewpoint to violate any principles. Again, there was no sign that the forecasters realized that they were making mistakes.
5. Forecasts are needed of the costs and benefits of alternative actions that might be taken to combat climate change.
Assuming that climate change could be accurately forecast, it would be necessary to forecast the costs and benefits of actions taken to reduce harmful effects, and to compare the net benefit with other feasible policies including taking no action. Here again we have been unable to find any scientific forecasts despite our appeals for such studies.
6. To justify using a climate forecasting model, one would need to test it against a relevant naïve model.
We used the Forecasting Method Selection Tree to help determine which method is most appropriate for forecasting long-term climate change. A copy of the Tree is attached as Appendix 1. It is drawn from comparative empirical studies from all areas of forecasting. It suggests that extrapolation is appropriate, and we chose a naïve (no change) model as an appropriate benchmark. A forecasting model should not be used unless it can be shown to provide forecasts that are more accurate than those from this naïve model, as it would otherwise increase error. In Green, Armstrong and Soon (2008), we show that the mean absolute error of 108 naïve forecasts for 50 years in the future was 0.24°C.
7. The climate system is stable.
To assess stability, we examined the errors from naïve forecasts for up to 100 years into the future. Using the U.K. Met Office Hadley Centre’s data, we started with 1850 and used that year’s average temperature as our forecast for the next 100 years. We then calculated the errors for each forecast horizon from 1 to 100. We repeated the process using the average temperature in 1851 as our naïve forecast for the next 100 years, and so on. This “successive updating” continued until year 2006, when we forecasted a single year ahead. This provided 157 one-year-ahead forecasts, 156 two-year-ahead and so on to 58 100-year-ahead forecasts.
We then examined how many forecasts were further than 0.5°C from the observed value. Fewer than 13% of forecasts of up to 65-years-ahead had absolute errors larger than 0.5°C. For longer horizons, fewer than 33% had absolute errors larger than 0.5°C. Given the remarkable stability of global mean temperature, it is unlikely that there would be any practical benefits from a forecasting method that provided more accurate forecasts.
8. Be conservative and avoid the precautionary principle.
One of the primary scientific principles in forecasting is to be conservative in the darkness of uncertainty. This principle also argues for the use of the naive no-change extrapolation. Some have argued for the precautionary principle as a way to be conservative. It is a political, not a scientific principle. As we explain in our essay in Appendix 2, it is actually an anti-scientific principle in that it attempts to make decisions without using rational analyses. Instead, cost/benefit analyses are appropriate given the available evidence which suggests that temperature is just as likely to go up as down. However, these analyses should be supported by scientific forecasts.
The reach of these models is extraordinary, for example, the CSIRO models are currently being used in Australia to determine water allocations for farmers and to justify the need for an Emissions Trading Scheme (ETS) – the most far-reaching of possible economic interventions. Yet, according to Dr Armstrong, these same models violate 72 scientific principles.
********************
1. Marc Morano, James Hansen’s Former NASA Supervisor Declares Himself a Skeptic, January 27,2009. http://epw.senate.gov/public/index.cfm?FuseAction=Minority.Blogs&ContentRecord_id=1a5e6e32-802a-23ad-40ed-ecd53cd3d320
2. “Analysis of the U.S. Environmental Protection Agency’s Advanced Notice of Proposed Rulemaking for Greenhouse Gases”, Drs. J. Scott Armstrong and Kesten C. Green a statement prepared for US Senator Inhofe for an analysis of the US EPA’s proposed policies for greenhouse gases. http://theclimatebet.com
Sponsored IT training links:
Get guaranteed success in 312-50 exam in first try using incredible 642-374 dumps and other 310-200 training resources prepared by experts.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
old construction worker (00:33:14)
Not really. In general the “equilibrium temperature” to which the earth is continually progressing towards is a short term “equilibrium” which is continually being “frustrated”. For example the Earth is warming towards a new “equilibrium temperature” under the influence of the present forcings (385 ppm of CO2 plus the effects of the other greenhouse gases, plus the solar contribution plus the aerosol contribution, and so on.) However the forcings are changing on various timescales and so the position of the potential “equilibrium temperature” is also changing.
So by “equilibrium temperature” I don’t mean to imply that the Earth has some sort of “proper” temperature. During ice age cycles, one might say that on the multi-millenial timescale there are two generalized “equilibrium temperatures”, one corresponding to a greenhouse gas concentration near 180 ppm, a lowish insolation pattern, and high albedo (glacial period), and the other corresponding to a greenhouse gas concentration near 270 ppm in CO2, more favourable insolation, lower albedo (snug interglacials)….
foinavon (03:42:37) :
‘So by “equilibrium temperature” I don’t mean to imply that the Earth has some sort of “proper” temperature. During ice age cycles, one might say that on the multi-millenial timescale there are two generalized “equilibrium temperatures”, one corresponding to a greenhouse gas concentration near 180 ppm, a lowish insolation pattern, and high albedo (glacial period), and the other corresponding to a greenhouse gas concentration near 270 ppm in CO2, more favourable insolation, lower albedo (snug interglacials)….’
What was the “equilibrium temperature” at 180 ppm?
What was the “equilibrium temperature” at 270 ppm?
foinavon (03:42:37) : Not really. In general the “equilibrium temperature” to which the earth is continually progressing towards is a short term “equilibrium” which is continually being “frustrated”. […]
During ice age cycles, one might say that on the multi-millenial timescale there are two generalized “equilibrium temperatures”,
No doubt about it, a very new and wondrous definition of equilibrium has been presented to the world… Not only is it now a dynamic process that can wander about and has emotions to be ‘frustrated’ but, so it seems, can even be of two minds. I guess the idea of being ‘balanced’ has now merged with that of hysteresis in this brave new world of “Climate Investigative Journalism” practiced in the name of AGW…
I think the words you are looking for are ‘oscillation’, ‘hysteresis’, and ‘unstable disequilibrium’…
@Gary Young gulrud (07:47:01) :
Thanks!
One thing that has bothered me for some time is the statement that unless you are a climate scientist who has written a peer-reviewed article for some science publication you are just not credible. Well, the fact remains that in a Republic the people actually do have something to say about public policy. And, since we are all going to be affected by climate policy for a long time to come, and since public policies on scientific themes have a way of being passed hastily and then not repealed even when the science is proven wrong it behooves all of us citizens to become informed and to take a stand. The global warmers are trying very hard to limit the discussion to those who accept their view. This is a serious mistake. I hope to write something like a global warming for dummies series (or similar) for those who really don’t want to do a lot of reading. If you have ideas I would love to hear them. jimhollingsworth@verizon.net
One thing that has bothered me for a very long time is the statement that unless you are a climate scientist who has written peer-reviewed papers for some science publication you are not credible. Well, the fact remains that in a Republic the people actually do have something to say about public policy. And, since we are all going to be affected by climate policy for a long time to come, and since public policies on scientific themes have a way of being passed hastily and then not repealed even when the science is proven wrong it behooves all of us citizens to become informed and to take a stand. The global warmers are trying very hard to limit the discussion to those who accept their view. This is a serious mistake. I hope to write something like a global warming for dummies serise (or similar) for those who really don’t want to do a lot of reading. If you have ideas I would love to hear them. jimhollingsworth@verizon.net
Jim Hollingsworth (09:10:21) :
One thing that has bothered me for a very long time is the statement that unless you are a climate scientist who has written peer-reviewed papers for some science publication you are not credible.
Yeah, that one cracks me up too. On one occasion I took a class in “Biomedical Applications of Computers”. It was taught by a professor from the Medical School. An MD? Nope. PhD in Engineering. But he knew more about his specialty in medicine than all the “MDs to be” studying in his classes… and was better at it than the MD professors on staff. By the Warmers logic, he was ‘not qualified’; luckily the Med School thought otherwise…
I hope to write something like a global warming for dummies serise (or similar) for those who really don’t want to do a lot of reading. If you have ideas I would love to hear them.
I think Lucy Skywalker has made a start on something like that. You might want to coordinate with her.
On another thread, someone asked what were ‘the basics’. Here’s my list, reproduced because I don’t know what thread it was on…
I would summarise ‘the basics’ as:
1) The raw data is defective. There are many pages here about various thermometer errors (placement, urban heat island, changed paint type leading to higher readings, etc.) that bias the data to the high side.
2) The raw data is missing. There are several variations here, too. Stations come and go. Sometimes large numbers (when the USSR collapsed a very large percentage of total thermometers just went away, many in Siberia. At the same time the recorded temperature average went up…)
3) The raw data is from too short a period. We are looking at a system with at least 1500 year cycles in it (Google “Bond Event wiki” for details). To do that and not be fooled by a cyclical slope needs about 3000 years data. For satellite data we have 30 years or so. For land based thermometers, a couple of hundred. That couple of hundred just happens to start at the bottom of a cold period known as the Little Ice Age and is rising due to a normal cycle. It was warmer in the past (several times in history, more times demonstrable by archaeology like the ice man from under a glacier…) and No Bad Thing happened.
4) Sometimes the raw data is just made up. GISS, it seems, can fill in the arctic temperatures with guesses via computer. ANY US land station can have missing readings ‘estimated’ by the person filling in the form if they want. (Missed a day? Just make it up…).
5) Once the data are collected, they are subject to strange and wondrous changes and manipulations. The exact methods are more or less secret. The changes are conducted by people who often have their entire self worth and career vested in ‘global warming’. The results often seem disjoint from observed reality. (I have a particular gripe with the GISS method that involves adjusting past temperatures down based on present temperatures. I’d rather that my history didn’t keep changing under my feet, but I’m old fashioned that way.) Where there are details on the adjustment available, they can often be shown to be bogus. (Removal of urban heat island effect by reference to ‘nearby’ ‘rural’ thermometers that are in fact hundreds of miles away in different microclimates and sometimes in large urban area.)
6) Based on this flakey data, folks build castles in the sky. They do this with computer models. (I’m a ‘computer guy’ by trade and managed a Cray supercomputer site that did modeling for plastic flow so this one galls me.) The models are ‘not very good’ to put it charitably. The don’t match reality. Their predictions are regularly shown to be bogus. When you do get a little look at how they work, it is not convincing. They leave out major, perhaps even dominant, features of climate. (Cloud formation of all sorts, cosmic rays that lead to cloud formation, variation in the sun, many most or all of the various ocean oscillations and heat transfer anomalies ENSO, AMO, etc.) Oh, and we have a specific admission by at least one of the modelers that they deliberately made the model run fast for more dramatic effect. That 50 year doom? Even their model would say it’s 150 years away if not run on ‘juice’. We have public quotes from ‘scientists’ in the field saying they need to punch up the results to create stronger public responses…
7. Many of the assumptions and science in the models are based on errors of assumption. I can only list a couple of examples here (too many…). It is assumed that CO2 causes warming. All the archeological data show CO2 follows heating by 800 years. How does cause follow effect? All sorts of positive feedbacks are assumed, but negative ones are ignored (cloud cooling anyone?)
8. They simply can not model what they do not know. ANY computer model can only tell you things in the domain of your present understanding. If your understanding is broken, so is your model. They “know” that CO2 is causal (despite the data) and that is what they model, ergo what they find. The truth is that we really don’t know how weather and climate work completely, so any ‘model’ can at best be used to show places to do more research, not to make policy. They don’t predict, they inform of our ignorance.
9) The thing they are trying to model, 30 year weather, is chaotic. (That does not mean random, it means that the state jumps all over from trivial input changes.) Chaotic models are, at the present state of the art, worse than guessing (and may always be, the math behind it leads me to think maybe so…) The input data are very flawed.
10) Based on these models saying the world will end Real Soon Now, many other folks run off to show that they ought to get funding for their grant because it is related to this hot topic of global warming. When you look into the ‘thousands and thousands’ of papers endorsing the global warming thesis you find the vast majority are of the form “If we assume that the model run by [foo] is right, this is the bad thing that will happen in MY field.” There are in fact only a few centers doing the modeling (a half dozen?) and their ideas are very inbred. We are really basing world decisions on the work of about a half dozen.
11) Dissent is to be crushed, ruthlessly. Frankly, this is what got me started down the “What the…” trail. I’ve worked in forensics and law enforcement from time to time. Sets off my Madoff Alarm. (Used to be Ponzi…) If you’re so sure you are right, demonstrate (share) your data, models, et. al. and we’ll have a nice debate. No? OK, WHAT ARE YOU HIDING? One of the hallmarks of a shared delusion is the ruthless attack of anything that would threaten the delusion. It just smells of cult. And there are plenty of alternative theories, including the established one of ‘it is natural variation’. The science is not settled and the debate is not over, even if one side is paranoid about being challenged.
12) The major drivers of the process are not scientists, but political bodies with agendas for control and a history of corruption and deception. UN? You want me to trust a UN Political Committee? The IPCC is NOT a bunch of scientists, it’s a bunch of politicians. They consult scientists. They have at times re-written scientists work (without notice). Many scientists have now begun speaking out against the IPCC. See #11 for how they are treated.
13) Mr. Albert Gore. His ‘inconvenient truth’ is a nice propaganda piece. It is decidedly not science. Polar bears are aquatic, they swim hundreds of miles sometimes (one swam from Greenland to Iceland). He shows them drowning… Their numbers are rising, he shows them near extinction. The list goes on. When a politician starts blatantly propagandizing for central power and authority my ‘peace in our time’ buzzer goes off…
14) The ‘cure’. The proposed cure will result in terrible death and poverty. It will misallocate trillions of dollars (that would be much better spent improving other things: education world wide, malaria, cooking stoves in the 3rd world, food supplies, etc.) Mr. Gore and others stand to profit greatly from it (he has a ‘carbon credit’ company from which he buys his own indulgences…) Further, since China and India get a free pass, the only real result is to move most industry there and kill the western democracies. (Hmmm socialist western-hating UN proposing ‘solutions’ that hobble western democracies…) The rate of ‘ramp up’ in coal consumption in China assures that no ‘control’ of CO2 is possible. Why are we ‘curing’ what is not broken with a solution that will not work?
15) The whole ‘tipping point’ thesis is simply and demonstrably false. For most of the history of the planet, CO2 has been much much higher. 10 times or more. We are actually at historic low levels. (Plants respond to CO2 as they do to any nutrient that is lower than their ideal value, up to about a 1000 ppm value. This implies they evolved expecting that much, and that is what the geological record shows.) Why are we trying to reduce CO2 to levels that restrict plant growth? Why are we trying to make the planet colder when that reduces food production? The potential harm here is stupendous. Why have we never ‘tipped’ before?
16) We may be doing exactly the wrong thing at exactly the wrong time. Google “pessimum’. Periodically these cold periods come along in our history. They result in the destruction of social order, starvation, disease, mass migrations. We are about to push in that direction. Why? Google ‘climate optimum’ and you find the Medieval Optimum, the Roman Optimum, etc. We are now in the Modern Optimum. Warm is good, cold is bad. Yet there is more. I can only briefly state that there are reasons to believe that the present optimum may be peaking (or maybe even ending). From the planetary theories of solar output modulation, to the simple calendar correlations, to observed physical oscillations like the PDO flip to a colder direction in the short run; some theory points to cooler. While there is not enough to show causality, there is enough to urge caution in pushing that particular direction really really hard right now.
I’m going to stop now, or this will not be ‘the basics’…
There is more, but you get the idea. It starts to be a bit more technical (Like why does a global average of all those temperatures mean anything? – it doesn’t; and that the temperatures gathered don’t contain enough information for sampling theory and control theory to allow anyone to know what to do even if warming were true and if we could do anything: we have a ‘hot shower’ with 30 years between turning the knobs and changed water temp out. The knobs are not labeled and non-linear. There are several toilets being flushed and dishwashers running. Keep the temperature at exactly the right temperature; and your thermometer is broken.)
I’m sure other folks will have their own ideas as to what are ‘the basics’ but I hope I’ve also shown that even 1/2 of ‘my basics’ are enough to say that we ought not be doing what we, as a country, are about to do…
Hmm. A professional forecaster is saying that forecasting is useless.
So why is he taking money for doing it?
Agap, he says forecasting CLIMATE is useless… Reading is fundamental.
I see that the International Climate Science Coalition is the current name for the club where Dr. Armstrong hangs out with Fred Singer, Tom Harris and Tim Ball. It’s a free-floating contrarian cocktail party!
Just what are those “clearly violated 72 scientific (sic) principles of forecasting”?
Climatology is a messy science that attempts to reconcile experimental and theoretical knowledge with an incomplete historical record. That doesn’t mean that facile nit-pickers should treat it as a rhetorical playground.