Guest post by Dr. David Stockwell
In Australia, the carbon-tax juggernaut rolls on, justified in part by fear of droughts, increasing in frequency and severity as CO2 increases.
I have always found that checking one’s assumptions was good advice, and with that in mind I checked the models used in a major drought study by the CSIRO and the Australian BoM. The study, the Drought Exceptional Circumstances Report (DECR), was widely used to support the contention that major increases in drought frequency and severity in Australia will result from further increases in CO2.
The results were recently published in the peer-reviewed journal Energy and Environment (PDF).
My paper contributes in the areas of validation of climate models (the subject of a recent post at Climate, Etc.), and regional model disagreement with rainfall observations (see post by Willis Eschenbach).
Specifically, droughts have decreased last century in line with increasing rainfall, but the climate models used in the DECR showed the opposite (and significantly so).
Overall, it is a case study demonstrating the need for more rigorous and explicit validation of climate models if they are to advise government policy.
It is reasonably well known that general circulation models are virtually worthless at projecting changes in regional rainfall, the IPCC says so, and the Australian Academy of Science agrees. The most basic statistical tests in the paper demonstrate this: the simulated drought trends are statistically inconsistent with the trend of the observations, a simple mean value shows more skill that any of the models, and drought frequency has dropped below the 95% CL of the simulations (see Figure).
The larger issue is how to get people to accept that there will always be worthless models, and the task of genuinely committed modellers to identify and eliminate worthless models. It’s not convincing to argue that validation is too hard for climate models, they are the only ones we have got, they are justified by physical realism, or they are ‘close enough’.
My study shows that the obvious testing regimes would have shown the drought models in the DECR study were unfit for use, if they had been tested. I asked CSIRO, but no validation results were supplied.
The concerns of scientists are different to decision-makers. While scientists are mainly interested with the relative skill of models in order to gauge improvements, decision-makers are (or should be) concerned primarily with whether the models should be used at all (are fit-for-use). Because of this, model-testing regimes for decision-makers must have the potential to completely reject some or all of the models if they do not rise above a predetermined standard, or benchmark.
There are a number of ways that benchmarking can be set up, which engineers or others in critical disciplines would be familiar with, usually involving a degree of independent inspection, documentation of expected standards, and so on. My favorite benchmark test is quick and easy: the Nash-Sutcliffe Efficiency, an indicator of whether a model shows more skill than a simple mean value.
I believe that decision-makers should not take results seriously unless rigorous validation of the models is also demonstrated.
It is up to the customers of climate studies to not rely on the authority of the IPCC, the CSIRO and the BoM, and to demand “Show us your tests”, as would be expected with any economic, medical or engineering study where the costs of making the wrong decision are high. Duty of care requires confidence that all reasonable means have been taken to validate all of the models and assumptions that support the key conclusions.
===============================================================
About the author (from his website here)
After receiving a Ph.D. in Ecosystem Dynamics from the Australian National University in 1992, I worked as a consultant (WHO, Parks and Wildlife, Land and Natural Resources services) until moving to the San Diego Supercomputer Center at University of California San Diego in 1997. There I helped to develop computational and data intensive infrastructure for ecological niche modeling mainly using museum collections data with grants from the NSF, USGS and DOT. I developed the GARP (Genetic Algorithm for Rule-set Production) system making contributions in many fields: modeling of invasive species, epidemiology of human diseases, the discovery of seven new species of chameleon in Madagascar, and effects on species of climate change. I have published in major journals and was judged by the US Immigration Service as an Outstanding Researcher, recognized internationally as outstanding in their academic field.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
I seems funny that some years ago CSIRO stated that models were to be viewed with caution because they relied too much on assumptions that were not always correct. The same applies today especially with climate models.
thanks!
we KNOW the Bom and CSIRO LIE!
we need info like this to enable us to fight back.
problem is, ignorant pollies who wont read anything but what their agenda( for revenue raising ie JuLiars Carbon Tax ) allows ie Ipcc drivel.
I have not yet heard anyone praise the computer models and their predictions, projections, forecasts or whatever one wants to to call their “garbage out”! When companies put out information to the public, it must be audited or be subjected to some form of official assurance review to minimize the risk of directors engaging in misleading and deceptive conduct. I cannot understand why similar standards cannot apply to model-based predictions… particularly since the IPCC so heavily relies on them and governments rely on the IPCC in formulating their climate change policies.
Based on the report ‘Carbon Dioxide and Earth’s Future: Pursuing the Prudent Path’ by Craig Idso and Sherwood Idso (see link below), it would seem that model-based predictions are seriously flawed and unreliable, which means the IPCC is wrong, which means governments forming policies based on the IPCC’s report must be flawed.
http://www.co2science.org/education/reports/prudentpath/prudentpath.php
Well well well…
“It is up to the customers of climate studies to not rely on the authority of the IPCC, the CSIRO and the BoM, and to demand “Show us your tests”, as would be expected with any economic, medical or engineering study where the costs of making the wrong decision are high. ”
In the case of climate studies, they should say “show independent scientists you raw data, our data manipulation methods and your tests”. I wouldn’t trust what hockey-stick manufacturers showed me.
Does their rainfall model include the Indian Ocean Dipole?
http://www.science.unsw.edu.au/news/indian-ocean-drought/
By the way, has anyone yet determined drivers for the IOD?
Fine essay as always Mr Stockwell! We have a carbon Tax here in my neck of the woods, and it might be indicative of what you will soon be experiencing. The tax has merely ended up paying for re-election campaigns or funnelled to radical advocacy groups. It has had zero effect on consumption and even less effect on emissions.
I’ll try and dig up some data for you.
Cheers!
I think that the idea of validating models, while novel, is an idea that should be considered in the future.
Can we have a bit more explanation of “A” please?
To me, it looks like exponential growth in area experiencing exceptional temperatures prior to 2000, so presumably reached 100% a few years back. Is this what it is supposed to mean?
As always it is not about the science it is about the agenda. Government is in control and wants to assert more control (read taxes), or dreams on an ideal world (read sycophant) .
In the UK attenuated storm water storage systems and storm water drainage systems are required to be designed for a 1 in 100 year storm plus 30% for climate change?
This results in the additional use of materials, transport and labour etc, which of course is very sustainable indeed? The inmates are defintaly running the lunatic asylum.
A bit of rain in Victoria of late. It will be interesting to see how they explain that. Personally, it is standard for a La Nina event.
Dr. Stockwell,
I commend your efforts at looking at bad science practices.
It is far easier to ignore current readings and cherry pick or manipulate whatever for the sake of more funding at the expense of good science.
Can you imagine the crap science would have us believe if people did not check their reports or studies?
Being just an interested observer and too lazy to go look up answers to my own questions, please forgive me for asking: Is A a striking example of another kind of hockey stick? The red line is observation of areas with exceptionally high temperatures rising drastically and off the graph starting in 1970 through 2000? I don’t understand and the post doesn’t seem to offer an explanation.
E&E is not a well regarded journal. Their peer review process has been called into question. I cannot comment on your work. Maybe it is valid. But you might have had more impact outside the blog-sphere had you submitted to a higher quality journal.
Sorry O/T, but I can’t post in the Tips & Notes for some reason. Please delete this if you wish.
The IPad problem is due to the onswipe theme that wordpress have introduced. It has made WUWT un-usable on my IPad. I believe that you can disable it, and I would ask that you consider doing this. I also notice it has infected Climate Audit and Climate Etc.
Brian Andrews
Basic science:
For a scientific theory to be valid it must:
– Explain available data
– Make predictions
– Be falsifiable
While some aspects of “Climate Theory” conform, there are too many which do not. Models are predictions, if the models do not match the observations then the models are wrong and the theory itself needs to be modified.
I just did a very quick read of your paper. Look’s like you made the same mistake Eschenbach acknowledged making: drought is not just a function of rainfall but of soil moisture. The did well on temperature. Higher temps generally lead to quicker evaporation. The report you are criticizing says:
“Projected increases in the areal extent and frequency of exceptionally low soil moisture years are slightly clearer than those for rainfall. If soil moisture were the sole criterion for EC declarations, then the mean projections indicate that more
declarations would be likely by 2030, particularly in the SW, SW WA and Vic&Tas regions. Under the high scenario, EC declarations would be triggered almost twice as often in most regions and almost four times as often in SWWA.”
http://www.bom.gov.au/climate/droughtec/
But again I have not read your work carefully as I need to get ready for work now. I hope your efforts get the consideration they are due.
I emphatically agree. There are so very many ways to get a model wrong. The only way to tell what it is worth for decision-makers is to show that it works. It has to work when things are going up, and it also has to work when things are going down. A badly broken model can still look OK as long as the wind is blowing in one direction, the preferred direction. But when the wind changes, we find out how well it really works.
As long as la Niña continues they should worry about floodings instead….then politicians are the only ones to experience drought…of votes.
I tried to post this in Tips and Notes, but it appears to be bunged up. Stats Canada, a government agency here in Canada known for political bias has announced that the climate models are correct.
http://cnews.canoe.ca/CNEWS/Environment/2011/03/23/17728111.html
Pretty bold language! I smell a big fat civil-serpent rat!
Cheers!
Dr David Stockwell shows good sense in sugesting that consumers of the results of climate models should know that those models actually work; what alarms me is that most politicians who are insisting that ‘the climate models show us…’ would not have any idea if the models have any p[redictive value or not, but want to plunge the developed world into a huge retrograde step on faith alone.
I’m enough of a peasant to insist that the slaesman shows me his product works.
The Hockey Stick lives! 😛
Okay, in a more serious vein, a bit more discussion of the provided graphs would be nice. Is the handful of years when the drought frequency dropped below the 95% CL of the simulations that significant? Though I suppose assuming an constant 5% in area with exceptionally low rainfall would work just as well.
Interesting. I’d read that many climate models predict an increase of total precipitation, but that precipitation will also become less evenly distributed over time. That is to say that average precipitation increases, but you also get more periods of drought interspersed with large downpours. People concerned about crop yields in a warming world often point to this prediction as worrisome, since crops are easily decimated under such conditions.
My questions:
Does the model in question have anything to say about the distribution of precipitation over time?
This model predicts a decrease in the average precipitation over time, which appears to be at odds with other climate models. Why is it different?
What does the graph mean and how does the graph relate to the text?
The presentation is not directly referencing the graphs.
At first glance, the graph suggests an exponentially increasing percentage of Australia is experiencing extreme heat and that this is as indicated by the various models.
It also indicates that areas of extreme drought are declining, contrary to the modeled trend.
On balance, if these two graphs are representative, the models are doing a fair job, imo. So what is the issue?
Yes, models may be lousy, but surely sacrificing the economy will lower global temperature. Any politician knows that much.