Guest post by Steven Goddard
In his recent article, NSIDC’s Dr. Meier answered Question #9 “Are the models capable of projecting climate changes for 100 years?” with a coin flipping example.
1. You are given the opportunity to bet on a coin flip. Heads you win a million dollars. Tails you die. You are assured that it is a completely fair and unbiased coin. Would you take the bet? I certainly wouldn’t, as much as it’d be nice to have a million dollars.2. You are given the opportunity to bet on 10000 coin flips. If heads comes up between 4000 and 6000 times, you win a million dollars. If heads comes up less than 4000 or more than 6000 times, you die. Again, you are assured that the coin is completely fair and unbiased. Would you take this bet? I think I would.
Dr. Meier is correct that his coin flip bet is safe. I ran 100,000 iterations of 10,000 simulated random coin flips, which created the frequency distribution seen below.
The chances of getting less than 4,000 or greater than 6,000 heads are essentially zero. However, this is not an appropriate analogy for GCMs. The coin flip analogy assumes that each iteration is independent of all others, which is not the case with climate.
[Note: Originally I used Microsoft’s random number generator, which isn’t the best, as you can see below. The above plot which I added within an hour after the first post was made uses the gnu rand() function which generates a much better looking Gaussian.]
Climate feedback is at the core of Hansen’s catastrophic global warming argument. Climate feedback is based on the idea that today’s weather is affected by yesterday’s weather, and this year’s climate is dependent on last year. For example, climate models (incorrectly) forecast that Arctic ice would decrease between 2007 and 2010. This would have caused a loss of albedo and led to more absorption of incoming short wave radiation – a critical calculation. Thus climate model runs in 2007 also incorrectly forecast the radiative energy balance in 2010. And that error cascaded into future year calculations. Same argument can be made for cloud cover, snow cover, ocean temperatures, etc. Each year and each day affects the next. If 2010 calculations are wrong, then 2011 and 2100 calculations will also be incorrect.
Because of feedback, climate models are necessarily iterative. NCAR needs a $500 million supercomputer to do very long iterative runs decades into the future. It isn’t reasonable to claim both independence (randomness) and dependence (feedback.) Climate model errors compound through successive iterations, rather than correct. How could they correct?
Speaking of Arctic ice cover and albedo, the sun is starting to get high in the sky in the Arctic, and ice extent is essentially unchanged from 30 years ago. How does this affect climate calculations?
GCMs are similar to weather models, with added parameters for factors which may change over time – like atmospheric composition, changes in sea surface temperatures, changes in ice cover, etc. We know that weather models are very accurate for about three days, and then quickly break down due to chaos. There is little reason to believe that climate models will do any better through successive iterations. The claim is that the errors average out over time and produce a regionally correct forecast, even if incorrect for a specific location.
A good example of how inaccurate climate forecasts are, is shown in the two images below. NOAA’s Climate Prediction Center issued a long range forecast for the past winter in February, 2009. Brown and orange represents above normal temperatures, and as you can see they got most of the US backwards.
NOAA CPC’s long range forecast for winter 2009-2010
NOAA’s reported results for winter 2009-2010
The UK Met Office seasonal forecasts have also been notoriously poor, culminating in their forecast of a warm winter in 2009-2010.
The Met Office climate models forecast declining Antarctic sea ice, which is the opposite of what has been observed.
NSIDC’s observed increase in Antarctic sea ice
Conclusion : I don’t see much theoretical or empirical evidence that climate models produce meaningful information about the climate in 100 years.
However, Willis claims that such a projection is not possible because climate must be more complex than weather. How can a more complex situation be modeled more easily and accurately than a simpler situation? Let me answer that with a couple more questions:1. You are given the opportunity to bet on a coin flip. Heads you win a million dollars. Tails you die. You are assured that it is a completely fair and unbiased coin. Would you take the bet? I certainly wouldn’t, as much as it’d be nice to have a million dollars.2. You are given the opportunity to bet on 10000 coin flips. If heads comes up between 4000 and 6000 times, you win a million dollars. If heads comes up less than 4000 or more than 6000 times, you die. Again, you are assured that the coin is completely fair and unbiased. Would you take this bet? I think I would.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Wren,
“The “do nothings” on CAGW have an implicit forecast of no change in global temperature or a forecast of no change that man could do anything about, depending on which “do-nothing” you ask.”
Completely wrong, and a misunderstanding of the sceptical position. Lindzen summarized it best. When asked about his prediction for average temperatures in 2100, he said they could be warmer than today, the same as today or cooler than today.
Except in your imagination, no sceptic has ever predicted no change.
” Warmer still will be the 2010-2019 decade.”
Can you please share with me the suppier of your crystal ball? Mine seems to have stopped working.
Sometimes I’ve used the decimal expansion of pi for non-crypto purposes when a good random number source is needed. You just loop over each digit in the expansion and do something like: if (d < 5) then heads, else tails. Or it might be more intuitive and efficient to use the binary expansion of pi. There are lots of fast algorithms available for generating millions of digits (or bits) of pi and it can be a fun programming project! You could just easily use the decimal expansions of e to accomplish the same thing. This is a personal preference.
A possible problem with using a binomial distribution to prove a point about climate models is that you are assuming a Frequentist statistical methodology is applicable when the model projections are probably using a Bayesian approach. Tamino discussed the differences between the Frequentist and Bayesian approaches to coin flipping in his blog a while back:
http://tamino.wordpress.com/2008/07/10/reverend-bayes/
Re: Bob Koss (Apr 10 20:36),
From experience: yes they would, and they have.
How does this happen?
– Neither vendors nor users are careful to test those defective functions.
– A careful test requires understanding and skill
– Those who receive defect reports rarely have that skill
– Bad policy caused my comprehensive report to be ignored
Why was my report ignored? Because “the lab” tried to independently produce the problem, without looking at the details. They couldn’t reproduce, so they ignored my report.
In my case, there was a flaw, for six years, in a fundamental C library math function. I had proof, I even created a binary patch that fixed the problem. (It happened to kill our commercial mapping software for customers who used certain CPU’s.)
We could not get MS to accept our bug report for six years. So we patched their library.
A little humility never hurt anyone, especially vendors.
Wren (12:41:26) :
Of course Meir is right about the coin flips. It’s amazing that some posters seemed to think otherwise. Is it an “appropriate” analogy for GCM’s? That’s debatable. Analogies usually aren’t perfect.
I prefer the stock market analogy. I would have much more confidence in a forecasted rise in the Dow for 50- year period than a 1-year period. Why? Because I have observed the market has fluctuated a lot from year to year, but has risen over the long term. Average global temperature, like the stock market, fluctuates from year to year but has been rising over the long term
I prefer the “throwing a stone of a cliff” analogy. There will be times when the stone bounces back up off the rough cliff face. These will be difficult to predict, but the probability is that it will still end up at the bottom of the cliff. So although we might not be able to track the exact path taken by the stone we can calculate the right end result.
By the way, there’s an inherent bias in finite-precision computer calculations that has to be carefully coded for in certain situations involving tiny differences… and I suspect climate models are subject to this bias.
Sadly, I’ve not had time to re-research and recreate a demonstration. A friend proved it mathematically in an unpublished (internal R&D lab) paper that I can’t find (nor can I find him), but I have code that implements the solution somewhere in my ancient archives.
The nature of the problem:
– Imagine a huge sheet of graph paper; all available finite-precision numbers are at the grid points.
– When you do a calculation, by definition the answer will be rounded to a grid point.
– Most of the time, this rounding is not a problem.
– Some of the time, this rounding causes terrible trouble.
My case: the intersection of two sloping lines.
Depending on how you do the calculation, and in what order (sorry, don’t recall the exact details, that’s what I need to go dig up), the rounded intersection-gridpoint will be different for the same two lines.
Which means the intersection gridpoint is shifted in a particular direction.
These errors can rapidly accumulate for highly iterative functions… like mapping… and perhaps climate models.
I’d be happy for someone else to grab the glory on this… my time is quite limited these days 🙂
area is not extent!
@ur momisugly Wren
FWIIW, I am a “do nothing” person unless it can be demonstrated to me beyond reasonable doubt that “doing something” is all of the following: necessary, possible, desirable, and not likely to do more harm than good.
Xi Chin (01:56:42) :
Thanks for the link! Looks like the poor Microsoft random number generator is a story in itself.
http://www.theregister.co.uk/2007/11/13/windows_random_number_gen_flawed/
Wren (22:58:27) :
“Warmer” is a relative term. So far we are running well below all three of Hansen’s A,B and C scenarios.
@Wren
And BTW, I used to be a CAGWist too until fairly recently when, thanks to “Climategate”, I discovered that my trust in climate scientists to give me honest, unbiased information on which to base my opinions and actions had been seriously misplaced.
Since then my eyes have become opened ever wider to the true state of affairs and I now actually feel almost like someone who has escaped from the grasp of Scientology.
It is worth noting that even in the better g++ plot, there is a lot of non-random behaviour exhibited. There are somewhat symmetrical gaps, bunching and other patterns, particularly near the top.
http://wattsupwiththat.files.wordpress.com/2010/04/meierquestion9probabilityplotgcc.jpg
None of the RNG issues make any difference though in choosing “weather” or not to take the bet.
Daniel H (03:19:20) :
My objection to the coin toss analogy has nothing to do with choice of statistics. Rather it has to do with physical processes – basically that each iteration of a GCM run is dependent on all previous ones.
Suppose that in year one, the GCM predicts a decrease in Arctic ice and cloud cover, when in fact we see the opposite. This is going to compound into even worse errors in year two. Then again the error gets worse in year three. Suppose you run your model on a granularity of one day – that would be 36,525 iterations of compounded error in 100 years.
I can’t think of a single reason to believe that GCMs have validity 100 years into the future.
I read up on how to get wordpress to ignore things that look like html tags. Here is another attempt to post the C++ code, hopefully in it’s entirety this time.
#include <iostream>
#include <vector>
int
main(int argc, char** argv)
{
size_t sample_size = (size_t)atoi(argv[1]);
int iterations = atoi(argv[2]);
std::vector<int> ones_count_vector(sample_size, 0);
for (int i = 0; i < iterations; i++)
{
int number_of_ones = 0;
for (size_t j = 0; j < sample_size; j++)
{
int random_number = rand();
if ( (random_number & 1) == 1 )
{
number_of_ones++;
}
}
ones_count_vector[number_of_ones]++;
}
for (size_t i = 0; i < sample_size; i++)
{
if (ones_count_vector[i])
{
std::cout << i << " " << ones_count_vector[i] << std::endl;
}
}
}
Steve Goddard (13:10:20) :
for (size_t j = 0; j < sample_size; j++) { int random_number = rand(); if ( (random_number & 1) == 1 ) number_of_ones++; }A common problem with psuedo random number generators is that the low bits often cycle over a small range. If you had looked at all the bits I suspect you would have gotten a better result, e.g. “if (random_number > (RAND_MAX/2))”
Other functions can be embarrassingly bad. When I joined DEC in 1974 I brought with me program I had written at CMU to display abused forms of a curve called the Rose (see http://wermenh.com/roses/index.html ). Before porting the assembly language version to their graphics hardware, I wrote a Fortran program to show what it looks like. The “0th order” rose is just a circle and I was astounded to see my circle was lumpy, sort of cross between an octagon and circle. It was much closer to a circle than an octagon, but my sine & cosine code in my assembler code simply had a table of since between 0 and 90 degrees and did linear interpolation to increase resolution to 1/8 degree. The value of the result was only 14 or 15 bits wide. Fast and far better than what the Fortran library gave me! DEC fixed it in the next release.
Old PDP-11 hackers(*) – while it didn’t save me any code, the linear interpolation was just a string of seven add instructions preceded with a an “add r0, pc” instruction. I always want to use a math instruction on the pc register, but was disappointed that it wasn’t tighter code than a more sensible form.
*: Are there any young PDP-11 hackers? Probably not….
Wren (22:26:02) :
[…]
===
The “do nothings” on CAGW have an implicit forecast of no change in global temperature or a forecast of no change that man could do anything about, depending on which “do-nothing” you ask. These forecasts are based on wishful thinking rather than climate models.
Same difference, I believe…
There is no such thing as an “unbiased” coin.
There are only coins for which the bias is not detectable within whatever measurement/calculation method used.
Any real coin will have a center of mass which differs from the center of its exterior. There are numerous other bias elements which will vary from test to test.
In effect, Dr. Meier’s wager is whether or not the bias of the coin is detectable with a specific method. It also assumes that the specific method will not introduce bias (ie, deform the coin by repeated tosses).
If I were to lose a $ million if the coin flip came out of the range I would not make the bet, because I can’t afford $million. But that is what those who propose cap and trade or carbon taxes are wanting us to do. Actually they want us to bet trillions of dollars that THEY are right. But if they are wrong we have lost trillions of dollars for nothing.
It is worse than that, the bet is that the computer models are right and that the cost of waiting to see if they are right is higher than the cost of making drastic changes now. As bad as GCM are, economic models are even worse at knowing what conditions will exist in 50-100 years especially when a solution to what ever problem we might have then could rest in technology that does not exist now.
Steve Goddard (13:44:30) :
Sneak preview of something I am writing up. Here is GISS vs. CO2 concentration.
http://docs.google.com/View?id=ddw82wws_5998xrxhzhc
Steve, you have to convert the temperature trend fit to the Ln(CO2).
John Finn (04:00:28) : Quote “I prefer the “throwing a stone of a cliff” analogy. There will be times when the stone bounces back up off the rough cliff face. These will be difficult to predict, but the probability is that it will still end up at the bottom of the cliff. So although we might not be able to track the exact path taken by the stone we can calculate the right end result.” End Quote
Yes that is because we know that the only forcing is gravity and the only feedbacks are air resistance and the cliff. We don’t know that the main forcing is only CO2 and we really don’t know what the feedbacks are. if the feedbacks are negative, in your analogy the rock could come back and hit you in the face, or in the case of climate change the climate could get colder.
“The Met Office has now admitted to BBC News that its annual global mean forecast predicted temperatures higher than actual temperatures for nine years out of the last 10.”
I have a suggestion for our Brit friends. You may be aware of “Punxsutawney Phil” our national forecaster of spring arrival. Now Phil is busy on February 2, but available for the other 364 days per year.
According to Wikipedia… “As to his accuracy, according to the StormFax Weather Almanac and records kept since 1887, Phil’s predictions have been correct just 39% of the time.”
http://en.wikipedia.org/wiki/Punxsutawney_Phil
With an accuracy rate almost 4 rimes greater than the Met, I’m sure the citizens of Punxsutawney would be willing to rent out Phil for your forecasting needs in exchange for the annual budget of the Met Office. Think about it.. almost 4 times the accuracy at no additional cost. And when you think about the added benefit of carbon footprint reduction of replacing that energy gobbling computer and elimination of staff commuting, how can you resist! Groundhogs of the world.. unite!
Bill Illis (07:48:53) :
I understand your desire to make the GISS trend look logarithmic, but so far it appears linear with CO2. I’m just plotting the data, not generating it.
Steve,
Ric Werme is correct. You are using the output from a random number generator incorrectly.
Unless you know specifically what algorithm is used, you should never use the lowest bits of the generators state. The fact that GNU’s standard library uses a different algorithm that does not have this short-comming in no way means that Microsofts generator is flawed.
Microsoft is using the Linear Congruential Generator (LCG) method, which is fairly standard and robust as long as good constants are used. All LCG’s have poorly behaving low bits, regardless of the constants chosen.
In general cases, you should be dividing the output of rand() by (RAND_MAX + 1) to get a value 0 <= x < 1.0, and then multiply this value by the range of values you want (if you want 2 ouputs, multiply by 2), finally truncating this resulting value to an integer.
For further reference, see Knuth – The Art of Computer Programming.
Ric Werme (06:20:06) :
I tried using some different bits from rand() in VS. Bit 1 gave a very weird plot, but bit 12 did reasonably well.
http://docs.google.com/View?id=ddw82wws_602dp5ffdtb
>> R. Gates (14:48:12) : Except 2010 follows on the heals of the warmest decade on instrument record, and so is part of an ongoing trend. … <<
The instrument record only goes back to 1979. The late 70's were notoriously cold. It's hardly surprising that one could find a three month period of record warmth on a noisy half-sinusoid starting at the bottom, nor that the top of the sinusoid is warmer than the incline.
If you want to use temperatures before 1979, then you need to compare apples to apples, and only use the current temperatures from those areas that were previously covered by thermometers. That eliminates the oceans, as well as much of Africa and South America. The early 2010 temperatures in the US, Europe, and Asia were near historic lows.
Joe (08:04:00) :
I expect a random number to be random in all bits. Microsoft doesn’t warn you to expect otherwise.
Do you work for Microsoft?