Is the NULL default infinite hot?
January 31, 2010 by E.M.Smith
see his website “Musings from the Chiefio”
The empty ocean goes infinite hot on a null anomaly
What to make of THIS bizarre anomaly map?
What Have I Done?
I was exploring another example of The Bolivia Effect where an empty area became quite “hot” when the data were missing (Panama, posting soon) and that led to another couple of changed baselines that led to more ‘interesting red’ (1980 vs 1951-1980 baseline). I’m doing these examinations with a 250 km ’spread’ as that tells me more about where the thermometers are located. The above graph, if done instead with a 1200 km spread or smoothing, has the white spread out to sea 1200 km with smaller infinite red blobs in the middles of the oceans.
I thought it would be ‘interesting’ to step through parts of the baseline bit by bit to find out where it was “hot” and “cold”. (Thinking of breaking it into decades…. still to be tried…) When I thought:
Well, you always need a baseline benchmark, even if you are ‘benchmarking the baseline’, so why not start with the “NULL” case of baseline equal to report period? It ought to be a simple all white land area with grey oceans for missing data.
Well, I was “A bit surprised” when I got a blood red ocean everywhere on the planet.
You can try it yourself at the NASA / GISS web site map making page.
In all fairness, the land does stay white (no anomaly against itself) and that’s a very good thing. But that Ocean!
ALL the ocean area with no data goes blood red and the scale shows it to be up to ‘9999′ degrees C of anomaly.
“Houston, I think you have a problem”…
Why Don’t I Look In The Code
Well, the code NASA GISS publishes and says is what they run, is not this code that they are running.
Yes, they are not publishing the real code. In the real code running on the GISS web page to make these anomaly maps, you can change the baseline and you can change the “spread” of each cell. (Thus the web page that lets you make these “what if” anomaly maps). In the code they publish, the “reach” of that spread is hard coded at 1200 km and the baseline period is hard coded at 1951-1980.
So I simply can not do any debugging on this issue, because the code that produces these maps is not available.
But what I can say is pretty simple:
If a map with no areas of unusual warmth (by definition with the baseline = report period) has this happen; something is wrong.
I’d further speculate that that something could easily be what causes The Bolivia Effect where areas that are lacking in current data get rosy red blobs. Just done on a spectacular scale.
Further, I’d speculate that this might go a long way toward explaining the perpetual bright red in the Arctic (where there are no thermometers so no thermometer data). This “anomaly map” includes the HadCRUT SST anomaly map for ocean temperatures. The striking thing about this one is that those two bands of red at each pole sure look a lot like the ‘persistent polar warming’ we’ve been told to be so worried about. One can only wonder if there is some “bleed through” of these hypothetical warm spots when the ‘null data’ cells are averaged in with the ‘real data cells’ when making non-edge case maps. But without the code, it can only be a wonder:
With 250 km ‘spread’ and HadCRUT SST anomalies we get bright red poles
The default 1200 km present date map for comparison:
GIS Anomaly Map for November 2009
I’m surprised nobody ever tried this particular ‘limit case’ before. Then again, experienced software developers know to test the ‘limit cases’ even if they do seem bizarre, since that’s where the most bugs live. And this sure looks like a bug to me.
A very hot bug…
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
I recall reading somewhere else that ‘9999’ isn’t a temperature but is instead an error code. Not sure what it’s complaining about, you’d need to look at the code to find out, and there are probably more than one set of conditions that produce that output.
wonder if that model was peer reviewed or beer reviewed?
Reminds me of all that Steigian red from last year’s Antarctic smearing.
So, if the thermometer is dead, it’s red!
http://bastardoldholborn.blogspot.com/2010/02/bbc-and-climate-change.html
fascinating!
Two winners today, no three
(1) Times are a-changin’ when Google Ads shows this
First-rate, too, IMHO. Want to run a thread on it here?
(2) important post here: the ridiculous QC of GISS has got to come out alongside that of CRU
(3) yesterday’s UHI arrived here this morning, just after I sent Warren Meyer a set of pics to think of using to enliven above video, one of which was my rendition of Anthony’s Reno urban transect overlaid on another pic resembling that used for Case 8. Ha, I feel proud.
But I still don’t know how Anthony manages to keep rolling it out, watching other blogs, writing papers, challenging Menne, and putting courteous notes in for the (declining) trolls here. Thank you from the bottom of my heart, moderators too.
If GISS truly uses 9999 for null, it just shows how out-of-date their software is.
It appears they don’t discard or ignore that value, either, when they create an image from the data. Wouldn’t it be better to show those areas as black?
One more indication that the code is ether poorly done or cooked.
Can we test this hypothesis with a bit of reverse engineering of data? Null values in data sets are a common problem I find in my work. (Software dev)
Why don’t they publish the freakin source code? Surely all these scientist types are open source zealots anyway.
So CRU don’t have a monopoly on crap warmmonger software. Who’d have thunk it?
999 or 9999 (and negative versions) are typically used to represent errors, as values such as 0 for no data can be misinterpreted as real data. Blank values can also cause an error in some code, hence the use of a silly low or silly high value, with the intention that the user imediately notice it or that it can be immediately identified as wrong. The code *should* ignore such values, but its not unheard of for these values to be pulled in as someone has used “9999” instead of the “999” value the software ignores.
I understand the code works by smearing hot spots around (homogenisation), so there is potential for one of these large numbers to make it through only to be smeared around between the other low numbers so the error is not immediately obvious to the eye. Hansen et al would expect (or would like) to see warming at the poles, so they make not see this as an error due to their preconcieved solid belief in global warming.
As chiefio states, this is why its so important to test the softwate or code to extremes. I use numerous engineering and hydraulic modelling packages, and reguarly find small and occassionaly large flows. I have a similar habit of testing extreme scenarios so I know where the software can and can not be trusted, but also manually replicating sections of the software engine using their own equations and comparing results – using this method I end up identifying faults that have gone un-missed for years!
Great work! This just demonstrates why we should never stop questioning – even if we are lowly peasent “non-climate scientists” incapable of computer engineering, maths, statistics etc..
The value 9999 is a standard ‘no data’ value. This is nonsense.
Interesting that all the errors of this nature (buggy code, etc) that I’ve seen show warming. I wonder what the odds of that are, if one assumes that they are indeed all errors and no bias was present? My guess is very long odds indeed, and that argues strongly for a non-accidental explanation.
Arizona CJ
Slightly OT, mod, but I need an answer fast and here’s the place to get it.
Australia’s Climate Change Minister, Penny Wong, claimed in an article in today’s The Australian that, “Globally,
Surely that’s a porky.
Slightly OT, mod, but Australia’s Climate Change Minister has claimed the following in today’s The Australian.
“Earlier this month, the Bureau of Meteorology released its 2009 annual climate statement. It found 2009 was the second hottest year in Australia on record and ended our hottest decade. In Australia, each decade since the 1940s has been warmer than the last…
Globally, 14 of the 15 warmest years on record occurred between 1995 and 2009.”
Breaking News;
Glenn Beck just reported on Dr Rajendra Pachauri’s steamy new sex novel and Obama’s new budget that includes $650 billion from carbon taxing, says Beck, ” what does the Obama administration know that we don’t know?”
The world wealthy elite are pushing ahead with their global government carbon taxing schemes.
And if you think this is a joke, it’s not.
I watched the Davos world economic forum interviews and the most powerful people in the world are talking as if the global carbon tax is a given.
Carbon taxing is based on a lie for Christ’s sake.
Just another example of poorly written or intentionally deceptive code.
If a computer model, cannot be audited in public and validated that the code that is reviewed is the actual code being used to run the simulations the output is no better than throwing dice or tossing darts at a wall map, and should never be used in any way shape of form in the development of policy or legislation.
We need a law passed that explicitly blocks use of any model simulation for policy, regulation or legislation without a verifiable audit process.
Any computer programmer that has gotten past “Hello world” level programming knows it is impossible to write error free code containing thousands or millions of lines of code. The best you can do is to thoroughly test for and eliminate the coding and logic errors that are most likely to bite you under unexpected conditions.
A couple decades ago, I was asked to beta test a relatively simple program used by FEMA to process some shelter data. The first think I did was every time it asked for a value I hit every odd ball key on the key board like ! @ur momisugly # etc. I blew the program up about a dozen times simply because such nonsense key strokes were not tested for and trapped with an error routine that enforced reasonable input limits.
If this code cannot be open sourced or audited by a professional soft ware audit process it should be barred from use.
Larry
You just broke their online widget by asking it to do something that didn’t make any sense. Do you really think the error flag 9999 is entering into real calculations anywhere?
I really don’t understand your fascination with changing the baseline. It’s a waste of time. You’ll just nudge the absolute values of the anomalies up and down, but the trends won’t change.
As for your Bolivia effect, you could probably figure it out with just one day’s effort. GISS finds the temperature at each grid point by adding in surrounding stations, linearly weighted based on the distance from the grid point. Just take the current surrounding stations, and you can confirm their calculation. Then, you can easily check to see how good the interpolation is.
If there is data from Bolivia before 1990 or whenever, just repeat that calculation, pretending that you can’t see the Bolivian stations. Then compare the interpolation to the actual Bolivian data, and you’ll see how good the interpolation is. If you want people to take you seriously that the interpolated values for Bolivia are not good, why don’t you actually do that analysis and find out?
OFF TOPIC –
The British monthly car magazine ‘What Car’ has an article on global warming. At first glance it seems to take the ‘rising CO2 is going to cause dangerous harm – and man is to blame’ claim as factual and even has a 700,000 year graph of CO2 which looks pretty hockey stickish to me at its end.
I hope if needs be someone will correct any of their errors . The magazine claims that current CO2 is at an all time high, but my dim memory thinks it has read something about it being much higher in the far past.
I see you haven’t noticed it doesn’t matter if this part of NASA is correct or not because they just keep getting the money. The moon program is now dead and Obama is rerouting that money to climate monitoring. Hansen and the boys are all grins-That’s what makes me see red!! Obama needs cap and trade for the taxes and the control. As long as these type people are in control we will all be seeing red ,especially in our bank statements. I don’t know when everyone will wake up and see this. Maybe Mass. was a start-we’ll see.
lengthy interview with Pachauri in the Hindustan Times:
Pachauri: ‘They can bend me, but they can’t break me’
Q: Can you provide us revenue for 10 years to prove there is no link between IPCC and TERI?
A: …. I have proved myself in several aspects in the world. Not in eyes of Sunday Telegraph. Fortunately, there are a few people, thank God, like the Sunday Telegraph. But yes, if you want, we can provide the accounts, the payments made over these 10 years.
http://www.hindustantimes.com/They-can-bend-me-but-they-can-t-break-me/H1-Article1-504204.aspx
COMMENT BELOW ARTICLE: Yes, we do want … what are you waiting for? We’ve already asked. Richard North, Sunday Telegraph
also from the interview which needs to be read in full:
Q: Did failure at Copenhagen help climate skeptics?
A: No agreement at Copenhagen in fact encouraged some of the deniers, and those who are financing them with, maybe millions, who knows, billions of dollars.
Hi,
I’ve been digging through the code (just getting it to run on my beefy Linux box). I notice that when they do the griding:
1) They seem to use a linear weighting around the center of each grid, so a station twice as far away from the center contributes half as much to the total – shouldn’t that be some form of inverse square law relationship instead? as the temperature at a given point interacts in all directions – using pure linear is very akin to resistance over distance in a wire.
2) From what I’ve seen in the code (I maybe wrong, being a good many years since I did FORTRAN) they have some weird rule around where a station can be outside 1200km yet still inside the grid – but in this case they give all such stations the weight of the furthest station within the 1200km radius – so pulling such external stations ‘inside’ the 1200km boundary.
once I have the code running I’ll see how this impacts the results.
It seems to me that temperature data analysis methods must accept both positive and negative single or double precision values for input. For this reason, the developer will use an out-of-range value to represent NULL data. If the application is written properly, whatever value is used to represent NULL should be tested for and not be used in any calculation that drives output.
Many modern programming languages now support the NaN value in numeric data types, which can be used to represent NULL or erred data. The beauty of NaN is that it will throw an exception if used in a calculation or used as direct output to any control that expects a real number. That assures that a NaN cannot be accidentally included in analysis, even if by programming error. Me thinks it is time for NASA to upgrade to modern programming languages and systems.
I wonder if they’ll “lose” data for Scotland this winter now that we’ve heard they have had their coldest ever Dec-Jan on record.
Oh, all you skeptic scientists have missed out and I’m sorry. But at least you have kept your dignity and everything else. Keep after them- it just might turn around in your favor. Everyone else help out as much as possible by voting and giving.