Metadata fail: 230 GHCN land stations actually in the water

Why is this important? Well if you are calculating UHI for stations by looking at satellite images of nightlights, like GISS does (see my post on it at CA) , you’ll find that there’s generally no city lights in the water, leading you to think you’ve got no urbanization around the station. Using only 10 lines of code, Steve Mosher finds 230 errors in NCDC’s Global Historical Climatological Network (GHCN) data that places the station over water, when it should be on land. Does this affect the calculation of Earth’s surface temperature? Steve Mosher investigates. – Anthony

Wetbulb Temperature

by Steven Mosher

click to enlarge

This google map display is just one of 230 GHCN stations that is located in the water. After finding  instances of this phenomena over and over, it seemed an easy thing to find and analyze all such cases in GHCN. The issue matters for a two reasons:

  1. In my temperature analysis program I use a land/water mask to isolate land temperatures from sea temperatures and to weight the temperatures by the land area. An area that would be zero in the ocean, of course.
  2. Hansen2010 uses nightlights based on station location and in most cases the lights at a coastal location are brighter than those off shore. Although I have seen “blooming” even in radiance calibrated lights such that “water pixels” do on occasion have lights on them.

The process of finding “wet stations” is trivial in the “raster” package of R. All that is needed is high resolution land/sea mask. In my previous work, I used a ¼ degree base map. ¼ degree is roughly 25km at the equator.  I was able to find a 1km land mask used by satellites. That data is read in one line of code, and then it is simple matter to determine which stations are “wet”. Since NCDC is updating the GHCN V3 inventory I have alerted them to the problem and will, of course provide the code. I have yet to write NASA GISS. Since H2010 is already in the publishing process, I’m unsure of the correct path forward.

Looking through the 230 cases is not that difficult. It’s just time consuming.  We can identify several types of case: Atolls, Islands, and coastal locations. It’s also possible to put the correct locations in for some stations by referencing either WMO publications or other inventories which have better accuracy than either GHCN or GISS. We can also note that in some cases the “mislocation” may not matter to nightlights.  These are cases where you see no lights whatsover withing the  1/2 degree grid that I show. In the google maps presented below, I’ll show a sampling of all 230. The blue cross shows the GHCN station location and the contour lines show the contour of the nightlights raster. Pitch black locations have no contour.

I will also update this with a newer version of Nighlights. A google tour is available for folks who want it. The code is trivial and I can cover that if folks find it interesting. with the exception of the graphing it is as simple as this:

Ghcn<-readV2Inv() # read in the inventory
lonLat <- data.frame(Ghcn$Lon,Ghcn$Lat)
Nlight <- raster(hiResNightlights)
extent(Nlight)<-c(-180,180,-90,90) # fix the metadata error in nightlights
Ghcn<-cbind(Ghcn,Lights=extract(Nlight,lonLat)) # extract the lights using “points”
distCoast <-raster(coastDistanceFile,varname=”dst”) # get the special land mask
Ghcn <- cbind(Ghcn,CoastDistance=extract(distCoast,lonLat))
# for this mask, Water pixels are coded by their distance from land. All land pixels are 0
# make an inventory of just those land stations that appear in the water.
wetBulb <- Ghcn[which(Ghcn$CoastDistance>0),]
writeKml(wetBulb,outfile=”wetBulb”,tourname=”Wetstations”)

Some shots from the gallery. The 1km land/water mask is very accurate. You might notice one or two stations actually on land. Nightlights is less accurate, something H2010 does not recognize. Its pixels can be over 1km off true position. The small sample below should show the various cases. No attempt is made to ascertain if this causes an issue for identification of rural/urban categories. As it stands the inaccuracies in Nightlights and station locations suggests more work before that effort is taken up.

Click to enlarge images:

0 0 votes
Article Rating

Discover more from Watts Up With That?

Subscribe to get the latest posts sent to your email.

96 Comments
Inline Feedbacks
View all comments
Dave L
November 8, 2010 5:42 am

Quick. Somebody notify John Abraham and the AGU. Those pesky skeptics are at it again. (See the preceding article on WUWT.)

November 8, 2010 5:53 am

@Robin
1) Since 1900, there are warming, cooling and warming periods visible in proxy data, here in glaciers:
http://www.ncdc.noaa.gov/paleo/pubs/oerlemans2005/fig3a.jpg
2) Considering the TLT record, Singer is correct.
http://climexp.knmi.nl/data/itlt_gl_1978:1997a.png
HadCRUT/GISS/global SST record shows warming at the same period, which can be from some part attributable to UHI.

Latimer Alder
November 8, 2010 6:06 am

Every one of these stations get serviced at least occasionally. How hard would it be to visit every single site over the course of, say, a year, get a GPS based location, and update the database? It could be done with little or no additional cost above the basic maintenance.

Actually getting the raw data would be the easy bit..and your proposal is a fine one.
But the real problem would be to construct the technical, procedural and cultural ‘infrastructure’ to make use of the data so collected.
We have seen in Harry_read_me that CRU for example are absolutely clueless about data archiving and retrieval, that they have no consistent process for handling their ‘adjustments’, that they most definitely do not want any form of outside scrutiny of their work in exchange for their grant money and that their standards of Information Technology disciplines fall far short of those expected of even a talented amateur in the field.
But the most revealing (and worrying) thing that Harry inadvertently revealed is that they are unashamed by this! Charged with keeping one of the three global datasets that may hold the key to ‘the most important problem facing the world’, they are content to bumble along in their shambolic way..occasionally wiping the fag ash and cobwebs off a pile of old Chinese papers just to ensure that they don’t want to let others see them.
They seemingly have never even thought to visit other institutions whose mission is to keep data secure and with meaning. No concept crosses their collective wisdom that others have faced and solved similar problems and that perhaps there are lessons that could be learnt. Nor that their ‘mission; is suffciently important (or so some believe) that they have a professional and social duty to use the highest standards that have been developed…not the lowest.
It will take years, even with a complete change of personnel in such an institution, to get the data they do have into a state where your most helpful suggestion can be fully exploited (which doesn’t mean that a start should not be made).
The changes needed are primarily cultural….to imbue the whole field with the importance of consistent accurate and verifiable data collection. With consistent accurate and verifiable ‘adjustments’ if these prove necessary. With a relentless focus on the data as the only actual truth…not on modelling predictions.
There is a long, long, long way to go. But until we arrive at somewhere much nearer that ideal, everything else that has been done is just castles in the air.

Tenuc
November 8, 2010 6:10 am

Robin Guenier says:
November 8, 2010 at 4:11 am
“O/T (slightly). In an article in American Thinker (here), Fred Singer says:
Even more important, weather satellite data, which furnish the best global temperature data for the atmosphere, show essentially no warming between 1979 and 1997.
Again, is that correct?”

Yes. RSS satellite data shows no statistically significant global warming 1979-97.
Here’s the data, you can see for yourself.
http://woodfortrees.org/plot/rss/from:1979/to:1997/plot/rss/from:1979/to:1997/trend
Woodfortrees is a great site for checking out what you read on climate issues as you can easily compile your own plots of climate data metrics, link here:-
http://woodfortrees.org/plot/

simpleseekeraftertruth
November 8, 2010 6:12 am

The fact that 230 stations were found to have coordinates in the sea proves that at least 230 stations are incorrectly assessed for UHI. 230 is the minimum number that are wrong as the method used (land/water masking) only detects those that have this charateristc.

Dave
November 8, 2010 6:14 am

It’s worse than we though! Look at the evidence right here of extensive recent sea level rises! All those points were classified as land previously, so if they’re not now, it must be due to rising sea level.

juanslayton
November 8, 2010 6:18 am

John S:
As they switched from relying on interpolating coordinates from maps to direct GPS measurements, they have been updating the database. One complication: rather than correct the existing reported coordinates, they enter the new, corrected coordinates as a station location change, leaving the old coordinates as an apparent previous location. Took me about a year and a half to figure this out; meanwhile I wasted a lot of time and effort trying to run document locations where there had never been a station.
Juan S.

LearDog
November 8, 2010 6:21 am

Wow. Just wow. Fantastic work Mosh.
And I liked that funny bit about ‘it seemed an easy thing to find and analyze all such cases in GHCN’. Ha ha ha! Cracked me up. I laughed, really did.
Finding and analyzing all cases was so easy in fact – that the folks in charge of the database hadn’t corrected it ? I doubt that. You clearly have a skill set that THEY do not possess.

Viv Evans
November 8, 2010 6:30 am

Now I understand why these are called ‘wet bulb readings’ ….
😉

Steve Fitzpatrick
November 8, 2010 6:31 am

Hi Mosh,
Nice work; certainly very interesting.
I have a question: If there is frequent inaccuracy, as your work clearly shows, might we not expect that inaccuracy to indicate that stations nowhere near the water to also suffer significant inaccuracies? That is, doesn’t any random inaccuracy in station location almost automatically imply an understatement of night-light based UHI adjustment? Urban areas are generally rather small, so on average any random inaccuracy would (I think) tend to locate the station further away from the brightest regions; it should not matter the direction of the inaccuracy. Placement of stations over water makes the error obvious, but I wonder how many other significant errors exist where the obvious clue of water vs. land is not available.

juanslayton
November 8, 2010 6:37 am

Don’t like salt water? Go to Northport, Washington. Both GISS and the MMS will put you into the Columbia. For real location check the Surfacestations.org gallery : > )

November 8, 2010 6:48 am

chu says: “It’s worse than we thought, undetected massive sea level rises.
Lovely!

November 8, 2010 7:03 am

Latimer Alder says:
The changes needed are primarily cultural….to imbue the whole field with the importance of consistent accurate and verifiable data collection. With consistent accurate and verifiable ‘adjustments’ if these prove necessary.
I would second that!
Irrespective of which side of the “fence” anyone sits, it ought to be common ground that the quality of these measurements should be first class. When they seem to have nothing else to do, what are we paying for except world class data handling?
Anyone who has ever installed a quality system knows the principle that in a quality system you have as much concern for the smallest/simplest problems such as the position of stations, because if you’re failing with the simple basic things then it is almost certain you are failing with the more complex issues.
We know the present system has no credibility because we know the quality is totally abysmal. That should not be a partisan issue — even if you believe the world is warming due to mankind, you still want to have good accurate data on which to base action.
One of the clearest indicators that global warming is not a serious problem is that we can see that none of the establishments are at all concerned about the abysmal quality of the present compilation of temperature data … poor quality, poor data handling, and a group of “professionals” who seem to spend more time editing wikipedia and real climate than correcting the many and obvious errors in the temperature record! If they don’t care about the temperature record, then why on earth should anyone else?

LearDog
November 8, 2010 7:09 am

Wow. Just wow. Great chunk of work Mosh. And I loved the line ‘it seemed an easy thing to find and analyze all such cases in GHCN’. Ha ha ha! Yeah, right. I laughed out loud on that one.
So easy in fact – that the professionals in charge of the database and conducting the analysis know about the issue but don’t care ? I don’t think so.
I think that YOU possess a set of skills and intellectual drive that THEY can’t possibly emulate. Amazing stuff here.

jaypan
November 8, 2010 7:11 am

Sure Abrahams’ “Climate Rapid Response Team” was built to work out such failures.

Bernie
November 8, 2010 7:13 am

Steven:
What kind of response did you get from NCDC? Were they appreciative of your work? Have they asked for more technical details?

LearDog
November 8, 2010 7:18 am

Am traveling and don’t have access to my data to address the question: is this merely an issue of precision (x.xx vs x.xxxxxx) in the database or are these systematic / siting problems?

November 8, 2010 7:21 am

I’m confused. Shouldn’t the headline read: Stations that are shown on water are actually on land? Are they actually in real life on water or land? I’m guessing the coordinates are wrong. It would be nice to know the ramifications of these errors as well. Once they fix the gps (if that is wrong) what would be change in data?

Gary
November 8, 2010 7:27 am

And this just audits errors in the current sitings. What about previous site locations which may have had much different temperature measurement issues?

November 8, 2010 7:28 am

Dave says:
November 8, 2010 at 6:14 am
It’s worse than we though! Look at the evidence right here of extensive recent sea level rises! All those points were classified as land previously, so if they’re not now, it must be due to rising sea level.

Not necessarily, Dave. You are making the “warmist” assumption.
It could easily mean that in those areas, the land has been sinking.
LOL

Martin Brumby
November 8, 2010 7:46 am

A bit O/T but also relevant. Read on….
The Royal Society is apparently having a bunfight to discuss / promote “Geo-engineering” solutions to the awful Irritable Climate Syndrome shock-horror-disaster.
The BBC invited some tame “boffin” onto their resolutely alarmist Today “news” programme this morning. After briefly mentioning some of the dopey solutions that had been suggested to stop us frying (even as we shiver), he was asked what was his personal favourite “Geo-engineering” wheeze. Interestingly, this “solution” didn’t involve mirrors, white paint or artificial volcanoes. Instead he suggested getting farmers to plant crops with shinier leaves. (It is to be hoped that these wouldn’t be products of GM technology…Aaaargh…the horror…).
The interesting point (and the one relevant to this thread) followed, with his estimate that the “shiney” leaf crops could make a difference of one degree to Global Temperatures!
Hmmmmmmmm.
Perhaps that’s what they mean about ‘green jobs’. Well someone has to take a damp cloth and polish those leaves.
But how much of a temperature increase have we seen so far since the start of the Industrial Revolution?
I wondered what allowance the “models” already include for changes in the ‘shineyness’ of vegetation in that period? Would this allowance be more or less than the allowances for UHI effect, dodgy records, the march of the thermometers, CRU tweaks, the end of the Little Ice Age, the effects of oceanic current fluctuations, moose dung next to Yamal trees and all those other exciting little things that we have learned about on here?
Definitively, worse than we thought.

Enneagram
November 8, 2010 7:46 am

“Meta-Data”, beyond data, beyond the “twilight zone” 🙂

1DandyTroll
November 8, 2010 7:51 am

No wonder those hippies are all screaming about the looming doom of sea level rise.
Bwaaap, haha, see what I did there?

j ferguson
November 8, 2010 8:06 am

Steven,
These 230 stations are part of what number of stations in current use by GISS?

Jeff
November 8, 2010 8:11 am

I would expect many locations on land are mislocated as well and may be improperly classsified as rural or urban depending on the error in location …
I think it is safe to say that nobody has good temperature data including location … and I mean nobody …
We need a to go tabula rossa on this … start over …