Since we’ve been talking about IPCC’s “Africagate” recently, it seemed like an opportune time to point out what sort of GISS station adjustment goes on in data from it’s nearby neighbor island. Welcome Verity Jones first guest post on WUWT. FYI for those who don’t get the implied data munging title, “Munge” is sometimes backronymmed as Modify Until Not Guessed Easily. – Anthony
Guest post by Verity Jones
This started out as a discussion point following E.M. Smith’s blog post Mysterious Madagascar Muse. The gist of the original article centered around the availability of data after 1990 in the GHCN dataset and the NASA/GISS treatment of temperature on the island. Well Madagascar has a bit of a further story to tell. I had offered to plot a ‘spaghetti’ graph of the temperatures from the ten stations used on Madagascar, and this has proven interesting as an example of how data is adjusted and filled in by GISS.
To start, the annual mean temperatures plotted on a graph (Figure 1) show clearly the differences between the stations – Antananarivo is high altitude and relatively cool, with a cooling trend; of the other stations, some have cooling trends, most are warming. Also noticeable is the very sparse data after 1990. Note the darker blue data for Maintirano, of which more later.
With such temperature differences between sites, obviously you cannot just average the temperatures. This is what it looks like if you do (Figure 2), and it clearly does not work as an average temperature for the island.
Normalizing each of the temperature series by calculating the mean temperature for that station for the baseline period of 1951-1980 allows plotting of an anomaly-based ‘spaghetti’ graph (Figure 3). This shows what looks like warming-cooling-warming climate cycles very clearly and it is possible to fit a third order polynomial trendline though the averaged data. I’ve seen this again and again for data I’ve plotted around the world (incidentally these were for WUWT regular TonyB).
E.M.Smith finds seven other rural stations within 1000km that may contribute to homogenization. They also show cooling to about 1965-1975, then a warming trend. This is lost from the homogenized data.
One final thing. Even the patchy data stops in 2005, so after this date Madagascar too gets ‘filled in’ data from elsewhere – it seems from the rural stations up to 1000km away – again. And even the stations used to ‘fill in’ have patchy data – many have a gap then ONE DATA POINT in 2009. Note that there was no data for 10 years prior to 2009 in this station.
This is unbelievable. Rather than give a lot of plot examples, check the station hyperlinks below for yourself:
| Ile Juan De N | 17.1 S | 42.7 E | 111619700000 | rural area | 1973 – 2009 | ||
| Dzaoudzi/Pama | 12.8 S | 45.3 E | 163670050000 | rural area | 1951 – 2009 | ||
| Iles Glorieus | 11.6 S | 47.3 E | 111619680000 | rural area | 1956 – 2009 | ||
| Ouani (Anjoua | 12.1 S | 44.4 E | 111670040000 | rural area | 1963 – 1984 | ||
| Serge-Frolow | 15.9 S | 54.5 E | 168619760000 | rural area | 1954 – 2009 | ||
| Ile Europa | 22.3 S | 40.3 E | 111619720000 | rural area | 1951 – 2009 | ||
| Porto Amelia | 13.0 S | 40.5 E | 131672150004 | rural area | 1987 – 200 |
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Fort-Dauphin is my favorite. Lots of 999’s near the end…
http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=125671970000&data_set=1&num_neighbors=1
Great post – but the title makes me feel a little ill.
Well it confirms what everybody suspected: if the US data gathering is as bad as the surface station project shows, it can only be worse from the rest of the world, with the political unrest and/or lack of technology that characterizes most of it.
So that is GHCN which seems to be patchy and erroneous?
One Ilya Goz and John Graham-Cumming have jointly discovered what maybe an error in the HADCRUT3 station error data.
The apparent mistakes include:
1. “.. it appears that the normal error used as part of the calculation of the station error is being scaled by the number of stations in the grid square. This leads to an odd situation that Ilya noted: the more stations in a square the worse the error range. That’s counterintuitive, you’d expect the more observations the better estimate you’d have.
2. “The paper [Brohan et al.] says that if less than 30 years of data are available the number mi should be set to the number of years”. In some cases though the number of years is less the error appears to have been incorrectly calculated based on 30 years.”
What is even more intriguing – “..Ilya Goz.. correctly pointed out that although a subset had been released, for some years and some locations on the globe that subset was in fact the entire set of data and so the errors could be checked…”
The apparent errors Ilya and JCG have apparently revealed is indeed interesting. But what is also most interesting is that “a subset” which only applies for “some locations on the Globe” are in fact “the entire set of data”?
Is it true then, that HADCRUT3, which is supposed to represent the temperatures for the entire Globe, only represents those of some locations on the Globe, and that too inaccurately?
The questions that arise: Which locations? What parts of the Globe are left out and not represented? Why is this? If they were represented how would that affect the temperatures?
Is this another scam of epic proportions in the making?
Like the NIWA data, this is simply fraud. Unlike NIWA, the justification, it does not even have the rationale of the bogus temp adjustments.
even with no scientific education whatsoever, it’s clear what verity’s observations are telling us:
verity and/or anthony may be able to make something of the following:
Bishop Hill: Has JG-C found an error in CRUTEM?
Climate John Graham-Cumming, the very clever computer scientist who has been replicating CRUTEM thinks he and one of his commenters have found an error in CRUTEM…
http://bishophill.squarespace.com/blog/2010/2/7/has-jg-c-found-an-error-in-crutem.html
which links to:
Something odd in the CRUTEM3 station errors
http://www.jgc.org/blog/2010/02/something-odd-in-crutem3-station-errors.html
a little levity to end:
Business Standard, India: Abdullah bats for Pachauri, says IPCC chief wrongly targeted
“Recently, Pachauri is under tremendous attack. People did not spare Jesus Christ, Prophet Mohammad when they spoke about harmony and good will. Gandhi was targeted for his good work also. Today, Pachauri is being targeted… One day, they will realise,” (Union Renewable Energy Minister Farooq) Abdullah said at the Delhi Sustainable Development summit…
http://www.business-standard.com/india/news/abdullah-bats-for-pachauri-says-ipcc-chief-wrongly-targeted/85119/on
Australian: Feral camels clear in Penny Wong’s carbon count
Scientists have found camels to be the third-highest carbon-emitting animal per head on the planet, behind only cattle and buffalo. Culling the one million feral camels that currently roam the outback would be equivalent to taking 300,000 cars off the road in terms of the reduction to the country’s greenhouse gases.
But Climate Change Minister Penny Wong told The Australian there was little point doing anything about Australia’s feral camels as only the CO2 of the domesticated variety is counted under the Kyoto Protocol…
http://www.theaustralian.com.au/news/nation/feral-camels-clear-in-penny-wongs-carbon-count/story-e6frg6nf-1225827641354
finally, the determination of the ‘players’ involved is expressed neatly in the final line of the FinTimes below:
UK Financial Times: Tony Jackson: Carbon trading’s riders hold on to the handlebars
In other words, the whole climate change agenda could head off in unexpected directions. In that context, carbon trading could turn out like a bicycle: it moves forward, or falls over. If the latter, bad luck. But it would not be the end of the story.
http://www.ft.com/cms/s/0/b32a9e14-1452-11df-8847-00144feab49a.html
Just eyeballing this …
… it appears that the 10 year running average of the summed averages of the anomalies for the adjusted data is a better fit to Fig 3 than the summed averages of the anomalies for the raw data.
Maybe you could run a diff between your two series in Fig 5 and that in Fig 3.
I’m betting that the adjusted data make a better fit. It seems that your ‘Normalized Unadjusted Annual Mean Temperatures for Stations on Madagascar’, at least superficially, resembles GISS normalization! 😀
I also notice that your station list seems to be truncated on Fig 1.
I count 10 stations, 6 warming and 4 cooling (linear trend).
You only name only 8 on the plot.
(missing red square and purple diamond)
A stage where I am surprised in any way at the manner of temp data handling, is already far behind.
When considering Madagascar climate date it is important to consider two key factors that may have a significant impact on that data – 1) Deforestation and 2) Political instability.
The island of Madagascar has lost up to 90% of its original forest cover – half of that in the past 50 years. Many parts of the country suffer from serious erosion and land degradation. The loss of forest cover of such an extensive area surely has climate impacts – especially concerning temperatures and precipitation.
Madagascar, like many former colonial nations, has been beset with political instability since independence in 1960. Worldwide, the loss of weather recording stations in the past half century has been dramatic – but even in those which remain, quality control is an issue that should be addressed.
I began here about half an hour ago and started on the click-here tree to follow some of the other goings-on around the world in other countries with temperature data issues. It is a comfort to know that in the USA our weather reporting stations are reporting continuously with the most technically advanced equipment, properly sited, and properly documented.
What’s that you say? — Really! — Where?
http://www.surfacestations.org/
Averaging temperatures from different sites is indeed a bad idea, but averaging each site’s temperature trend over consistent time frames can give an idea how the area fared.
I used this approach for the hadCRUT3 temperatures for the USA.
http://sowellslawblog.blogspot.com/2010/02/usa-cities-hadcrut3-temperatures.html
I also just noted that the simple averaged trends of GISTemp data shows less warming than the simple averaged trends on VJ’s TEKtemp database. VJ’s Quality Control increases the linear warming trends
I was wondering why Fig 3 had a greater range than Fig 5.
VJs TEKtemp, adj – raw ave trend: .949 C
GISTemp, adj – raw ave trend: .717C
VJ, what adjustments are you making? Is the TEKtemp just using GISTemp, but applying different filters to determine what is included in the final numbers? Or are you creating your own ‘adjusted’ series?
Verity,
I could follow your post easily. Good read. Although what it shows about GISStemp and NOAA/NCDC/GHCN dataset treatment of Madagascar temperatures is not good.
Question 1 – Have you seen, online or in literature, any NASA/GISS rationalization(s)/explanation(s) about why they treated the Madagascar temperatures the way that you show? Or another way to ask, have you seen anything that tells us their intent?
Question 2 – Do you know if there are GHCN stations that exist in Madagascar that are not being used in the GISStemp product? If so, again have you seen any rationalizations/explanations from NOAA/NCDC or NASA/GISS as to why some stations may not be used? Again looking for their intent.
John
I am sure by now everyone realizes there are 2 pats here. LOL. And both from the nether regions.
I have done a similar analysis of GISS data for stations near Mackay in Australia and also found:
cooling trends turned into warming trends;
UHI correction DOUBLING a warming trend;
a nearby rural station reclassified as urban so its cooling trend can be adjusted;
the GISS homogenization adjustment MORE THAN the average trend of 7 rural stations up to 500km away;
data being up to 6 years in error.
I have put this study up at http://kenskingdom.wordpress.com/
So it’s definitely not a one-off accident!
With that sort of patchy data sets, they should switch to use treemometers!!! ;<)
From watching all of the inherent data recording errors, and problems uncovered by this continuing line of inquiry, in trying to establish a real surface temp average, that is properly weighted, for the total surface area, is a crap shoot.
Compared to just taking the past data from regional areas separately, and combining them into an analog weather forecast, that can then be applied to the separate regions, to better utilize the local resources available.
Investing this much time, researching into the patterns of natural variability, that generate the local characteristics of the weather, typical to the separate regions, would give a much more profitable outcome, in the short and long run.
Might be a much better option, that generates solutions to the real local problems, rather than just trying to further define the Global problem, and still getting nothing done any one place, for the time and money spent.
Not to devalue the process we are trying to sort out here, but if real solutions are to be gotten rapidly , it will come from better prediction of short and long term weather, so local/regional, land use behavioral changes can be made in a timely manor.
Not from taxing or penalizing random peoples, for the lack of a good forecast system, that serves the needs of the world, while being individually suited for each of the separate climate regions.
If the UN was the organization it claims to be, (with the moral fiber it should have), they would be working toward the end game, of sharing all data and methods that work, to benefit the most people, irregardless of who they are or where they live.
The best form of government, is the one that best serves the interests of the common man/woman, and cares for the needs of the people, governed with the least interference in their freedom, to live as they would like, assisting the formation of robust family structure, with open communication between neighbors, that relieves stress and increases productivity.
Another question as I try to understand what you did, VJ …
When you plot “unadjusted” data for a particular station, are you plotting all the records for that station – or have you merged the multiple records into a single series? If you have merged them, how? If you have only plotted record 0, then you have left out multiple data points (records 1, 2, …) in the raw data sets that are incorporated into the homogenized data.
I note that the following stations in your linked list include multiple records:
Iles Glorieus
Ouani (Anjoua
Serge-Frolow
Ile Europa
Porto Amelia
Has someone standardised UHI as a negative effect rather than a positive one for all these corrections?
Seems to be the case pretty often that UHI is used to adjust UP…
The real scandal here is not the adustments themselvs but the fact that so many adjustments have to made due to lack of data. I have worked with climate data in more than 30 countries worldwide (including Madagascar) and know that the lack of reliable data is widespread. The installation of, say, just 100 stations in those continents with poor data would cost a miniscule proportion of the trillion earmearked for climate change mitigation and be a marked improvement on what we have at the moment. After a year or two these new stations could also be used to add value to current patchy records.
Ron Broberg (21:58:46) :
VJ’s Quality Control increases the linear warming trends
The QC applied to the database is that it does not plot any years with a missing monthly value. This is because we found that the way the seasonal and monthly averages were being calculated seemed rather conveniently ‘filled in’, sometimes with higher temperatures. An example is here for Ile Europa:
http://data.giss.nasa.gov/work/gistemp/STATIONS//tmp.111619720000.1.1/station.txt
Note the D-J-F Average for 2009 is 28.3. Dec 2008 is missing (999.9) but if you back calculate GISS must have ‘assumed’ a value of 27.7 for Dec 2008) to calculate this value, which seems on the high side. This will also affect the annual mean temperature; that is why we don’t use years with missing months.
When you calculate a trend, it is sensitive to the span of years. If you toss out years at either end of the series it can affect the trend, that is why the database values differ. My collaborator Kevin has quantified the years with missing data – it is worse than we thought.
I was wondering why Fig 3 had a greater range than Fig 5.
Fig 3 is Unadjusted data, Fig 5 is Adjusted.
The only ‘adjustment’ the database makes is in two QC criteria:
– years with missing months are not plotted
– the code for automated graph plotting requires a minimum of 20 years (not necessarily consecutive) to plot a trend. Kevin has now reduced this to 10 years, because we have started to look at short-term trends which show the climate cycles very clearly.
I’ve had problems with Excel truncating the graph legends and cannot get all the stations to display in Fig 1. However, the missing ones are preferentially displayed in Figs 3/4 so it is possible to work them out.
Maastricht airport (Netherlands) according to GISS (USA)
http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=633063800000&data_set=1&num_neighbors=1
according to KNMI (Netherlands)
http://www.knmi.nl/klimatologie/daggegevens/datafiles3/380/etmgeg_380.zip
(daily values from 1906/01/01 until yesterday )
John Whitman (22:06:26) :
Thanks for the kind comments.
Question 1 – Have you seen, online or in literature, any NASA/GISS rationalization(s)/explanation(s) about why they treated the Madagascar temperatures the way that you show? Sorry, no I haven’t and I really can’t comment on intent.
Question 2 – Do you know if there are GHCN stations that exist in Madagascar that are not being used in the GISStemp product? In other parts of the world, yes, but I have not specifically looked at Madagascar.
Ron Broberg (23:04:25) :
Re treatment of multiple records. This was initially a problem for us. Kevin looked long and hard at it and checked everything he did against GHCN and GISS data (since we have separate databases for each). Overall he concluded that the way he combined stations was very close to that of NOAA/GHCN and NASA/GISS and the data and trends were not impacted. We have only started to see differences when looking at stations with patchy data for the reasons mentioned above.
All the graphs above use GISS’ own data – only the trends in the table use mention the database.
You Americans have a thing called class action lawsuits i think…