Guest Post by Willis Eschenbach
Recapping the story begun at WUWT here and continued at WUWT here, data from the temperature station Darwin Zero in northern Australia was found to be radically adjusted and showing huge warming (red line, adjusted temperature) compared to the unadjusted data (blue line). The unadjusted data showed that Darwin Zero was actually cooling over the period of the record. Here is the adjustment to Darwin Zero:
Figure 1. The GHCN adjustments to the Darwin Zero temperature record.
Many people have written in with questions about my analysis. I thank everyone for their interest. I’m answering them as fast as I can. I cannot answer them all, so I am trying to pick the relevant ones. This post is to answer a few.
• First, there has been some confusion about the data. I am using solely GHCN numbers and methods. They will not match the GISS or the CRU or the HadCRUT numbers.
• Next, some people have said that these are not separate temperature stations. However, GHCN adjusts them and uses them as separate temperature stations, so you’ll have to take that question up with GHCN.
• Next, a number of people have claimed that the reason for the Darwin adjustment was that it is simply the result of the standard homogenization done by GHCN based on comparison with other neighboring station records. This homogenization procedure is described here (PDF).
While it sounds plausible that Darwin was adjusted as the GHCN claims, if that were the case the GHCN algorithm would have adjusted all five of the Darwin records in the same way. Instead they have adjusted them differently (see below). This argues strongly that they were not done by the listed GHCN homogenization process. Any process that changed one of them would change all of them in the same way, as they are nearly identical.
• Next, there are no “neighboring records” for a number of the Darwin adjustments simply because in the early part of the century there were no suitable neighboring stations. It’s not enough to have a random reference station somewhere a thousand km away from Darwin in the middle of the desert. You can’t adjust Darwin based on that. The GHCN homogenization method requires five well correlated neighboring “reference stations” to work.
From the reference cited above:
“In creating each year’s first difference reference series, we used the five most highly correlated neighboring stations that had enough data to accurately model the candidate station.”
and “Also, not all stations could be adjusted. Remote stations for which we could not produce an adequate reference series (the correlation between first-difference station time series and its reference time series must be 0.80 or greater) were not adjusted.”
As I mentioned in my original article, the hard part is not to find five neighboring stations, particularly if you consider a station 1,500 km away as “neighboring”. The hard part is to find similar stations within that distance. We need those stations whose first difference has an 0.80 correlation with the Darwin station first difference.
(A “first difference” is a list of the changes from year to year of the data. For example, if the data is “31, 32, 33, 35, 34”, the first differences are “1, 1, 2, -1”. It is often useful to examine first differences rather than the actual data. See Peterson (PDF) for a discussion of the use of the “first-difference method” in climate science.)
Accordingly, I’ve been looking at the candidate stations. For the 1920 adjustment we need stations starting in 1915 or earlier. Here are all of the candidate stations within 1,500 km of Darwin that start in 1915 or before, along with the correlation of their first difference with the Darwin first difference:
WYNDHAM_(WYNDHAM_PORT) = -0.14
DERBY = -0.10
BURKETOWN = -0.40
CAMOOWEAL = -0.21
NORMANTON = 0.35
DONORS_HILL = 0.35
MT_ISA_AIRPORT = -0.20
ALICE_SPRINGS = 0.06
COEN_(POST_OFFICE) = -0.01
CROYDON = -0.23
CLONCURRY = -0.2
MUSGRAVE_STATION = -0.43
FAIRVIEW = -0.29
As you can see, not one of them is even remotely like Darwin. None of them are adequate for inclusion in a “first-difference reference time series” according to the GHCN. The Economist excoriated me for not including Wyndham in the “neighboring stations” (I had overlooked it in the list). However, the problem is that even if we include Wyndham, Derby, and every other station out to 1,500 km, we still don’t have a single station with a high enough correlation to use the GHCN method for the 1920 adjustment.
Now I suppose you could argue that you can adjust 1920 Darwin records based on stations 2,000 km away, but even 1,500 km seems too far away to do a reliable job. So while it is theoretically possible that the GHCN described method was used on Darwin, you’ll be a long, long ways from Darwin before you find your five candidates.
• Next, the GHCN does use a good method to detect inhomogeneities. Here’s their description of their method.
To look for such a change point, a simple linear regression was fitted to the part of the difference series before the year being tested and another after the year being tested. This test is repeated for all years of the time series (with a minimum of 5 yr in each section), and the year with the lowest residual sum of the squares was considered the year with a potential discontinuity.
This is a valid method, so I applied it to the Darwin data itself. Here’s that result:
Figure 2. Possible inhomogeneities in the Darwin Zero record, as indicated by the GHCN algorithm.
As you can see by the upper thin red line, the method indicates a possible discontinuity centered at 1939. However, once that discontinuity is removed, the rest of the record does not indicate any discontinuity (thick red line). By contrast, the GHCN adjusted data (see Fig. 1 above) do not find any discontinuity in 1941. Instead, they claim that there are discontinuities around 1920, 1930, 1950, 1960, and 1980 … doubtful.
• Finally, the main recurring question is, why do I think the adjustments were made manually rather than by the procedure described by the GHCN? There are a number of totally independent lines of evidence that all lead to my conclusion:
1. It is highly improbability that a station would suddenly start warming at 6 C per century for fifty years, no matter what legitimate adjustment method were used (see Fig. 1).
2. There are no neighboring stations that are sufficiently similar to the Darwin station to be used in the listed GHCN homogenization procedure (see above).
3. The Darwin Zero raw data does not contain visible inhomogeneities (as determined by the GHCN’s own algorithm) other than the 1936-1941 drop (see Fig. 2).
4. There are a number of adjustments to individual years. The listed GHCN method does not make individual year adjustments (see Fig. 1).
5. The “Before” and “After” pictures of the adjustment don’t make any sense at all. Here are those pictures:
Figure 3. Darwin station data before and after GHCN adjustments. Upper panel shows unadjusted Darwin data, lower panel shows the same data after adjustments.
Before the adjustments we had the station Darwin Zero (blue line line with diamonds), along with four other nearby temperature records from Darwin. They all agreed with each other quite closely. Hardly a whisper of dissent among them, only small differences.
While GHCN were making the adjustment, two stations (Unadj 3 and 4, green and purple) vanished. I don’t know why. GHCN says they don’t use records under 20 years in length, which applies to Darwin 4, but Darwin 3 is twenty years in length. In any case, after removing those two series, the remaining three temperature records were then adjusted into submission.
In the “after” picture, Darwin Zero looks like it was adjusted with Sildenafil. Darwin 2 gets bent down almost to match Darwin Zero. Strangely, Darwin 1 is mostly untouched. It loses the low 1967 temperature, which seems odd, and the central section is moved up a little.
Call me crazy, but from where I stand, that looks like an un-adjustment of the data. They take five very similar datasets, throw two away, wrench the remainder apart, and then average them to get back to the “adjusted” value? Seems to me you’d be better off picking any one of the originals, because they all agree with each other.
The reason you adjust is because records don’t agree, not to make them disagree. And in particular, if you apply an adjustment algorithm to nearly identical datasets, the results should be nearly identical as well.
So that’s why I don’t believe the Darwin records were adjusted in the way that GHCN claims. I’m happy to be proven wrong, and I hope that someone from the GHCN shows up to post whatever method that they actually used, the method that could produce such an unusual result.
Until someone can point out that mystery method, however, I maintain that the Darwin Zero record was adjusted manually, and that it is not a coincidence that it shows (highly improbable) warming.
Sponsored IT training links:
Want to pass HP0-J33 at first try? Gets certified 000-377 study material including 199-01 dumps to pass real exam on time.
Discover more from Watts Up With That?
Subscribe to get the latest posts sent to your email.
Neil Gibson (21:52:04) :
Couldn’t agree more. The computer programs (both climate models and the CRU/GISS/GHCN “adjustment” programs) receive no serious investigation at all. There is a whole branch of computer science which develops and uses tools for software Verification and Validation (V&V) and Software Quality Assurance (SQA). These tools are applied to all mission critical software around the world, software for everything from subways to elevators to submarines to airliners to moon shots.
The fact that we are discussing spending many, many billions of dollars based on untested (and in many cases unknown and unexamined) software is a sick joke. We pay more attention to the software running our subways than to global climate models that make unbelievable claims … how bizarre is that?
Earth’s upper atmosphere cooling dramatically
New observations may help detect changes from carbon dioxide increases
http://www.msnbc.msn.com/id/34479085/ns/technology_and_science-space/
Doesn’t anybody understand? Only the past changes. The current temperature always stays the same.
Nick Stokes (22:46:31) :
I am sorry, but I still find it a bit odd that adjustments to temperature balance out. I would expect there to be a bias in one direction or the other. The adjustments are supposedly made to correct data because of spurious readings, not preserve a neutral value in the amount of adjustments. It does not make sense to me that there are just as many stations that read too cold as there are that read too hot, in an equal manner. Why would you expect the readings to be off in both directions in an equal amount? What explanation is there for that?
Re: Nick Stokes (22:46:31)
If your intention is to draw attention to problems with the homogenization paradigm, you have succeeded at least twice with your post. Perhaps I have misunderstood and your intention is to give Willis more material…
@ur momisugly pat (20:10:47) :
I followed your link through to the pdf of Hessell’s paper and note that in addition to his NZ work, he references in the last paragraph a 1975 paper by Tucker finding in Australia “no clear evidence for long term secular warming or cooling in Australasia over the last 50 years.” Tucker, G.B. 1975: Climate: Is Australia’s changing? Search 6 (8); 323-8.
I suspect these two papers are well known to Willis and Anthony (not to mention Salinger), but certainly new to me. Thanks,
David
Dave F
No, the distribution is not perfectly symmetric. The mean trend adjustment is 0.0175 C/decade (upwards). For those >80 year histories, 30 were adjust up more than Darwin. 17 had an adjustment which was bigger but downward.
Wyndham? Wyndham is not only a long way from Darwin, but has a much hotter climate. It is in the Kimberley in north-west Australia. This area is sometimes 5-10 degrees C hotter than Darwin. I went to the Kimberley not so far but inland from Wyndham in October (in the Australian Spring) to a place called the Bungle Bungles NP. When I was there the temperature was well over 40C during the day, while in Darwin it was about 32C at the same time. Once you go inland from Darwin the humidity falls and the temperature climbs. As another example in July (winter), I was in Kakadu NP (say) 300km south-east of Darwin in July. The temperature there was 36C, while it was 30C in Darwin. It is entirely fallacious to modify Darwin temperatures using stations around it, especially those even a short distance inland from the city.
Just wondering if there are other low latitude stations (those in the tropics) in which the unadjusted data might show a similar century long decline in temps. Or even a general unchanged trend. It just seems counterintuitive for temps to be showing a decline to the idea that the planet was recovering from the LIA from the 19th through the 20th century.
Hi Willis and Anthony,
The Quality Assurance question is surely a weapon to be used in the debate.
I will write to my local ( skeptic ) pollie and put it to him. It might be easier to question the complete lack of Quality Assurance by “scientists “than to argue about accuracy especially about software given the garbage I have seen.
The lack of accuracy would naturally come out later.
RE: Nick Stokes (22:46:31) :
You have attempted to divert the argument to make your own point and then try to falsify your own point.
The original post in relation to this temperature set, is relating to the adjustments made at a single station and the fact that they do not conform to either the proposed list of changes that can be made, nor do the adjustments make any sense – hence the question asking how exactly the adjustments were made (as they do not, as i just said conform to the claimed sequence of adjustments).
Instead, you have taken the argument that “well Darwin shows up others show down therefore whats the issue?”. I am unsure of the point you are trying to make? Are you saying that because others adjust downward then the total effect cancels out and therefore the temp record is reliable? In what way, does showing that adjustments were made to other data nulify the fact that there are issues with this station?
The point of this issue, was not to show that the entire database of temperatures have all been adjusted to show warming, rather that this one has, and it does not have adjustments that make sense. In which case, what proportion of the entire dataset contains adjustments that are wrong (either up or down is irrelevent). The fact that the entire response to this station and its issues was to claim that there are an equal number of adjustments to other stations does not excuse the fact that there are issues and is itself, another attempt to mislead and confuse people reading both articles.
YouNews: Giant snowball rolled into street in Madison, WI
Reply: I loved both of these. ~ ctm
This would be a great protest strategy.
Snowmen face down the plows on Gilman Street, Madison, WI
The Great Snowman Wall of Gilman.
Nick Stokes, things aren’t quite as copacetic as Giorgio Gilestro implies.
If you look at the trend of the adjustment over time,, it is far from random. This trend in the data are the explanation for the obviously leptokurtic distribution obtained by Gilestro.
This sort of residual test is a rather elementary one that any of us who actually do data analysis are taught to perform. I’m not surprised Gilestro failed to do it (if only because I don’t know him from Adam), but I’m a bit surprised the GHCN guys didn’t address it though. The high amount of kurtosis itself is also a bit of a warning sign—usually it is an indicator than non-random processes are involved in the observed residuals. (Think about the extreme example of histogramming a sine wave… you’d get a symmetric distribution, but something that is very non-gaussian.)
Nick Stokes (23:24:30) :
The mean trend adjustment is 0.0175 C/decade (upwards).
So there is basically no difference in the trend if you remove all the adjustments, right? It would look the same either way, basically?
David44 (23:22:32) :
Oddly enough, in a much earlier life, I was the number-cruncher for a follow-up study to Brian Tucker’s 1975 paper. Despite the title, we also analysed temperature trends. Again the results were negative. With the better knowledge we have nowadays, that still may have been a correct conclusion for that time. However, that kind of analysis could then only be done much less accurately. GISS temp, which came later, was a huge advance.
Nick Stokes (22:46:31), thanks for your interesting questions.
Not sure what your point is here. What does the average have to do with what happened to Darwin?
I say it is improbable simply because the raw data from Darwin and the sites around it clearly indicate that Darwin is in fact not warming at six degrees per century. I know of no other station on the planet which goes along relatively level for half a century, and then suddenly starts rising at six degrees per century and maintains that trend for a half century.
If you know of one, break it out … but if you don’t know of one, then why don’t you think the claimed Darwin rise is very improbable?
And the fact that the GHCN algorithm may produce other huge adjustments is an indictment of the GHCN method, not of my claim.
In addition, in reading Gilestro (your cite above) you have turned off your skepticism, and that is very dangerous in climate science. Gilestro claims that
Nonsense. Both you and he should know better. 0.2C/decade is 2 degrees per century, and the planet warming trend in the last century was nowhere near that, it was only on the order of 0.06C/decade.
Nor do “the warming and cooling adjustments end up compensating each other” as he claims. First, you would need to grid and area-adjust the temperatures to determine that. It is very sensitive to where the stations are located.
Next, even the raw average which he uses (0.017C/decade) is more than a quarter of the estimated warming, which is is a significant amount.
Given those clearly untrue claims of his, I fear that I can’t trust Giorgio’s numbers or conclusions, and neither should you.
Well … in a word, no. Read the description again, the algorithm doesn’t do that at all. The algorithm looks at the change in the total of the residual sum of squares of the trends on both sides of each candidate year, which is a very different beast which does not depend on the length of sample. I suspect that’s one of the reasons that they use that particular algorithm. You really should run the algorithm, as I have in Fig. 2 above, before drawing conclusions.
Most assuredly. The main reason for still maintaining my statement is that there are no suitable stations within 1500 km. of Darwin to use for the 1920 step, so they could not have used the algorithm that they claimed to use. That algorithm requires 5 stations whose average correlation with Darwin is 0.80. For the 1920 adjustment I can’t find a single station, much less five stations, within 1500 miles that has a correlation greater than 0.43 with Darwin. So I can only conclude that they didn’t use the method they claimed to use.
I list a number of other reasons for maintaining my statement above.
All the best,
w.
Nick Stokes (22:46:31) :
Can “Giorgio’s kind of analysis” also indicate why should I trust ANY record in GHCN database at all – whether adjusted or “UNADJUSTED” means? 🙂 If the algorithm ostensibly creates data for Darwin Airport (prior to 1941) out of “thin air” in adjusted means, where is the guarantee that it didn’t calculate/select/concatenate something funny while doing so called “unadjusted” means, for other stations, too?
Had i not come across this blog, I’d never have the slightest idea how much the climate scientists are massaging the data. I think it’s a scandal – in my layman opinion, it can hardly be called ‘instrumental record’. It’s rather an estimation based on instrument record, of difficult to judge reliability.
John in Australia (23:25:58) :
Aussie John, your intuition and experience is amply supported by the lack of correlation between the Wyndham and Darwin records (first differences of temperatures as used by the GHCN), which is -0.14, in other words, they are not significantly correlated at all …
Which is why Wyndham could not have been used in the GHCN algorithm.
However, it is worth noting that the fact that Wyndham is hotter than Darwin is not the issue. It is that they don’t move in parallel, that there is very little correlation between their swings. If Wyndham warmed when Darwin warmed and cooled when Darwin cooled, we could use it to check Darwin for inhomogeneities. But since it doesn’t, we can’t.
Thank you Willis. This is very interesting.
A thought though.
When talking about climate and Darwin, has any one mentioned Cyclone Tracey.?
Darwin was pretty much destroyed in 1974. It looks like a bit of a jump in the record about then.
Is there any mention of change of instruments or location about then?
According to
http://www.enjoy-darwin.com/cyclone-tracy.html
“Wind speeds of 217 km/h registered at Darwin Airport before the equipment was itself blown away and there are estimates of maximum speeds of up to 300 km/h.”
I suppose this has been said before about Darwin temperature records from 1941 (I have been away and not reading this blog), BUT, Darwin was bombed off the map from Feb 19, 1942, until Nov 12, 1943 by Japanese air raids.
The population withdrew south to Adelaide River and points beyond for years before rebuilding began and any reliable weather service was restored.
Darwin was again smashed on Christmas Day 1974 by an intense cyclone that the local military weather station did not detect.
How could Darwin have adequate and reliable weather records with interruptions and failures such as these?
I just dont understand why they make adjustments to the data in the first place. And why are the adjustments almost always up. Should they not be adjusted down in the citys?.
Leon Brozyna (23:26:37) :
An interesting issue. In general, the tropics has warmed much less than the extratropics. According to the UAH MSU satellite temperatures, there has been no significant warming of the tropics in the thirty years of the satellite record. This inconvenient fact is usually ignored by AGW supporters, who claim that poor tropical countries will bear the brunt of the hypothesized warming.
“RE Dave F (23:15:19) : I am sorry, but I still find it a bit odd that adjustments to temperature balance out. I would expect there to be a bias in one direction or the other.”
It does, to some extent come across odd. The strangness of this standard deviation model is hard to grasp unless explained very simply – something that this standard deviation article fails to (and certainly would not want you to notice). A link to graphical analysis is here http://statpad.wordpress.com/2009/12/12/ghcn-and-adjustment-trends/ this should make it easier to follow the issue.
I will simplify.
A standard deviation of adjustments will show only one thing – how many adjustments were made and in which direction. The thing is, noone is interested in how MANY adjustments were made, or whether the DIRECTION was equal. What is missing is TIME.
There is no plot or analysis on Giorgio’s page relating to WHEN the adjustments were made and unfortunately that is the entire crux of the argument.
If you want an example, take a single station that was basically neutral trend. If you were to apply several cooling adjustments to prior to 1950 (say 0.3 degrees), then several warming adjustments post 1950 for the same amount, you would get a standard deviation of 0, but the temperature trend would have gone from stable to cooling slightly until zomg increeased CO2 + 0.6 degrees increase BURN!!!.
In addition, if you had 2 temperature series one had a trend of 0.3+ the other was stable up until 1960 then had a slight upward trend of say 0.1. we apply 3 adjustments down to the lower end of the first one and get an amplified warming up to 0.6 and then apply the upward adjustments to the higher end of the second one and again, amplified warming – this time showing hockey stick – but again with a 0 standard deviation.
In fact, mathematically if you wanted to attempt to hide this increase, the best way to do it would be to apply corrections that give a 0 standard deviation – this gives it the illusion that there is no issue – and that is all that paper is – an illusion designed to give the semblance of a rebuttal, framed in such a way that makes it look indisputable (unless you actually investigate the data).
In the end, as i said previously the entire argument is moot from the get go. This station has adjustments that appear to be uncalled for, are not explained through GHCN’s own adjustments “manual” and show that the adjusted data may well have errors. The only real way to fix that is to reassess the entire database – not allude to some standard deviation model that just confuses the issue (even though warmists are very good at doing so).
Commenters,
When going down a thread and encountering the words “Nick Stokes” – right away, go up to Edit, Find on this page, enter Nick and then every hit is highlighted. This means that when you see the highlight approaching you can quickly scroll through, thereby saving precious time.