Guest essay by Ed Thurstan
Abstract
The Australian Bureau of Meteorology (BoM) released a new temperature dataset – the “Australian Climate Observations Reference Network – Surface Air Temperature“ (ACORN-SAT) in early 2012, with data to the end of 2011. It is supposedly a ground-breaking daily homogenised dataset.
Ground-breaking it certainly is. The BoM has brought new meaning to the term “temperature inversion”. In “refining” the data, the homogenisation process creates, from apparently normal reported daily raw Max/Min data, homogenised data where the Maximum daily temperature is less than the Minimum temperature for that same day. This error is visible in 70 of the 112 stations included in the ACORN-SAT dataset.
In February 2013 the BoM released an update to the previous ACORN-SAT dataset, adding 2012 to the dataset. But the BoM has not corrected the previous errors a year later in their latest release.
I find that:
· The methods used to create ACORN-SAT are wrong.
· Maintenance of ACORN-SAT as a production database is not practicable and not reproducible – Not by the BoM, and not by its author. It is one person’s view of Australian temperature history.
· The original Raw temperature data used to create the ACORN-SAT dataset has been corrupted by data merging and homogenising, and in particular by the Percentile Matching method employed.
· At least 60% of the entire ACORN-SAT dataset could be corrupted, and is of questionable value.
· BoM’s use of ACORN-SAT as their principal temporal temperature database is unsound. It should not be a regular BoM data product.
· ACORN-SAT in its present form cannot become the output of a “production” BoM system.
Background
In 2012 the Australian Bureau of Meteorology (BoM) released a new Temperature dataset, known as the “Australian Climate Observations Reference Network – Surface Air Temperature” (ACORN-SAT). It replaced the “High Quality” dataset, and the “Reference” dataset that were developed to emulate the NOAA administered US Climate Reference Network (USCRN). The USCRN consists of a number of purpose-built surface weather stations designed to be protected from local effects that might introduce human induced temperature errors. It has only about 10 years of history. Faced with the threat of an audit by the Australian National Audit Office, the BoM has established a 100 year history for 112 stations by synthesising, merging and homogenising station data on a daily basis using a variety of mathematical and statistical methods. This ACORN-SAT database was announced in 2012 with much fanfare. The BoM documentation covering development and implementation of ACORN-SAT is here.
An Expert Panel was appointed to review the product. It was chaired by Ken Matthews AO, and comprised:
· Dr Thomas Peterson, Chief Scientist, National Climatic Data Center, National Oceanic and Atmospheric Administration, United States
· Dr David Wratt, Chief Scientist (Climate), National Institute of Water and Atmospheric Research, New Zealand
· Dr Xiaolan Wang, Research Scientist, Climate Research Division, Environment Canada.
That Panel provided this report to to Dr. R Vertessy, then Deputy Director of the BoM. It concluded, among other favourable comments, that
“ ‘The Panel is convinced that, as the world’s first national-scale homogenised dataset of daily temperatures, the ACORN-SAT dataset will be of great national and international value. We encourage the Bureau to consider the dataset an important long-term national asset.’
ACORN-SAT International Peer Review Panel Report, 2011”
In July 2012 I published a note concerning the quality of the first release of ACORN-SAT data. It appeared in Jo Nova in July 2012 and later in Andrew Bolt. (I think this might be the article Willis Eschenbach referenced in his recent note on ACORN-SAT). John McLean discussed the same subject in the January 27, 2013 edition of Quadrant Online, although incorrectly attributing authorship.
I reported in July 2012 that BoM data merging and homogenisation methods were introducing errors in temperature series that were not apparent in their “raw” data. I said that I had found 954 such errors, but that this was low estimate of the magnitude of the problem. These adjustments were made to large blocks of data, spanning many years. Thus if some were clearly wrong, then whole blocks of data must be suspect. I said in that report that ACORN-SAT should be withdrawn until these errors were corrected.
The second release of ACORN-SAT was in January 2013, to Y/E Dec 31, 2012. My data was downloaded Jan 21, 2013. I performed a detailed comparison between Releases 1 and 2.
This time I reported on April 28, 2013 that
1. All the Release 1 errors to Y/E Dec 2011 are repeated in Release 2.
2. The errors (in a 0900-0900 reporting regime) where today’s maximum is reported as less than tomorrow’s minimum are repeated in Release 2.
3. There was missing data in some Release 1 series. I did not comment on it at the time because it was all at the end of 2011 data, and I thought the problem might be simply late delivery of data. But Release 2 appends a further year of data, and those missing dates are still there, now embedded in the series.
4. Every recorded temperature in the Release 1 dataset was compared with its matching one in the Release 2 dataset to the end of 2011. There were no differences.
I suggested that no effort had been made to review the Release 1 product before appending 2012 data to form the Release 2 database. I said that it appears that the 2012 data is simply raw BoM data, with only rudimentary quality control checks applied.
Purpose of this report
This report attempts to quantify the extent that observed errors have propagated through the entire ACORN-SAT database, given that the observed errors appear in large blocks of data that have the same “adjustments” made by BoM, and which must therefore all be suspect.
Observations
Max < Min Error
The data runs from 1910 to 2012 – 103 years. The 954 errors appear in 95 of those years. The distribution of errors by year ranges from 2.6% occurring in 1916, to 0.1% in 2006, the last year in which the error appears.
The distribution and age of Australian stations until the mid 20th Century was weighted towards cooler higher latitude and coastal locations. As additional stations were added in the hot north of the continent, a gridded average continental temperature was artificially biased towards a warming trend. ACORN-SAT corrected that problem by creating long term time series by “compositing” (merging/homogenising) neighbouring stations to synthesize an early temperature history of the selected stations. This mostly occurred in the 1910-1960 period, and may account for the apparent trend in error rate to 1960.
The improvement in error rate shown after 1990 may well be due to the progressive introduction of automated stations from about 1990.
The errors appear to be scattered over the whole Australian continent, appearing in 70 of the 112 stations in the dataset.
Max today < Min Tomorrow Error
These 353 errors (of which 118 also show the Max < Min error) are distributed differently.
The error occurs in substantially different sets of stations.
Errors virtually cease at the end of the period where there is synthesis of data to fill the 1910-1960 gap.
Analysis of selected stations
Three stations were reviewed in more detail:
· Sydney Observatory – A long history station with major UHI influences
· Cabramurra –A remote high altitude station with a single move
· Cape Otway Lighthouse – A remote single long history station
Two other stations – Wilson’s Promontory and Mackay – were considered for analysis, but rejected as matching raw data was not available for comparison.
The BoM supplied an “Adjustments.xls” workbook to describe the changes made to create the ACORN-SAT database. It is in a diary format. The sections corresponding to the three stations are shown below. Note that in these examples, the worksheets are truncated to delete 8 columns of “stations used”, and occasional one line text notes. The full Adjustments workbook is available here.
The “Raw” data cited comes from here. This data is updated daily by the BoM and has undergone only basic quality control checks before publication.
Sydney Observatory – BoM 66062
This station was established in 1859. It had multiple screen types until the early 1900s when the first Stevenson screen was installed. It has had one physical move of about 200 metres. It is one of Australia’s worst sited stations, with large UHI influences. The BoM notes on adjustments made to create the ACORN-SAT database are:
Looking at the Max < Min errors in this station by date:
The blank Raw temps indicate that these entries in the raw data files were blank. That is, the ACORN data shown for those dates comes from an unknown source, or has been synthesized.
Calculating the differences between ACORN and Raw on an annual basis gives:
The changes specified in the worksheet are visible in the graph:
· Min was raised 0.2 from 1964 back to 1910.
· Max was raised 0.3 from 1984 to 1978
· Max was raised 0.5 from 1946 to 1938
· Max was raised 0.9 from prior to 1917
The BoM “Adjustments” workbook, when it says things like “1 Jan 1938 Max -0.53; 1 Jan 1946 +0.53”, implies to me that the intention is (going back in time) to raise the Max data in that period by 0.53. The result is evident in the above graph of the annualised difference between ACORN Max and Raw Max.
Similarly, for the period between 1920 and 1935 (which encompasses 10 of the 12 errors) the intention appears to be that ACORN Maximums should be raised by about 0.1, and the Minimums raised about 0.2. The distribution of these daily adjustments was examined.
And the distributions of those 16 years of daily adjustments are:
So the ACORN Maximum data has moved by a mean of about 0.1, but far from being a constant adjustment, it has a substantial skewed distribution.
Correspondingly, the Minimums have been shifted up about 0.2, but have been given another skewed distribution.
The adjustments have also had an effect on the DTR, as shown. It gives the mean DTR by month for the period 1910-1982. After 1982 Raw and ACORN data are equal. That is a substantial change in DTR, not discussed by the BoM.
Cape Otway – BoM 90015
This site is at Cape Otway Lighthouse on the Victorian coast at 38.86S. It has moved only slightly since the 1800s. Adjustments have only been required for screen and thermometer issues.
The spike at 1995 is reported in the Metadata as a failure in the AWS unit.
This station has 63 reported cases where ACORN Max < ACORN Min. Of these, 23 appear in the period 1959-1987, which seems to be a stable period from this graph, and will therefore be examined. BoM adjustments are noted to be:
The distribution of the daily adjustments made to the 1959-87 is as follows:
While the annualised adjustment intention appears to have been for the period 1959-87 to
· Lower Maximums by 0.5
· Raise Minimums by 0.1
Some maximums have been lowered by as much as 1.8 when the intended annual mean was 0.5, and some minimums have been raised by as much as 1.1 when the intended annual mean increase was 0.1.
An odd artefact was noticed in 90015 Cape Otway. The ACORN data since 2006 is exactly the same as the Raw data, with one exception. This occurs on Dec 24, 2012 when you see
· Raw data: Max 18.6 Min 16.9
· ACORN data: Max 31.8 Min 16.9
That is a puzzling adjustment.
Cabramurra – BoM 72161
Cabramurra (72161) is an alpine (by Australian standards) station at 1482m. It started as Station 72091 in 1962, which was open until April 1999. The replacement station, 72161 is 400m from the earlier one, and opened in 1997. In the following analysis, 1997, 98 and 1999 data are omitted in analysing ACORN adjustments to avoid issues concerning how the merge was handled. So we can compare the “adjusted”, “refined” ACORN-SAT data with a single source of Raw data, either 72091 or 72161 – about 48 years of data.
In total, Cabramurra exhibits 209 cases of Max < Min. This excludes 3 cases where the error occurs in the 1997-1999 period.
A plot of all data, ACORN and Raw, annualised and including the overlap period shows:
The break between the two contributing stations is evident. Looking at the distribution of adjustments made to the Raw data, excluding the 3 year overlap period, we see the same type of distribution seen in earlier Sydney and Cape Otway cases:
The Max distribution is skewed with the tail on the low side. Rising to meet it, the Min distribution is skewed to the high side, in effect colliding with the tail of the Maximums. That collision creates the errors where adjusted Maximums are less than adjusted Minimums. The adjustments, even if appropriate, are excessive. But by how much ? We don’t know.
The effect of the ACORN adjustments is to lower the DTR over the period 1962-1996 by about 0.9oC.
Wilson’s Promontory – BoM 85096
This station was supposedly opened in 1872, although publically available Raw data starts in 1957.
All 79 cases of Max < Min in the ACORN-SAT data occur before 1949, so an analysis of the distribution of adjustments is not possible.
Mackay – BoM 33119/33297/33047/33046
This station has a complex history, stable only since about 1959. See here for detail.
It exhibits 61 errors where Max < Min. All of these appear in the period 1913 to 1958. – that is, in the period where data compositing/homogenization has occurred. An analysis of distribution of adjustments is therefore not practicable.
“Adjustments” Workbook
This workbook supplied by the BoM purports to record all the adjustments made to Raw data to create ACORN-SAT. It lists, by ACORN Station ID, the changes made to each time series, whether it was Max or Min, a reason, and up to 10 other stations which have been used in some way to the adjustment.
There are 625 line entries in that table. Of these, 100 stations are named where an adjustment was made to the Maximum. There are 103 stations named whose minimums were adjusted. (Remember, there are 112 ACORN-SAT stations). In total there are 109 stations listed as having an adjustment of some sort. Four of these are not ACORN stations. They are 43034, 70014, 84030 and 94069. That leaves 7 ACORN stations that are not listed as having adjustments made. These are:
Four have Max < Min errors as shown, implying that they have been adjusted by some process. The other three listed appear to have not had adjustments, although in each there are ACORN Temps where the corresponding Raw is null, and vice versa.
So the “Adjustments” workbook is an incomplete record of the changes made.
There are 5186 stations listed as contributing to all adjustments. 930 of those are unique, and represent all the stations since 1910 that have contributed in some way to ACORN-SAT.
findings
1. The ACORN-SAT database method of creation is wrong
Paraphrasing the well- known adage:
No matter how elegant and ground breaking the ACORN-SAT daily homogenised database might be, if it produces absurd results, it is wrong.
The current ACORN-SAT database is wrong. It creates absurd data – in particular, records that say on many occasions that the daily Maximum temperature was less than the Minimum for the same day.
2. The ACORN-SAT creation method is impractical for production use
The construction of the ACORN-SAT dataset is very labour intensive. It was built essentially by one person. The methods employed are not a formally designed sequence of procedures. Rather, they are a description of the possible procedures that might be applied to the data in order to achieve the desired daily homogenised dataset. The author acknowledges this.
The Expert Review Panel politely alluded to this in saying
“4. The Panel also encourages the Bureau to more systematically document the processes used, and to be used, in the development and operations of ACORN-SAT. Some aspects of current arrangements for measurement, curation and analysis are non-transparent even internally, and are therefore subject to significant “key persons risk”, as well as a risk of inconsistency over time.”
That seems to imply that the process of creating and maintaining the ACORN-SAT dataset is known to few people in the BoM (possibly only one person), and that his methods are at least in part, subjective. In my view the current ACORN-SAT dataset is not reproducible – not by the BoM, and not by Blair Trewin, the author.
Trewin mentions this in a very expansive report concerning the construction of ACORN-SAT.
“8. Spatial intercomparison of daily data – first iteration
Data were flagged if the temperature anomaly at the candidate site, T, varied from Tint by more than a specified value L. L was set at either 4°C, 5°C or 6°C, based on a subjective assessment of network density and local climatological gradients”
Subjectivity is also evident in his varying choice of neighbouring stations used to check and homogenise data. That is most undesirable, if the dataset is to be used for anything other than academic interest.
In talking about the extensive checks applied to the ACORN-SAT data, Trewin says
“These checks are being used in preference to processing through QMS, as the checks for ACORN-SAT were carried out on all the necessary temperature data by one individual. The combination of these two factors requires specifically designed tools, that allowed the user to make well-informed decisions by using their detailed knowledge of the observing network and local influences contributing to temperature at particular locations, as discussed in section 6.2. This is in contrast with QMS, which has been primarily designed for data managers (and not necessarily climate scientists) to make use of more labour-intensive interactive tools that cover additional observation quantities such as air pressure and wind speed, and would have been too time-consuming to apply to the volumes of data involved in ACORN-SAT.”
While inferring that much of the Raw BoM temperature data (more than 700 stations)is of a less than desirable quality for ACORN-SAT purposes (because it has only been subjected to BoM “QMS” checks) it is curious that Trewin uses mostly those non ACORN-SAT stations to synthesise, adjust and homogenise components of the ACORN-SAT dataset. So the errors in those 700 stations will become ingrained in the 112 “superior” ACORN-SAT 100 year temporal records.
3. It is the compositing and homogenisation applied to daily data that creates absurdities in the output
Trewin fails to consider that the percentile matching method creates a distribution in both Maximum and Minimum data such that the lower tail of the maximums approaches the upper tail of the minimums. On days when the DTR is small, the two tails effectively collide, creating the absurdity that the “refined, homogenised” Maximum is less than the Minimum for that same day.
CAWCR Report 050 (Fawcett, Trewin, Braganza, Smalley, Jovanovic and Jones – March 2012), reviewing the output of ACORN-SAT, also fails to consider that possibility. Their focus appeared to be on comparing ACORN-SAT output with other published datasets concerning AGW.
No one in the BoM appears to have noticed the bad data in the ACORN-SAT output, or else they have ignored it. The BoM has treated the ACORN-SAT database as two independent sets of temperatures. That is, they have ignored the necessary relationship that must exist between daily Max and Min – that Max must be greater than, or equal to Min. They check the inputs for adherence to this rule, but not the outputs.
The University of East Anglia now incorporate ACORN-SAT data in CRUTEM 4.2.0.0. I’m sure it won’t affect their calculation of global temperatures much, but I wonder if they realise they have included data that is constructed by the BoM to be patently wrong. ? I wonder if they would care if it is wrong?
4. How much of the ACORN-SAT might be corrupted ?
Alice Springs (a hot, dry town in the centre of Australia) is typical of all stations that have been “adjusted”. The following graph shows the annual average DTR for 1941 to 2012, and the count of the days in each year where ACORN and Raw DTRs differed. That count is mostly in the 340 to 350 range. The 15 to 25 balance were mostly where there was missing data.
It is evident that there has been massive adjustment of Max, Min or both in almost every daily record, all a consequence of homogenisation and Percentile Matching. These types of adjustment pervade the whole ACORN-SAT dataset up until about 1990.
I estimate that 60% of the ACORN-SAT data is corrupted by the homogenising and Percentile Matching methods employed. That is, most of the data from 1910 to 1960, and much of the data between 1960 and 1990.
Look at Blair Trewin’s statistics:
“All observations flagged by the checks described in the previous section were subject to followup
investigations in order to make a final decision as to whether to accept or reject the value. This was the most time-consuming part of the project as several hundred thousand observations were involved (out of a total of about 7 million observations in the ACORN-SAT data set).
As a result of these follow-up investigations, 18,400 individual observations and 515 blocks of observations of three or more days were flagged as suspect and excluded from further analysis or amended, while 50 blocks of observations were shifted in time (mostly maximum temperatures brought forward by one day, but also including a few cases of months that had been swapped). The bulk of these issues were between 1957 and the early 1970s. Relatively few errors were identified after the early 1970s (and particularly after the mid-1990s), presumably because of improved quality-control procedures over time, whilst most pre-1957 data were only digitised in the last 10 years and therefore also underwent relatively effective quality control.”
5. BoM use of ACORN-SAT for their stated purposes is unsound.
The BoM has recently developed a pre-occupation with “Extreme Events”, reporting these events in “Special Climate Statements” such as this one.
“Special Climate Statement 43 – extreme
heat in January 2013
Fourteen of the 112 stations in the Bureau’s long-term high-quality temperature observation network (ACORN-SAT) set all-time record high maximum temperatures during the 2013 event (Table 1), with a fifteenth (Mount Gambier) equalling its record.
No previous event has resulted in so many records at ACORN-SAT stations, the previous benchmark being set in the January 1939 heatwave, in which eleven ACORNSAT stations set records and three equal records.”
The BoM states the purpose of ACORN-SAT to be:
“… to provide the best possible data set to underlie analyses of variability and change of temperature in Australia, including both analyses of annual and seasonal mean temperatures, and of extremes of temperature and other information derived from daily temperatures.
Documentation and traceability of the data and adjustments at all stages, an increasing priority
as described in Thorne et al. (2011), are also a high priority in the ACORN-SAT data set.”
Warwick Hughes, in correspondence with the BoM (Comment 5 here) was told that:
“…. the (ACORN-SAT) data could be considered ‘official’ at the time of publication, but subject to the qualification that the data are subject to change. In other words, the data are kept current. That is, they are subject to retrospective changes as required. This includes changes to account for additional digitised historical data, additional quality control, and changes, corrections and updates to methodologies. All these occur operationally as required.”
That is, the BoM will use the 100 year ACORN-SAT dataset to highlight record-breaking extremes, while reserving the right to go back in time to adjust earlier data “as required”. That does not sound like good scientific practice.
But perhaps the BoM is having second thoughts about the merits of ACORN-SAT. When Warwick asked:
“Is it your intention to update ACORN-SAT regularly, and if ‘Yes’, at what frequency will those releases occur, and how long after the end of the reporting period will they appear ?”
The reply was:
“Yes.
The Bureau has no official reporting period for ACORN-SAT. The Bureau produces monthly, seasonal and annual summaries, but these are not coupled to specific data set development. The AWAP data are updated daily, including real-time spatial homogenisation, and published publicly on the web the next day.
The ACORN-SAT data set is updated in real-time each day, internally, by the Bureau, and that data is used in reporting as required. The ACORN-SAT data set will be updated publicly online around once a year, though this is subject to various considerations. Complete revisions of the data will be required from time to time, and tracked via version control, to account for changes such as those mentioned in point 1 above. It is impossible to temporally homogenise data in real-time, as opposed to the spatial homogenisation that is performed for AWAP. Only limited temporal homogenisation can be applied after gathering an additional year of data. A full analysis of required temporal homogenisation will be applied to new data at five to ten year intervals. This could be shorter in the event of a significant systematic change affecting the underlying temperature network (e.g. a change in observing practice causing significant data inhomogeneities).
Regards”
The latest 2013 update to ACORN-SAT shows very minor corrections, possibly applied by BoM’s QMS system, rather than ACORN.
Three West Australian stations, Dalwallinu, Bridgetown and Katanning, were flagged to be replaced in the first ACORN-SAT station catalog. They appear to have closed in August 2012, but no replacements have appeared.
The commitment to a five to ten year update interval suggests a lack of enthusiasm for the ACORN-SAT product. Even UEA does better, with a lot more data.
6. ACORN-SAT can cause headaches
I started looking at ACORN-SAT when Blair Trewin, the architect of the product provided this very thorough report CAWCR Report No. 049, this set of Python code to describe the “system” that creates the product, and this description of the adjustments he actually made to the raw data.
My intention was to treat the report as a programming specification, then define an automated system that would create the product.
I came to a halt at this point of Report 049:
6. Data quality control within the ACORN-SAT data set ………………………………30
6.1 Quality control checks used for the ACORN-SAT data set ……………………………… 31
6.2 Follow-up investigations of flagged data ………………………………………………………. 39
6.3 Common errors and data quality problems …………………………………………………… 41
6.4 Treatment of accumulated data…………………………………………………………………… 42
7. Development of homogenised data sets…………………………………………………42
7.1 What particular issues exist for climate data homogenisation in Australia? ………. 43
7.2 The detection of inhomogeneities…………………………………………………………………44
7.3 Adjustment of data to remove inhomogeneities – an overview…………………………47
7.4 The percentile-matching (PM) algorithm ……………………………………………………….49
7.4.1 The overlap case……………………………………………………………………………………………………49
7.4.2 The non-overlap case…………………………………………………………………………………………… 50
7.5 Monthly adjustment method…………………………………………………………………………53
7.6 Evaluation of different adjustment methods……………………………………………………53
7.7 Implementation of data adjustment in the ACORN-SAT data set ……………………..60
7.8 Identification of locations whose extremes were not homogenisable ………………..64
I invite you to study these sections of the report where I bogged down, and failed. You too can share my headache.
Great reporting and Article..
Now I get it, Australia’s “Angry Summer” of 2012-2013 was not that Angry, Not caused by AGW and WAS caused by MAN messing with the Numbers. CARBON TAX fuel..
http://www.climatecentral.org/news/study-shows-global-warmings-role-in-australias-angry-summer-16163
Reblogged this on Rashid's Blog.
The only educated response I can make about the BoM is PMSL!
Thank you Ed – the stridency of ABOM(inables) in this enterprise is because of the unfettered support of money-grubbing government of Australia and the apparent acquiescence and support of the sheep-like majority. Whatever has happened to the wonderfully individualist, blunt-nosed, straight-talking, iconoclastic Ozzies that I met in various parts of the world in earlier decades. I considered this type of Ozzy to be a world resource for preserving sanity. Obviously you are not a part of the herd and I hope your species is still numerous enough to remove Australia from the centre of this climate world idiocy. Oh, and keep accumulating raw data or it will go the way of UEA data and the dodo bird.
Actually, it’s worse than they thought. The data on their website contradicts their MSM statements.
Apparently, “maximum” and “minimum” are difficult concepts that are not easily comprehended by these climate scientologists.
If errors are pointed out and yet they are repeated, then the implication is that they are not errors but intentional changes to the information.
“He uses statistics as a drunken man uses lamp-posts… for support rather than illumination.” – Andrew Lang (1844-1912)
“ABM gets it wrong”
So not much new there, then.
Here is an explanation of why this happens from one of the scientists involved in the preparation of the ACORN-SAT dataset:
http://tamino.wordpress.com/2013/06/29/a-clue-for-willis/#comment-82207
I posted at Tamino’s:
The problem, as I understand it, is they separately adjust the high and low and thanks to the random noise, days with a small diurnal difference have a chance of introducing this error.
The BoM’s work is totally worthless as it has been fundamentally corrupted by warmism.
My real question is ‘Why is the map of Australia upside down?’ Is this a case of Tasmanian triumphialism.
I have worked with the Chair of the Expert Panel that pretty much endorsed what the BOM is doing.
Let’s just say that in his long career as a bureaucrat, he never found a government policy he couldn’t enthusiastically support.
Mmmm…
Why does Australia, inverted, on a map look very like the US?
Why is the coastline of Kent very like the coastline of Caithness?
Inquiring minds would like to know.
Cheers,
Neil
Memo to Tony Abbott & Greg Hunt (incoming Government whenever):
We need to make savings as the country is broke: here are three:
1. disband the ABC
2. disband the execrable ‘Climate Commission’ – and sue its members for scientific fraud
3. disband and privatise the BoM
@Gary Young Pierce: ” … iconoclastic Ozzies … ”
Not all lost fortunately. Kevin Bloody Wilson is still going strong. Reading through Ed’s excellent if painful ACORN-SAT analysis, I had Kevin’s one-liner running through my head:
” …half of what you read is bull and the other half’s all s**t”.
Obviated the need to reach for the pain-killers.
Wot…the BOM incompetent?
You’re not talking about the same BOM who totally failed to predict the Queensland floods a few years back are you?
You know the BOM that had been advising the water authorities there that rainfall was in a state of steady decline so that the water authorities kept Wyvenhoe dam full, then when the unforeseen deluge came the water had to be released drowning 16 people?
That BOM?
ntesdorf
Given that the coriolis effect works in reverse in the SH and that the Antarctic/Southern ocean is a major climate player, I find that studying Australia on an upside down globe rather interesting.
On Thursday July 4 at 8pm Australian Eastern Standard Time, the ABC1TV had their scientific ‘Catalyst’ programme – “Reporter Anja Taylor looks at the domino effect of environmental and atmospheric factors which drive climatic extremes.” I am not a scientist but it made me frustrated and disappointed. Can someone with the knowledge I lack, please look at this show and weigh up their assertions. Everyday busy people listen to this in their lounge rooms and believe it. The site is http://www.abc.net.au/catalyst
Thank you.
Anthony, the title should be “The Australian Bureau of Meteorology Gets it Wrong – Again”. It’s not hard really, I mean 112 devises, in Australia? One device for each ~68,500 square kilometres can, somehow, accurately measure temperature in such varied climatic conditions? Tell ’em they’re dreamin’!
Australia’s climate change parasites exposed. Your greedy proboscises are just about to be ripped out of the public purse (thanks Tony).
Doesn’t BOM state that Sydney Observatory data isn’t suitable for climate data ?
Here’s what BOM does to data: http://kenskingdom.wordpress.com/2010/09/14/the-australian-temperature-record-part-9-an-urban-myth/
Reblogged this on The GOLDEN RULE and commented:
I know we are ‘down-under’ but this map orientation is absurd.
Nevertheless, errors from the Australian BOM have to have been deliberate and create a bias which supports the ‘global warming’ theory. Great science, what?
There are a number of issues here.
I recently looked at the Sydney Observatory sites. There have been more changes than alluded to by Ed. Significantly, there was a switch to automatic weather detection; but I don’t know when. The site is also fully exposed to the east; which is the direction of both the prevailing wind, the approaches to the Harbor Bridge and the city itself and I photographed these aspects. It is also encircled by the “up” ramp of the Cahill expressway which was completed in the 1950’s but now is constantly clogged by traffic. So it has trended hotter anthropologically, but not due to climate warming.
It is now easy to statistically test for step changes caused by, for example, station shifts or measurement issues and adjust for those.
The attribution problem then becomes sensibly lining up detected shifts with known changes. Deducting those effects, trends should then be investigated in the residuals. There are time-lags and time-wise confidence intervals to be aware of, but overall it is not statistically challenging to investigate time-related data.
A final point is that “the debate” ought not be steered by cheer-squads. Remember 95% of everything falls within an unremarkable envelope statistically called “not significantly different to any other value”. Determining that values or trends lie outside that envelope is not trivial. However, after looking carefully at many datasets there exists reasonable grounds to suspect bias due to data issues; and perception bias, due to the use of selective data periods (such as post 1950) in Australian Bureau of Meteorology data.
After all, BoM collects data and adjusts it. It analyses it. It makes its carefully homogenised data available on-line and it runs its own peer-reviewed Journal (Australian Meteorological and Oceanographic Journal). It also operates an expensive marketing and policy branch all funded by the Government it serves.
In the British sense it would be fair tp conclude it is fully independent.
Cheers
Bill
At the risk of doing this again and boring people, here it is again – raw data v ACORN data for Bourke in Jan 1939.
Each daily temp in the ACORN record has been adjusted, some by up to 0.9C.
All temps over 30C have been adjusted down, temps under 30C adjusted up.
It emphasises the point that the ACORN series is totally unworthy of being an official data set.
Bourke 1939
Jan raw ACORN
1st 38.9 38.4
2nd 40 39.1
3rd 42.2 41.9
4th 38.1 37.9
5th 38.9 38.4
6th 41.7 41.5
7th 41.7 41.5
8th 43.4 43
9th 46.1 45.7
10th 48.3 47.9
11th 47.2 46.8
12th 46.2 45.8
13th 45.7 45.3
14th 46.1 45.7
15th 47.2 46.8
16th 46.7 46.3
17th 40 39.1
18th 40.1 39.1
19th 40 39.1
20th 41.9 41.7
21st 42.5 42.1
22nd 44.2 43.8
23rd 36.7 36.5
24th 40.3 39.2
25th 36.6 36.5
26th 29.4 29.5
27th 29.3 29.4
28th 28.8 28.9
29th 30.6 30.5
30th 35.6 35.4
31st 38.6 38.3
Highest daily 48.3 47.9
Lowest daily 28.8 28.9
Monthly mean 40.4 40.03548387
The 1939 heatwave was widespread and killed over 400 people.
But the adjustments make it easier for them to claim ‘an angry summer’ in 2013.
No place had 17 days of over 40C in 2013 like Bourke did in ’39.
In 1896, Bourke had 22 days of +40C – that’s an ‘angry month’.
“Ian George says:
July 5, 2013 at 4:49 am”
Not a bore at all, but can you provide a source?
@Johanna
“I have worked with the Chair of the Expert Panel that pretty much endorsed what the BOM is doing.
Let’s just say that in his long career as a bureaucrat, he never found a government policy he couldn’t enthusiastically support.”
That’s what the AO stands for.
Patrick
Here is the link for the raw data for Bourke, Jan 1939 (BOM data).
http://www.bom.gov.au/jsp/ncc/cdio/weatherData/av?p_nccObsCode=122&p_display_type=dailyDataFile&p_startYear=1939&p_c=-461105304&p_stn_num=048013
This is the ACORN data for Bourke.
You will need to scroll through until you find 19390101.
http://www.bom.gov.au/climate/change/acorn/sat/data/acorn.sat.maxT.048245.daily.txt
I find the entire concept of “adjustments” to raw data suspicious. Raw data clearly don’t always tell the whole truth, but they must nevertheless always be treated with respect. If concerns exist about systematic errors in the raw data, these concerns should be documented and published along with the raw data, and estimated correction factors maybe applied to produce a second, separate data set. However, there can never be a valid reason to mix raw data with corrected or “synthesized” ones and then substitute the mixture for the real thing.
You are right and wrong Michael Palmer. The nature of long-term meteorological data is that it is collected by a number of observers, using instruments that change from time-to-time; measurement scales that also changed (from imperial to metric); housed in shelters that also changed; located at specific sites where conditions may have also changed (trees grow; carparks get sealed etc.), and sometimes specific sites have shifted to somewhere else without data overlap, and sometimes without specific notes about the move. This can all show up in data. For trend analysis this leads to spurious results.
It is very important that raw data is checked for homogeneity through time and adjusted for strange, non-trend impacts. Simply taking a bunch of raw data at face value, and putting a line through it, using say EXCEL, without firstly looking at its properties can lead to serious misinterpretation.
Unfortunately Australia’s Bureau of Meteorology presents numerous interpretations of the same data.
It has raw data, but there is reason to believe data after about 1970 has been pre-processed before release.
Then there is the so-called “High Quality” (post-1910) data (HQ), which allegedly represents a clean-up of the raw data. (HQ data was used to estimate Australia’s average temperature up to about the end of 2011, and this compiled dataset was widely used in publications relating to ‘global warming’.) HQ data may still be available as downloadable files.
HQ data has been superseded by the ACORN-SAT data. (ACORN-SAT is daily ; HQ was monthly (at least the data I’ve seen). Australia’s average temperature is now calculated using ACORN-SAT.
Interestingly, of one takes raw and HQ monthly data for a single station, and compares values with monthly ACORN-SAT data (which needs to be calculated from the daily data stream) there are differences. As you point out out, for a particular station, differences between raw data and HQ data possibly indicate systematic bias (pre-about 1970). Differences also occur between raw, HQ and ACORN-SAT data. I’ve looked at the issue of homogeneity in the raw data for a couple of stations, and found HQ corrections that did not seem to be justified.
There are another 2 sources of data for particular stations, but you need to purchase them. These are the SILO ‘patch-point’ and ‘data-drill’ daily data. I have not tried to compare those data, and I don’t know if they are based on raw, HQ or ACORN-SAT interpretations. Patch-point is filled; data-drill is estimated from climate surfaces.
Obviously if everybody uses the same prepared data, say the ACORN-SAT Australian average temperature series, they are likely to arrive at the same answer. If bias exists in the data, then it will carry through to conclusions of papers written using the series. I am not terribly confident that the last 20 years of RAW data is in fact RAW-RAW, especially data that is collected automatically without backup liquid-in-glass thermometer readings.
Cheers
Bill
Bill, I don’t think there is any disagreement between us at all. I’ve got nothing against data analysis. My only point is that a clear distinction between raw data and processed data must be maintained, and no raw data should ever be replaced with “new and improved” processed data. Your travails in getting to the bottom of things very nicely illustrate my point.
Michael,
I’m with you. I would like to see the original data kept intact, with a set of Date/Temp adjustment vectors stored with it. So the homogenised end result, or what ever you want to call it, is the sum of the lot. I would think more about where instrument calibration adjustments should be. But all that is much easier said then done….. Ed
I have no argument with any of this; I don’t see a “taking of sides”.
I started writing a serious paper about Australia’s temperature records but put it on hold to work on another issue.
It is really the responsibility of a researcher to ‘know their data’ and defend its use. ALL the data I referred to (except SILO) is/was available from BoM straight off the web, although now it seems that HQ data is not being maintained or its being merged into or from ACORN-SAT (I’m not sure which, because I’ve not come back onto the data recently.)
Raw data for current and discontinued sites is all there. One has to trust the post 1970 data because that is all there is. However I have been somewhat cautious about automatic weather station (AWS) data which now forms the backbone of the current datasets. This is because AWS data leaves no paper-trail; it goes straight to Melbourne and may be homogenized on receipt; and it is derived using instruments that measure resistance not temperature. (Resistance is converted into temperature using complex algorithms.)
Without paying for it, it is not possible to obtain liquid-in-glass (LIG) and AWS data for stations where both systems are in use (Canberra; Norfolk Island and Wagga are stations that I’ve visited and that I know record concurrent AWS and LIG data; there may be many more.) I’m not saying the data is sus; I am just cautious about it.
It is a big job to start from scratch with raw data; fill missing cells; and for many stations, merge broken datasets to create a long time series. Then to evaluate that series and compare it with its HQ and now ACORN-SAT equivalent.
The problem as I see it is that if warming is happening, it is represented by very small incremental numbers against highly volatile background noise. It is not hard to analyse linear trend; the difficult part is to be confident about the data, and to strip out non-trend signals (event impacts). If there is a real trend, it will be in the residuals of this process, not in the data themselves. I could provide a simple example of the process.
Without being aware of the pitfalls we run the risk of claiming trends exist when in fact they don’t; or alternatively, not detecting trends that do exist and are in fact real. Hence the need to have a thorough handle on the data being analysed. The least transparent part of the BoM’s data manipulations is in deriving regional and overall average values. It is simply impossible to repeat what the have done as a check against their average values.
Cheers
Bill
Ed, have you numerically analysed what potential impact the erros might have on monthly averages? To be applied to such questions as – could a 0.2C increase in the in summer maximums (that was how much warmer BoM claims summer2012/13 was) be entirely explained by data error?
Ken Stewart compared the BoM trend with UAH for the UAH whole period and found a close match. This kind of analysis is where the rubber meets the road. All data sets have erros, but the important question is, “does error of a few tenths of a percent in the data have a significant impact on averaged results?”
Barry, No – not at monthly level. I have run ACORN data thru a CRUTEM 5×5 area weighted gridder. Then the Max<Min error is not visible. You can fiddle with, add or drop large chunks of data at that level and see very little difference to the Mean continental temp. But at a daily level of course 0.2 would be significant when discussing extremes, although it would be nit-picking, and probably less than the instrument error.
I just don't like the way PM effectively MAKES UP temperatures within the month, from a bunch of neighbouring stations. Why bother ?
Then BoM says they reserve the right to adjust historical Ts for temporal analysis, including extremes. And those adjustments tend to raise minimums and lower maximums.
We are confounding data here.
RAW data, from whatever source has errors associated with instruments (typically very small) and observers (observers may consistently round-up or down for example; or they may always be later than the 9AM time). There are also potential biases introduced for instance, by daylight saving. There are systematic changes, for instance, a station move, or a tree growing. These may or may not impart a signal to the data. Correspondence from the met bureau suggests ‘uncertainties’ for AWS data are +/- 0.5 degrees, which means that for for 2 data to be different, their difference needs to exceed (0.5 + 0.5) = 1 degree. BoM refer to this as an ‘uncertainty’ but I’ve not been able to find out its statistical equivalent – it could be for instance a confidence interval; 1-sigma or 2 sigma and I don’t know on what basis the uncertainty is actually calculated.
Error for an averaged value can be directly calculated from the input values. However, this is not the case for area averaging, or weighted averaging. Here, the area itself presents an “error” component. So Australia’s average temperature, which is the average of many area-averaged values, would have a large error associated with it. No publications relating to the “Angry summer” have discussed errors.
BoM would say values have low uncertainty, but uncertainty is not a parametric error measurement. It can be derived using an “expert opinion” approach, where a bunch of professors feel pretty certain that the answer is right; or it can be derived statistically. IPCC have written on the concept of uncertainty. Even though they sound the same, error and uncertainty are not the same.
Here is what the Bureau said in an email to me:
“The Bureau of Meteorology prepares and maintains multiple temperature analyses that are fit for purpose. As a result, the Bureau reports unhomogenised (but quality controlled) station records shortly after they occur, which is appropriate for the analysis of climate extremes associated with weather events. Temperature data at sites forming part of the Australian Climate Observations Reference Network-Surface Air Temperature (ACORN-SAT) dataset are homogenised following additional quality control. This dataset is used for monitoring long-term climate variability and change in Australia and is described extensively throughout the tabs associated with this page http://www.bom.gov.au/climate/change/acorn-sat/ . Data used for both station records and for the ACORN-SAT dataset are subjected to the Bureau’s observation practices for measuring surface air temperature.”
I personally don’t think there is a conspiracy here. However, in order to interpret it well, its up to the user of data to get a handle on it. I have found that BoM objectively answer sensible questions, and I’m sure they find themselves to be people under pressure. Don’t ask them for an opinion, and they probably wont contest yours.
By the same token, I don’t believe everything that their marketing branch spins out for public consumption. Based on looking at data, I especially don’t believe the last summer was remarkably hotter than any other.
Cheers,
Bill
Can you expand on that, Bill? The implication is that 2012/13 summer was not much hotter than any other summer in the century long record, but I’m not sure that was what you meant? Also, can you estimate the impact of the errors to determine that the claim that 2012/13 was a record-breaker is not supported?
Barry says “can you expand on that Bill?”
My statement was “I especially don’t believe the last summer was remarkably hotter than any other.” A concise answer is not possible.
My belief is based on:
1. Anecdotal evidence. The 1895 to 1903 drought was the worst ever experienced in Australia. There were extensive reports of devastation, heat and bushfires in the press of the time that can now readily be accessed thanks to the National Library of Australia’s TROVE database. Many reports, especially in the Sydney Morning Herald contained data telegraphed from stations all across NSW and beyond. (See The Town and Country Journal January 18th 1896 (page 14) for instance.)
(There were other hot periods of course, but the 1895-1903 heat waves stand out as being particularly severe.)
Broken Hill’s Barrier Miner (January 14th 1896) reported that Mr Russell, Government Astronomer (in Sydney) said, “the heat was due to the heating of the interior plains, through a clear sky, and under such circumstances, the temperature of the soil rises frequently to 140F (60C)” (When the landscape is dry, this sensible heat accumulates. It is then only lost by radiation and transference to the atmosphere.)
Bureau of Meteorology’s Bulletin 43 (1957) presents a review of Drought in Australia from earliest years of settlement to 1955; and whilst there are differences from regions to regions, that drought stands out as extreme. Bulletin 43 reiterates that in January 1896, temperatures of 110 to 123F (43 to 50.5C) were experienced across wide areas; and that in December they were only a little cooler. Dust storms were endemic across NSW and other Colonies as they were in the 1920’s and 1940’s.
During the same period (1896-1903) many major Rivers ceased to flow (Castlereagh; Macquarie; Lachlan; Murrumbidgee; Billabong Creek; Murray; Darling (just to name a few)). Water-trains were in service to supply rural communities. By 1902, Sydney’s water supply was in dire straits when the Cataract and Cordeaux Rivers also ceased to flow. Lake Illawarra was the lowest it had ever been in memory; Lake George dried up (as it has a few times).
In NSW, Stock Department losses were estimated as 21,000 horses; 90,664 cattle and 4,562,930 sheep. In 1903, the Department estimated that the 1902 drought (which was a continuation) caused 16 million sheep to perish. Many people simply walked away from their farms. Queensland and South Australia were reportedly in worse condition than NSW/Vic.
You could find out much more if you are interested; the point being that no droughts since that time has been as hot or as devastating.
2. Data. There has been much data published that indicates that the late 1800’s was hotter than the late 2000’s to early 2013’s but it has yet to be compiled. Data were published in newspapers and bulletins by the Department of Public Instruction (which ran Sydney’s observatory) (some are available at Australia’s National Library; in the Bureau of Meteorology Library and possibly the NSW State Archives.) There were reports aplenty that hundreds of people perished of “heat apoplexy”.
It is true that temperatures were not measured in Stephenson Screens. This would have made a side-by-side difference to daily data. However, some of the differences would have cancelled out by averaging into months; more, by averaging max-min averages into grand monthly averages. (The more numbers, the less influential are the extremes (the law of large numbers of numbers!)
(Sydney’s observatory (and others) was not set up to measure the weather. Its main roles were to set the correct time of day (through planetary observation); and calibration of ships barometers.) Chief Astronomer Henry Chamberlain Russell branched out into research and information, which was pretty unique for the time and a lot of what he wrote, is available.
There were some measurements made of the difference between shade and sun temperature and some values were reported in the Sydney Morning Herald. Elsewhere and later, there were comparisons published of various sensor shelters in Australia and in England. While differences between sun and shade are large; differences between sensor shelters on monthly averaged data is much smaller – in the order of tenths to 1 degree C.
BoM’s SOI (ENSO) goes back to 1876 and if you analyse the monthly series, it is clear that El Ninõ/La Niña sequences are clustered through time. (To observe this you could plot the upper and lower 95-percentile values through time for instance.) Clusters of En Niño months correspond (more or less) with droughts (and vice-versa). The cluster of El Niño months late in the 1800’s and from late 1990 to early 2000 is obvious.
The longest continuous official rainfall dataset seems to be Buckalong Station, south-west of Cooma NSW. Because of its location, it’s a funny record though. It is 6 months longer than Sydney. It shows some interesting step-changes related to droughts, as does Sydney, which corroborates well with anecdotal accounts.
Sydney’s raw temperature record is also interesting but it is contaminated. (A few months ago I visited Sydney Observatory and photographed the met enclosure. The enclosure is encircled by the up-ramp of the Cahill expressway and it is only 50 m or so from the southern breakdown lane of the Bradfield Highway. (Sydney Observatory is more interesting than Greenwich (UK) (which I visited a couple of weeks ago), and is well worth a visit!))
The evidence that urban effects have influenced Sydney’s record is compelling as is evidence that data generally has been fiddled with. Unfortunately, comparisons need to be made on a station-by-station basis, because unless you have archived data yourself, there is little way of checking that “new” data has not been changed. Luckily I have some archive data.
3. The critical question is not about data in general, it is the claims made by BoM relating to the Angry Summer; and in particular the claims about summer temperatures in 2012/13.
You may have missed the importance of Mr Russell’s 1896 statement. (Using sparse data and without resorting to a computer model, 1877 Russell produced Australia’s first weather map.)
If averaged temperature = area times temperature increment; it follows that a small increment multiplied by a large area results in a large number (and vice versa). However, “area” is not about warming, it’s simply a multiplier (that can be juggled about).
(Back in January, someone from BoM (whom I can’t remember) was asked about that and in reply said the unprecedented heat was due largely to the area not the temperature.)
There are two error sources in the estimate. One has to do with the temperature; the other of the area. The average Australian temperature is the average of the sum of a bunch of temperature by area calculations, so the errors get passed along and they are probably larger than the final average temperature estimate.
Another bit of evidence is the comparison of Australia’s averaged temperature with an independent dataset, such as Roy Spencer’s UAH satellite series. I’m working on looking at the monthly data and am not ready to comment, but for the annual series it is suspicious that while Australia’s 2012 average shows a large increase from 2011, the UAH data does not.
(Calculated in degrees; Australia’s averaged temperature for 2010, 2011 and 2012 of 21.86C was just 0.42 degrees higher than the average between 1910 and 1956 (21.44 degrees)). This does not mean anything at all, except that temperature data are also clustered and they can go down as well as up.)
“Records” only stand and should only be claimed if there exists no “reasonable doubt” of their veracity. For the Angry Summer’s case, the combined weight of evidence seems stacked against the claims that have been made about “record heat”.
(P.s. I to would also like to see original data kept intact.)
Cheers
Bill